Training method, image denoising method and readable storage medium

Abstract

The present invention provides a training method, an image denoising method and a readable storage medium. The training method includes: calculating the training degree of the denoising model based on the feature space loss value, the gradient loss value and the regularization term weightedly, wherein the regularization term is used to characterize the spatial smoothness of the image; judging whether the denoising model has been trained based on the training degree. With such a configuration, model training based on the training degree can take into account the features, gradients and smoothness of the output image, thereby solving the problem in the prior art that while removing noise, a lot of high-frequency detail information originally possessed by the image is erased, resulting in an image that is too smooth or even loses key content.

Description

Technical Field

[0001] The present invention relates to the technical field of image signal processing, and in particular to a training method, an image denoising method and a readable storage medium. Background Art

[0002] For noise-contaminated images, there are two common supervision methods for denoising model training in the prior art.

[0003] Method 1: Calculate the mean absolute error, that is, the average of the absolute values ​​of the differences between the predicted image of the denoising model and the corresponding pixel grayscale values ​​of the target image. Calculate according to the following formula:

[0004]

[0005] Where MAE stands for mean absolute error, and m is the total number of pixels in the predicted image, which is also the total number of pixels in the target image. i Represents the grayscale value of the i-th pixel of the predicted image, y i ' represents the gray value of the i-th pixel of the target image. The pixel numbering rule can be arbitrary as long as it can satisfy the traversal. The pixel numbering rule of the predicted image and the target image is the same.

[0006] Method 2: Calculate the mean square error, which is the average of the squares of the differences between the predicted image of the denoising model and the corresponding pixel grayscale values ​​of the target image. Calculate according to the following formula:

[0007]

[0008] Among them, MSE stands for mean square error, and the other symbols can be understood by referring to the description of method 1.

[0009] The defects of the image denoising model trained based on the above two methods are: based on the pixel space, the generated image is too smooth and loses high-frequency details.

[0010] That is, the existing technology has the problem that while removing noise, a lot of high-frequency detail information originally possessed by the image will be erased, resulting in the image being too smooth or even losing key content. Summary of the invention

[0011] The purpose of the present invention is to provide a training method, an image denoising method and a readable storage medium to solve the problem in the prior art that while removing noise, a lot of high-frequency detail information originally possessed by the image will be erased, resulting in the image being too smooth or even losing key content.

[0012] In order to solve the above technical problems, according to a first aspect of the present invention, a training method is provided, which is applied to a denoising model, and comprises the following steps: training the denoising model based on a training set; obtaining a feature space loss value based on a predicted image and a target image of the denoising model; obtaining a gradient loss value based on the predicted image and the target image; obtaining a regularization term based on the predicted image, wherein the regularization term is used to characterize the spatial smoothness of the image; obtaining a weighted loss value by weighted summation of the feature space loss value, the gradient loss value and the regularization term; obtaining a training degree of the denoising model based on the weighted loss value; and, if the training degree meets a preset condition, taking the denoising model as a training result; otherwise, re-training the denoising model based on the training set.

[0013] Optionally, the step of obtaining a feature space loss value based on the predicted image and the target image of the denoising model specifically includes: obtaining a first spatial feature spectrum of the predicted image based on a spatial feature network; obtaining a second spatial feature spectrum of the target image based on the spatial feature network; and calculating the mean absolute error between the first spatial feature spectrum and the second spatial feature spectrum to obtain the feature space loss value.

[0014] Optionally, the spatial feature network includes a preset number of convolutional layers, and the preset number of layers is greater than 1; the spatial feature network is trained based on the mean absolute error loss; the step of obtaining the first spatial feature spectrum of the predicted image based on the spatial feature network includes: the predicted image is input into the spatial feature network; and the output result of the first convolutional layer of the spatial feature network is fused to obtain the first spatial feature spectrum; the step of obtaining the second spatial feature spectrum of the target image based on the spatial feature network includes: the target image is input into the spatial feature network; and the output result of the first convolutional layer of the spatial feature network is fused to obtain the second spatial feature spectrum.

[0015] Optionally, the spatial feature network includes three convolutional layers, wherein each convolutional layer includes a 10-dimensional feature spectrum.

[0016] Optionally, the fusion method is: calculating an average value of output values ​​at the same position in the feature spectrum to obtain the first spatial feature spectrum or the second spatial feature spectrum.

[0017] Optionally, the step of obtaining the gradient loss value based on the predicted image and the target image specifically includes:

[0018] calculate

[0019] Among them, L Grepresents the gradient loss value, W represents the width of the predicted image, H represents the height of the predicted image, G x (i) Represents G x The gradient value of the i-th pixel, G x (i)' represents G x 'The gradient value of the i-th pixel, G y (i) Represents G y The gradient value of the i-th pixel, G y (i)' represents G y 'The gradient value of the i-th pixel, G x Represents the gradient map of the predicted image along the width direction, G x ' represents the gradient map of the target image along the width direction, G y Represents the gradient map of the predicted image along the height direction, G y ' represents the gradient map of the target image along the height direction, G x , G x ', G y and G y ' are numbered in the same way.

[0020] Optionally, the step of obtaining the regularization term based on the predicted image specifically includes:

[0021] calculate

[0022] Among them, L T represents the regularization term, W represents the width of the predicted image, H represents the height of the predicted image, G x (i) Represents G x The gradient value of the i-th pixel, G y (i) Represents G y The gradient value of the i-th pixel, G x Represents the gradient map of the predicted image along the width direction, G y Represents the gradient map of the predicted image along the height direction, G x and G y The pixels are numbered in the same way.

[0023] Optionally, the preset condition includes: the training degree is less than a preset threshold; or, the preset condition includes: the training degree is a minimum value.

[0024] In order to solve the above technical problem, according to a second aspect of the present invention, an image denoising method is provided, comprising the following steps: processing an image based on a denoising model to obtain a denoised image, wherein the denoising model is obtained based on the above training method.

[0025] A readable storage medium, characterized in that the readable storage medium stores a program, and when the program is run, the above-mentioned training method or the above-mentioned image denoising method is executed.

[0026] Compared with the prior art, the training method, image denoising method and readable storage medium provided by the present invention include: obtaining a feature space loss value based on the predicted image and the target image of the denoising model; obtaining a gradient loss value based on the predicted image and the target image; obtaining a regularization term based on the predicted image, wherein the regularization term is used to characterize the spatial smoothness of the image; obtaining a weighted loss value by weighted summation of the feature space loss value, the gradient loss value and the regularization term; obtaining the training degree of the denoising model based on the weighted loss value, and judging whether the denoising model has been trained based on the training degree. With such a configuration, model training is performed based on the training degree, and the features, gradients and smoothness of the output image can be taken into account, thereby solving the problem in the prior art that while removing noise, a lot of high-frequency detail information originally possessed by the image is erased, resulting in an image that is too smooth or even loses key content. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Those skilled in the art will appreciate that the accompanying drawings are provided for a better understanding of the present invention and do not constitute any limitation on the scope of the present invention.

[0028] Figure 1 is a flow chart of a training method according to an embodiment of the present invention;

[0029] Figure 2 is a schematic diagram of the structure of a spatial feature network according to an embodiment of the present invention;

[0030] Figure 3a is the first test image;

[0031] Figure 3b yes Figure 3a The image shown is obtained after being processed by the image denoising method according to an embodiment of the present invention;

[0032] Figure 3c It is a prior art method - the denoising model trained as a supervised method is Figure 3a The output result obtained after image processing shown;

[0033] Figure 3d The denoising model obtained by training the method 2 of the prior art as a supervised method is Figure 3a The output result obtained after image processing shown;

[0034] Figure 4a is the second test image;

[0035] Figure 4b yes Figure 4a The image shown is obtained after being processed by the image denoising method according to an embodiment of the present invention;

[0036] Figure 4c It is a prior art method - the denoising model trained as a supervised method is Figure 4a The output result obtained after image processing shown;

[0037] Figure 4d The denoising model obtained by training the method 2 of the prior art as a supervised method is Figure 4a The output result after image processing is shown.

[0038] In the accompanying figure: 1-total input; 2-first convolutional layer; 3-second convolutional layer; 4-third convolutional layer; 5-total output; 20-feature spectrum. DETAILED DESCRIPTION

[0039] In order to make the purpose, advantages and features of the present invention clearer, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be noted that the drawings are all in a very simplified form and are not drawn to scale, and are only used to conveniently and clearly assist in explaining the purpose of the embodiments of the present invention. In addition, the structure shown in the drawings is often a part of the actual structure. In particular, the emphasis of each drawing is different, and sometimes different scales are used.

[0040] As used in the present invention, the singular forms "one", "an" and "the" include plural objects, the term "or" is generally used to include the meaning of "and / or", the term "several" is generally used to include the meaning of "at least one", and the term "at least two" is generally used to include the meaning of "two or more". In addition, the terms "first", "second" and "third" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first", "second" and "third" may explicitly or implicitly include one or at least two of the features, "one end" and "the other end" and "the proximal end" and "the distal end" generally refer to two corresponding parts, which include not only the endpoints, and the terms "installed", "connected" and "connected" should be understood in a broad sense, for example, it can be a fixed connection, a detachable connection, or an integral body; it can be a mechanical connection or an electrical connection; it can be directly connected or indirectly connected through an intermediate medium, it can be the internal connection of two elements or the interaction relationship between two elements. In addition, as used in the present invention, an element disposed on another element generally only indicates that there is a connection, coupling, cooperation or transmission relationship between the two elements, and the connection, coupling, cooperation or transmission between the two elements may be direct or indirect through an intermediate element, and cannot be understood as indicating or implying the spatial position relationship between the two elements, that is, one element may be in any orientation such as inside, outside, above, below or on one side of another element, unless otherwise clearly indicated in the content. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0041] The core idea of ​​the present invention is to provide a training method, an image denoising method and a readable storage medium to solve the problem in the prior art that while removing noise, a lot of high-frequency detail information originally possessed by the image will be erased, resulting in the image being too smooth or even losing key content.

[0042] The following description is given with reference to the accompanying drawings.

[0043] Please refer to Figures 1 to 4d ,in, Figure 1 is a flow chart of a training method according to an embodiment of the present invention; Figure 2 is a schematic diagram of the structure of a spatial feature network according to an embodiment of the present invention; Figure 3a is the first test image; Figure 3b yes Figure 3a The image shown is obtained after being processed by the image denoising method according to an embodiment of the present invention; Figure 3c It is a prior art method - the denoising model trained as a supervised method is Figure 3a The output result obtained after image processing shown; Figure 3dThe denoising model obtained by training the method 2 of the prior art as a supervised method is Figure 3a The output result obtained after image processing shown; Figure 4a is the second test image; Figure 4b yes Figure 4a The image shown is obtained after being processed by the image denoising method according to an embodiment of the present invention; Figure 4c It is a prior art method - the denoising model trained as a supervised method is Figure 4a The output result obtained after image processing shown; Figure 4d The denoising model obtained by training the method 2 of the prior art as a supervised method is Figure 4a The output result after image processing is shown.

[0044] like Figure 1 As shown, this embodiment provides a training method, which is applied to a denoising model and includes the following steps:

[0045] S10 trains the denoising model based on the training set;

[0046] S20 obtains a feature space loss value based on the predicted image and the target image of the denoising model;

[0047] S30 obtains a gradient loss value based on the predicted image and the target image;

[0048] S40: obtaining a regularization term based on the predicted image, wherein the regularization term is used to characterize spatial smoothness of the image;

[0049] S50: The feature space loss value, the gradient loss value and the regularization term are weightedly summed to obtain a weighted loss value;

[0050] S60 obtains the training degree of the denoising model based on the weighted loss value.

[0051] S70: If the training degree meets the preset conditions, the denoising model is used as the training result; otherwise, the denoising model is retrained based on the training set.

[0052] In step S50, the weighting coefficient of each weight item can be set according to specific needs and is not limited in this specification. In step S60, the specific expression form of the training degree, including univariate or multivariate, and what kind of correction method is superimposed on the weighted loss value or what kind of calculation is performed can be set according to actual needs. The target image can be understood as an ideal output image. For example, in one embodiment, a clear image is used as the target image, and the above image is superimposed with noise as the input image for model training, and the result output by the current model based on the input image is used as the predicted image.

[0053] The feature space loss value is used to characterize the degree of feature loss of the image, that is, focusing on the local details of the image, the gradient loss value is used to characterize the degree of gradient loss of the image, that is, focusing on the degree of detail loss of the entire image, and the regularization term is used to characterize the spatial smoothness of the image. The weighted calculation result of the three can take into account the characteristics, gradient and smoothness of the output image at the same time, and will not produce the problem of losing image details in the prior art because only the overall smoothness of pixels is pursued during model training, thereby obtaining a better imaging effect.

[0054] The specific calculation method of the feature space loss value, the gradient loss value and the regularization term can be understood by using the content below this description. In other embodiments, other calculation methods can also be set for calculation.

[0055] In an exemplary embodiment, the step of obtaining a feature space loss value based on the predicted image and the target image of the denoising model (i.e., step S20) specifically includes: obtaining a first spatial feature spectrum of the predicted image based on a spatial feature network; obtaining a second spatial feature spectrum of the target image based on the spatial feature network; and calculating the mean absolute error between the first spatial feature spectrum and the second spatial feature spectrum to obtain the feature space loss value. The spatial feature network can be trained according to common knowledge in the art. The calculation method of the mean absolute error can be understood by referring to the relevant description in the background technology.

[0056] Preferably, the spatial feature network includes a preset number of convolutional layers, and the preset number of layers is greater than 1; the spatial feature network is trained based on the mean absolute error loss; the step of obtaining the first spatial feature spectrum of the predicted image based on the spatial feature network includes: the predicted image is input into the spatial feature network; and the first spatial feature spectrum is obtained by fusing the output results of the first convolutional layer 2 of the spatial feature network; the step of obtaining the second spatial feature spectrum of the target image based on the spatial feature network includes: the target image is input into the spatial feature network; and the second spatial feature spectrum is obtained by fusing the output results of the first convolutional layer 2 of the spatial feature network. It should be understood that the total output 5 of the spatial feature network itself is calculated based on the output of the last convolutional layer, and when calculating the spatial feature spectrum, only the output of the first convolutional layer of the spatial feature network is used. With such a configuration, the characteristics of the output result can be retained and the influence of the tendency to smoothness caused by model fitting can be reduced.

[0057] Please refer to Figure 2In one embodiment, the spatial feature network includes three convolution layers, namely, a first convolution layer 2, a second convolution layer 3 and a third convolution layer 4. The total input 1 is sequentially calculated through the first convolution layer 2, the second convolution layer 3 and the third convolution layer 4 to obtain a total output 5. Each convolution layer includes a 10-dimensional feature spectrum 20. In other embodiments, the feature space loss value can also be calculated based on the second convolution layer 3 or other intermediate parameters.

[0058] The fusion method is: calculating the average value of the output values ​​at the same position in the feature spectrum to obtain the first spatial feature spectrum or the second spatial feature spectrum. For example, the output of the feature spectrum is a P*Q matrix, and the calculation result is still a P*Q matrix, and the element in the jth row and kth column of the matrix is ​​the average value of the elements in the jth row and kth column of the feature spectrum of the 1st dimension to the 10th dimension.

[0059] In one embodiment, the step of obtaining the gradient loss value based on the predicted image and the target image (i.e., step S30) specifically includes:

[0060] calculate

[0061] Among them, L G represents the gradient loss value, W represents the width of the predicted image, H represents the height of the predicted image, G x (i) Representative G x The gradient value of the i-th pixel, G x (i)' represents G x 'The gradient value of the i-th pixel, G y (i) Representative G y The gradient value of the i-th pixel, G y (i)' represents G y 'The gradient value of the i-th pixel, G x Represents the gradient map of the predicted image along the width direction, G x ' represents the gradient map of the target image along the width direction, G y Represents the gradient map of the predicted image along the height direction, G y ' represents the gradient map of the target image along the height direction, G x , G x ', G y and G y ' are numbered in the same way. W also represents the width of the target image, because the widths of the two are the same, and H also represents the height of the target image, because the heights of the two are also the same. It should be understood that the measurement units of width and height here are both the number of pixels.

[0062] In one embodiment, the step of obtaining the regularization term based on the predicted image (i.e., step S40) specifically includes:

[0063] calculate

[0064] Among them, L T represents the regularization term, W represents the width of the predicted image, H represents the height of the predicted image, G x (i) Representative G x The gradient value of the i-th pixel, G y (i) Representative G y The gradient value of the i-th pixel, G x Represents the gradient map of the predicted image along the width direction, G y Represents the gradient map of the predicted image along the height direction, G x and G y The pixels are numbered in the same way.

[0065] In step S70, the preset conditions can be set according to actual conditions, for example, the training degree is less than a preset threshold, or when the number of training times reaches an upper limit, the training degree is less than other models that have been trained, that is, the training degree is the minimum value. The "re-" here should be understood as starting training again, but some parameters obtained based on training can be retained, and it does not mean starting training from scratch. In step S10, the creation process of the training set can be set according to common knowledge and will not be described in detail here.

[0066] This embodiment further provides an image denoising method, comprising: processing a denoised image based on a denoising model to obtain a denoised image, wherein the denoising model is trained based on the above-mentioned training method.

[0067] In order to verify the effectiveness of the image denoising method, this embodiment further sets a first test image and a second test image, wherein: Figure 3a The first test image is shown, Figure 3b The result obtained by processing the first test image by the image denoising method is shown. Figure 3c and Figure 3d The results of the control group are shown. Figure 3c The denoising model is processed by training the supervised method introduced in the background technology. Figure 3d The denoising model is processed by training the method 2 introduced in the background technology as a supervision method. Figure 3b to Figure 3d It can be seen from the comparison that the result processed by the prior art loses the image details and looks flat and fuzzy as a whole, while the result processed by this embodiment still retains the image details and has a better effect.

[0068] Figure 4a The second test image is shown, Figure 4b~4d The content can also be understood by referring to the above description. Figure 4b to Figure 4d It can also be seen from the comparison that the processing result of this embodiment retains more image details while ensuring the denoising effect.

[0069] This embodiment further provides a readable storage medium, which stores a program. When the program is run, the above-mentioned training method or the above-mentioned image denoising method is executed.

[0070] Based on the above analysis, it can be seen that the above-mentioned training method, image denoising method and readable storage medium are all conducive to solving the problem in the prior art that while removing noise, a lot of high-frequency detail information originally possessed by the image will be erased, resulting in the image being too smooth or even losing key content; it has beneficial effects.

[0071] In summary, in the training method, image denoising method and readable storage medium provided in this embodiment, the training method includes: obtaining a feature space loss value based on the predicted image and the target image of the denoising model; obtaining a gradient loss value based on the predicted image and the target image; obtaining a regularization term based on the predicted image, wherein the regularization term is used to characterize the spatial smoothness of the image; obtaining a weighted loss value by weighted summation of the feature space loss value, the gradient loss value and the regularization term; obtaining the training degree of the denoising model based on the weighted loss value, and judging whether the denoising model has been trained based on the training degree. With such a configuration, model training is performed based on the training degree, and the features, gradients and smoothness of the output image can be taken into account, thereby solving the problem in the prior art that while removing noise, a lot of high-frequency detail information originally possessed by the image is erased, resulting in an image that is too smooth or even loses key content.

[0072] The above description is only a description of the preferred embodiment of the present invention, and is not intended to limit the scope of the present invention. Any changes or modifications made by a person skilled in the art in the field of the present invention based on the above disclosure shall fall within the scope of protection of the technical solution of the present invention.

Claims

1. A training method, characterized in that: The training method is applied to the denoising model, comprising the following steps: Training the denoising model based on the training set; Obtaining a feature space loss value based on the predicted image and the target image of the denoising model; Obtaining a gradient loss value based on the predicted image and the target image; Obtaining a regularization term based on the predicted image, wherein the regularization term is used to characterize spatial smoothness of the image; The feature space loss value, the gradient loss value and the regularization term are weightedly summed to obtain a weighted loss value; Obtaining the training degree of the denoising model based on the weighted loss value; and, If the training degree meets the preset conditions, the denoising model is used as the training result; otherwise, the denoising model is retrained based on the training set; The step of obtaining a feature space loss value based on the predicted image and the target image of the denoising model specifically includes: Obtaining a first spatial feature spectrum of the predicted image based on a spatial feature network; Obtaining a second spatial feature spectrum of the target image based on the spatial feature network; and, Calculating the mean absolute error between the first spatial characteristic spectrum and the second spatial characteristic spectrum to obtain the characteristic space loss value; The spatial feature network includes a preset number of convolutional layers, where the preset number of layers is greater than 1; the spatial feature network is trained based on mean absolute error loss; The step of obtaining the first spatial feature spectrum of the predicted image based on the spatial feature network comprises: The predicted image is input into the spatial feature network; and, The first spatial feature spectrum is obtained by fusing the output results of the first convolutional layer of the spatial feature network; The step of obtaining a second spatial feature spectrum of the target image based on the spatial feature network comprises: The target image is input into the spatial feature network; and, The second spatial feature spectrum is obtained by fusing the output results of the first convolutional layer of the spatial feature network; The total output of the spatial feature network itself is calculated based on the output of the last convolutional layer, and when calculating the first spatial feature spectrum or the second spatial feature spectrum, only the output of the first convolutional layer of the spatial feature network is used; The spatial feature network includes three convolutional layers, wherein each convolutional layer includes a 10-dimensional feature spectrum.

2. The training method according to claim 1, characterized in that: The fusion method is: calculating an average value of output values ​​at the same position in the feature spectrum to obtain the first spatial feature spectrum or the second spatial feature spectrum.

3. The training method according to claim 1, characterized in that: The step of obtaining the gradient loss value based on the predicted image and the target image specifically includes: calculate Among them, L G represents the gradient loss value, W represents the width of the predicted image, H represents the height of the predicted image, G x (i) Representative G x The gradient value of the i-th pixel, G x (i)' represents G x 'The gradient value of the i-th pixel, G y (i) Representative G y The gradient value of the i-th pixel, G y (i)' represents G y 'The gradient value of the i-th pixel, G x Represents the gradient map of the predicted image along the width direction, G x ' represents the gradient map of the target image along the width direction, G y Represents the gradient map of the predicted image along the height direction, G y ' represents the gradient map of the target image along the height direction, G x , G x ', G y and G y ' are numbered in the same way.

4. The training method according to claim 1, characterized in that: The step of obtaining the regularization term based on the predicted image specifically includes: calculate Among them, L T represents the regularization term, W represents the width of the predicted image, H represents the height of the predicted image, G x (i) Representative G x The gradient value of the i-th pixel, G y (i) Representative G y The gradient value of the i-th pixel, G x Represents the gradient map of the predicted image along the width direction, G y Represents the gradient map of the predicted image along the height direction, G x and G y The pixels are numbered in the same way.

5. The training method according to any one of claims 1 to 4, characterized in that: The preset condition includes: the training degree is less than a preset threshold; or the preset condition includes: the training degree is a minimum value.

6. An image denoising method, characterized in that: The method comprises the following steps: processing an image based on a denoising model to obtain a denoised image, wherein the denoising model is obtained based on the training method according to any one of claims 1 to 5.

7. A readable storage medium, characterized in that: The readable storage medium stores a program, and when the program is run, the training method according to any one of claims 1 to 5 or the image denoising method according to claim 6 is executed.

Images