Image denoising method and device, terminal and storage medium

Abstract

The invention discloses an image denoising method and device, a terminal and a storage medium, which can reduce noise in a smooth region and avoid information blurring and information loss of a boundary region. The method comprises the following steps: setting an image variable to point to a to-be-processed image; setting a preset numerical value and the number of cycles with the value of 1; under the condition that the cycle index is smaller than or equal to a preset numerical value, the following steps are executed: boundary information corresponding to a target point is extracted by adopting a multi-scale image boundary extraction algorithm based on a structure tensor, and the target point is any pixel point in an image pointed by an image variable; constructing an adaptive exponential variational method based on the boundary information; calculating an image pointed by the image variable by adopting a self-adaptive exponential variational method to obtain an optimized image, and modifying the image pointed by the image variable into the optimized image; under the condition that the cycle index is equal to a preset numerical value, determining the optimized image as a target de-noised image, and outputting the target de-noised image; and performing self-adding 1 operation on the cycle times.

Description

Technical Field

[0001] The present application relates to the technical field of image denoising, and more specifically, to an image denoising method, device, terminal and storage medium. Background Art

[0002] In practical applications, images usually contain noise, and the noise is integrated with important structural information in the image, which makes it difficult to distinguish and remove the noise. In the prior art, the total variation method is usually used to remove noise in the image. The total variation method uses the characteristics of structural information in ideal images and noisy images to design an effective image denoising method. The method believes that the structural information in the ideal image is less than the structural information (such as boundaries) in the noisy image, mainly because the noise increases the pseudo-structural information in the image, and this phenomenon is particularly obvious in the smooth area of ​​the image. Therefore, the method uses the minimization process of the regularization term in the total variation method to remove noise in the smooth area. However, this noise removal process will cause information blurring and information loss in the boundary area of ​​the image. Therefore, the existing image denoising method still cannot achieve the image denoising task. Summary of the invention

[0003] The purpose of the embodiments of the present application is to provide an image denoising method, device, terminal and storage medium, which can reduce noise in smooth areas and avoid information blurring and information loss in boundary areas, thereby efficiently achieving image denoising tasks. The embodiments of the present application are mainly implemented through the following technical solutions:

[0004] According to a first aspect of an embodiment of the present application, there is provided an image denoising method, comprising:

[0005] Obtain an image to be processed, and set an image variable to point to the image to be processed;

[0006] Set a predetermined value and a number of cycles, and initialize the number of cycles to 1;

[0007] When the number of cycles is less than or equal to the predetermined value, the following steps are performed:

[0008] A multi-scale image boundary extraction algorithm based on structure tensor is used to extract boundary information corresponding to a target point, wherein the target point is any pixel point in the image pointed to by the image variable;

[0009] Constructing an adaptive exponential variational method based on the boundary information;

[0010] The adaptive exponential variation method is used to calculate the image pointed to by the image variable to obtain an optimized image, and the image pointed to by the image variable is modified to be the optimized image;

[0011] When the number of cycles is equal to the predetermined value, determining the optimized image as a target denoised image, and outputting the target denoised image;

[0012] When the predetermined value is not 1, the number of cycles is incremented by 1.

[0013] According to an embodiment of the present application, the step of extracting boundary information corresponding to the target point using a multi-scale image boundary extraction algorithm based on a structure tensor includes:

[0014] Calculating the structure tensor corresponding to the target point;

[0015] Performing eigenvalue decomposition on the structure tensor to obtain a first eigenvalue and a second eigenvalue;

[0016] Set the first parameter, the second parameter and the third parameter;

[0017] The boundary information is obtained based on the first eigenvalue, the second eigenvalue, the first parameter, the second parameter, and the third parameter.

[0018] According to one embodiment of the present application, the step of calculating the structure tensor corresponding to the target point includes:

[0019] Calculate the first-order partial derivative of the target point in the x direction;

[0020] Calculate the first-order partial derivative of the target point in the y direction;

[0021] Calculate the Gaussian kernel corresponding to the target point;

[0022] A structure tensor corresponding to the target point is calculated based on the first-order partial derivative of the target point in the x-direction, the first-order partial derivative of the target point in the y-direction, and the Gaussian kernel.

[0023] According to one embodiment of the present application, the calculation formula of the structure tensor is:

[0024]

[0025] Among them, K σ is the Gaussian kernel; I is a single-channel image, that is, the image pointed to by the image variable; ω is the target point, is the spatial range of the image pointed to by the image variable, that is, the pixel space of the image pointed to by the image variable or the set of pixel points of the image pointed to by the image variable; J x (ω) is the first-order partial derivative of the target point in the x direction; I y (ω) is the first-order partial derivative of the target point in the y direction; The target point is x The square of the first partial derivative in the direction; is the square of the first-order partial derivative of the target point in the y direction; is the gradient of the target point in the image pointed to by the image variable; for The transpose of ; * is the convolution operation.

[0026] According to an embodiment of the present application, the step of constructing an adaptive exponential variational method based on the boundary information includes:

[0027] Calculating an adaptive index of the target point using the boundary information;

[0028] The adaptive exponential variational method is constructed based on the adaptive exponential and the total variation method.

[0029] According to one embodiment of the present application, the calculation formula of the adaptive exponential variation method is:

[0030]

[0031] Among them, u * represents the target denoised image; arg min represents the minimum value finding function; E p (u) represents an energy function; u represents a possible denoised image, i.e., the optimized image; U represents a set of all possible denoised images, i.e., the set of optimized images; represents the gradient of u; represents the regularization term; p(S(ω)) represents the adaptive index; Y represents the image to be processed, that is, the noisy image; represents the data fidelity term; γ represents the regularization parameter; Represents the spatial range of the image pointed to by the image variable, that is, the pixel space of the image pointed to by the image variable or the set of pixel points of the image pointed to by the image variable.

[0032] According to one embodiment of the present application, the energy function satisfies the following Euler-Lagrange equation:

[0033]

[0034] Wherein, u represents a possible denoised image, that is, the optimized image; E p (u) represents the energy function; Represents E p (u) partial derivative with respect to u; u x represents the first-order partial derivative of u in the x direction; Represents E p (u) About ux The partial derivative of represent Partial derivative in the x direction; u y represents the first-order partial derivative of u in the y direction; Represents E p (u) About u y The partial derivative of represent Partial derivative in the y direction.

[0035] According to a second aspect of the embodiments of the present application, there is provided an image denoising device, comprising:

[0036] The image acquisition module to be processed is used to acquire the image to be processed and set the image variable to point to the image to be processed;

[0037] A setting module, used for setting a predetermined value and a number of cycles, and initializing the number of cycles to 1;

[0038] A loop module, used to perform the following steps when the number of loops is less than or equal to the predetermined value: extract boundary information corresponding to a target point using a multi-scale image boundary extraction algorithm based on a structural tensor, wherein the target point is any pixel point in the image pointed to by the image variable; construct an adaptive exponential variation method based on the boundary information; use the adaptive exponential variation method to calculate the image pointed to by the image variable to obtain an optimized image, and modify the image pointed to by the image variable to the optimized image; when the number of loops is equal to the predetermined value, determine the optimized image as a target denoised image, and output the target denoised image; when the predetermined value is not 1, the number of loops performs a self-increment operation.

[0039] In a third aspect of an embodiment of the present application, a terminal device is provided, including a processor and a memory, the memory being used to store a computer program, the processor being used to call and run the computer program stored in the memory, and executing the steps of the image denoising method described in the first aspect of the embodiment of the present application.

[0040] According to a fourth aspect of an embodiment of the present application, a computer-readable storage medium is provided, wherein the computer-readable storage medium is used to store a computer program, wherein the computer program enables a computer to execute the steps of the image denoising method described in the first aspect of the embodiment of the present application.

[0041] The beneficial effects of the embodiments of the present application include at least:

[0042] The embodiment of the present application uses the boundary information of the image to improve the regularization term of the total variation method, so that the improved method can adaptively process the smooth area and boundary area in the image. Specifically, the embodiment of the present application is to obtain the image to be processed, and set the image variable to point to the image to be processed; set a predetermined value and the number of loops, and initialize the number of loops to 1; when the number of loops is less than or equal to the predetermined value, perform the following steps: use a multi-scale image boundary extraction algorithm based on a structure tensor to extract the boundary information corresponding to the target point, wherein the target point is any pixel point in the image pointed to by the image variable; construct an adaptive exponential variation method based on the boundary information; use the adaptive exponential variation method to calculate the image pointed to by the image variable, obtain an optimized image, and modify the image pointed to by the image variable to the optimized image; when the number of loops is equal to the predetermined value, determine the optimized image as the target denoised image, and output the target denoised image; when the predetermined value is not 1, the number of loops performs a self-increment operation. Therefore, the embodiment of the present application can take into account both the smooth areas and the boundary areas in the image. Compared with the prior art, the embodiment of the present application can not only reduce the noise in the smooth areas of the image, but also avoid information blurring and information loss in the boundary areas, thereby efficiently realizing the image denoising task. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the conventional technology, the drawings required for use in the embodiments or the conventional technology descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0044] Figure 1 A flowchart of the image denoising method of the present application in some embodiments;

[0045] Figure 2 A flowchart of the image denoising method of the present application in some other embodiments;

[0046] Figure 3 is a reference image of the image to be processed in this application;

[0047] Figure 4 is a reference image of the ideal image in this application;

[0048] Figure 5 is a reference image of the target denoised image in this application;

[0049] Figure 6 is a reference image of the target denoised image in this application;

[0050] Figure 7 This is a principle block diagram of the image denoising device of the present application in some embodiments;

[0051] Figure 8 This is a functional block diagram of the terminal device of the present application in some embodiments. DETAILED DESCRIPTION

[0052] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are described in detail below in conjunction with the accompanying drawings. In the following description, many specific details are set forth to facilitate a full understanding of the present application. However, the present application can be implemented in many other ways different from those described herein, and those skilled in the art can make similar improvements without violating the connotation of the present application, so the present application is not limited by the specific embodiments disclosed below.

[0053] It should be noted that the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of this application, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0054] The terms "exemplary" or "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0055] The terms "comprises," "includes," or any other variations thereof are intended to cover a non-exclusive inclusion, for example, a process, method, system, product, or apparatus that comprises a series of steps or elements is not necessarily limited to those steps or elements explicitly listed but may include other steps or elements not explicitly listed or inherent to such process, method, product, or apparatus.

[0056] Unless otherwise defined, all technical and scientific terms used in the specification of this application have the same meaning as those commonly understood by those skilled in the art to which this application belongs. The terms used in the specification of this application are only for the purpose of describing specific embodiments and are not intended to limit this application. The term "and / or" used in the specification of this application includes any and all combinations of one or more related listed items.

[0057] Since the image is affected by various factors during the imaging process, there is usually noise in the acquired image. The formation process of the noisy image can be described as: Among them, X is the ideal image without noise, Y is the acquired noisy image, ε is the noise in the image, is all the pixels in image Y, and ω is one of the pixels in image Y. From the perspective of the formula, the existence of noise is relatively simple. However, in the actual image, the noise is integrated with the important structural information in the image, which makes it difficult to distinguish and remove the noise. At present, the relevant technologies of image denoising mainly include filtering technology based on smoothing operation, frequency domain method based on spatial transformation, and denoising algorithm based on optimization iteration. Specifically:

[0058] The filtering technology based on smoothing operation removes noise by weighted averaging of similar structures. This type of method is most common in image denoising tasks. Initially, the Gaussian filter used the spatial distance between pixels to define the relevant weights, and calculated the estimated values ​​by weighted averaging of neighborhood values, thereby smoothing and removing the noise. Subsequently, in order to avoid the problem of blurred image boundaries caused by this weighting method, some researchers used the similarity between values ​​to define weights. In order to avoid the influence of noise on the calculation of numerical similarity, the weights are designed by the difference of patch areas, thereby avoiding blurred boundaries to a certain extent. Although many researchers have improved the weights of such algorithms, the influence of noise on the accuracy of weight calculation is still a difficult problem.

[0059] The frequency domain method based on spatial transformation analyzes the characteristics of noise through relevant transformations and removes it. Usually, people obtain images in the spatial domain through various methods. Images in the spatial domain can be converted into data in the transform domain through other transformations (such as Fourier transform, discrete cosine transform, etc.). Among them, frequency domain images are a common transform domain data. The significance of noise features in the spatial domain and transform domain is different. Therefore, by using the properties of noise in the transform domain, a reasonable transform domain filter can be designed to effectively remove noise. In order to further improve the denoising effect of the transform domain filtering algorithm, the frequency domain filtering algorithm is usually used in combination with the spatial domain filtering algorithm.

[0060] The denoising algorithm based on optimization iteration uses the denoising result of the previous step to further improve the effect of the next step. Due to the influence of noise, the accuracy of the calculation cannot be guaranteed when the numerical similarity is calculated directly using the noisy image. Therefore, in order to describe the numerical similarity in the ideal image as accurately as possible, using the results of preprocessing and iteration for related calculations is also a common denoising method. The total variation method uses the structural information in the image as the optimization function to gradually remove the noise. This method is not constrained by the type of noise, and the algorithm will not change significantly with the change of the noise type. Therefore, this type of algorithm has good universality.

[0061] However, the total variation method in the denoising algorithm based on optimization iteration will cause information blurring and information loss in the boundary area of ​​the image during the noise removal process. In order to solve this problem, the inventor uses the boundary information of the image to improve the regularization term of the total variation method, so that the improved method can adaptively process the smooth area and boundary area in the image.

[0062] The specific implementation of the present application is further described below in conjunction with the accompanying drawings.

[0063] <Image denoising method>

[0064] like Figure 1 and Figure 2 FIG. 1 is a flowchart of an image denoising method provided in the first aspect of the embodiment of the present application. Figure 1 In the method, the image denoising method comprises:

[0065] S1. Obtain an image to be processed, and set an image variable to point to the image to be processed.

[0066] It should be noted that the image to be processed is a single-channel image. A single-channel image usually refers to a grayscale image or a binary image, in which the color of each pixel is represented by a numerical value, which reflects the brightness information of the point. In a grayscale image, the color of each pixel is represented by an 8-bit or 32-bit numerical value, and this numerical value ranges from 0 to 255, where 0 represents black, 255 represents white, and the intermediate values ​​represent different grayscale levels. This image mode is suitable for scenes that do not require color information, such as black and white photos, document scans, etc. In addition, a binary image is also a type of single-channel image, which has only two color levels: white and black, and is usually used to represent text or simple graphics in an image.

[0067] In the embodiment of the present application, the image to be processed is also a noisy image, which can be referred to Figure 3 shown.

[0068] S2. Set a predetermined value and a number of cycles, and initialize the number of cycles to 1.

[0069] The predetermined value is a positive integer, and for example, the predetermined value is 7 or 12. In other embodiments, the specific value of the predetermined value can be set by those skilled in the art according to actual needs, and the present application embodiment does not further limit this. Figure 2 It is represented as the termination condition N.

[0070] The number of cycles is the number of times steps S4, S5, S6, S7 and S8 are executed.

[0071] S3. When the number of cycles is less than or equal to the predetermined value, execute the following steps S4, S5, S6, S7 and S8.

[0072] S4. Using a multi-scale image boundary extraction algorithm based on a structure tensor to extract boundary information corresponding to a target point, wherein the target point is any pixel point in the image pointed to by the image variable.

[0073] Furthermore, the step of extracting boundary information corresponding to the target point by using a multi-scale image boundary extraction algorithm based on a structure tensor includes:

[0074] S41. Calculate the structure tensor corresponding to the target point.

[0075] Furthermore, the step S41 includes:

[0076] S411. Calculate the first-order partial derivative of the target point in the x direction.

[0077] S412. Calculate the first-order partial derivative of the target point in the y direction.

[0078] S413: Calculate the Gaussian kernel corresponding to the target point.

[0079] S414. Calculate a structure tensor corresponding to the target point based on the first-order partial derivative of the target point in the x-direction, the first-order partial derivative of the target point in the y-direction, and the Gaussian kernel.

[0080] The steps S411 and S412 can be understood as Figure 2 The step of "calculating the first-order partial derivatives of the image to be processed in each direction" in the step S413 can be understood as Figure 2 The step S414 can be understood as Figure 2 The steps in "Compute all structure tensors in the image" in [1].

[0081] Furthermore, the calculation formula of the structure tensor is:

[0082]

[0083] Wherein, Kσ is the Gaussian kernel; I is a single-channel image, that is, the image pointed to by the image variable; ω is the target point, is the spatial range of the image pointed to by the image variable, that is, the pixel space of the image pointed to by the image variable or the set of pixel points of the image pointed to by the image variable; x (ω) is the first-order partial derivative of the target point in the x direction; I y (ω) is the first-order partial derivative of the target point in the y direction; is the square of the first-order partial derivative of the target point in the x direction; is the square of the first-order partial derivative of the target point in the y direction; is the gradient of the target point in the image pointed to by the image variable; for The transpose of ; * is the convolution operation.

[0084] More specifically, the K σ is a Gaussian kernel with standard deviation σ.

[0085] Further, Among them, K σ is the Gaussian kernel, is the effective range of the Gaussian kernel, x∈W is a point in the effective interval of the Gaussian kernel, is the square of the first-order partial derivative of the target point in the x direction.

[0086] It should be understood that the size of W is related to the noise level, and the size of W can be adjusted to reduce the impact of noise on the image.

[0087] It can be seen from the calculation formula of the structure tensor above that the structure tensor uses both gradient information and neighborhood information. The use of neighborhood information and its smoothing operation can reduce the impact of noise to a certain extent.

[0088] It should also be understood that in the embodiments of the present application, it is relatively complicated to directly use the structure tensor to define the boundary of the image. The reason is that the structure tensor provides image information in the form of a matrix, while the commonly used image processing method mainly uses pixel value information to complete the task. Therefore, the calculation formula of the structure tensor cannot be directly applied to the commonly used image processing method. However, the scalar information in the structure tensor can be extracted in the form of dimensional compression. It can be seen from the above definition (that is, the calculation formula of the structure tensor) that the structure tensor is a symmetric matrix. Therefore, the structure tensor is decomposable.

[0089] The multi-scale image boundary extraction algorithm based on the structure tensor proposed in the embodiment of the present application can use the multi-scale information in the structure tensor to reduce the impact of noise on the boundary.

[0090] S42, perform eigenvalue decomposition on the structure tensor to obtain a first eigenvalue and a second eigenvalue. The step S42 can be understood as Figure 2 Follow the steps in "Performing eigenvalue decomposition of the structure tensor" in .

[0091] The calculation formula for performing eigenvalue decomposition on the structure tensor is:

[0092]

[0093] Among them, VUV T represents the eigenvalue decomposition of the structure tensor S(ω), e1 represents the first eigenvalue, e2 represents the second eigenvalue, v1 represents the eigenvector of e1, v2 represents the eigenvector of e2, represents the transpose of v1, Represents the transpose of v2. v1 and v2 form the feature matrix.

[0094] In the smooth area of ​​the image pointed to by the image variable and the boundary area of ​​the image pointed to by the image variable, the eigenvalues ​​also have different values. For example, the eigenvalue of the structure tensor in the smooth area is close to 0, that is, the first eigenvalue and the second eigenvalue corresponding to the smooth area are close to 0.

[0095] In other implementations, the step of extracting boundary information corresponding to the target point using a multi-scale image boundary extraction algorithm based on a structure tensor further includes: saving the first eigenvalue and the second eigenvalue. Figure 2 , then click the "Save Results" step in the .

[0096] S43. Set the first parameter, the second parameter and the third parameter.

[0097] The first parameter is represented by α1, the second parameter is represented by α2, and the third parameter is represented by α3. In other implementations, the representation forms of the first parameter, the second parameter, and the third parameter can be set by those skilled in the art according to actual needs.

[0098] S44, obtaining the boundary information based on the first eigenvalue, the second eigenvalue, the first parameter, the second parameter and the third parameter. Figure 2 Follow the "Formula for Computing Multiscale Image Boundaries Based on the Structure Tensor" step in .

[0099] It should be understood that, from a geometric point of view, there is a corresponding relationship between the structure tensor and the ellipse, because the first eigenvalue and the second eigenvalue may correspond to the two radius lengths of the ellipse, and the other eigenvector corresponds to the two radius directions of the ellipse.

[0100] In order to extract the scalar information in the structure tensor and avoid the loss of matrix information, the calculation formula of the boundary information is designed as follows:

[0101]

[0102] Among them, c iis the multi-scale information corresponding to the structure tensor; α1 is the first parameter; e1 represents the first eigenvalue; e2 represents the second eigenvalue; e1+e2 overall describes the average size information of the structure tensor (ellipse), and the larger value corresponds to the boundary of the image pointed to by the image variable; The boundary properties in the structure tensor are described from the perspective of anisotropy; is the average eigenvalue of the structure tensor; The relationship between the two eigenvalues ​​of the structure tensor is described in the form of a ratio, is the smallest eigenvalue of the structure tensor, is the maximum eigenvalue of the structure tensor; ∈>0, ∈ is used to avoid the meaningless situation where the denominator is equal to 0.

[0103] The calculation formula of the boundary information takes multiple perspectives into consideration, and extracts multi-scale information in the structure tensor from multiple perspectives.

[0104] The multi-scale image boundary extraction algorithm based on structure tensor proposed in the embodiment of the present application utilizes multi-scale information integration to make up for the defect of information error caused by dimensional compression when using scalar information to describe matrix information to a certain extent, thereby improving the accuracy of information extraction and ultimately ensuring that the boundary information of the image can be better restored during the denoising process.

[0105] S5. Construct an adaptive exponential variational method based on the boundary information.

[0106] Furthermore, the step S5 includes:

[0107] S51, using the boundary information to calculate the adaptive index of the target point. Figure 2 Use the image boundaries to compute all adaptive indices.

[0108] Specifically, the calculation formula of the adaptive index is:

[0109] p(S(ω)0=1+e -C(ω) ;

[0110] Among them, p(S(ω)) is the adaptive index, which can also be understood as the exponential term of spatial adaptation; C(ω) is the boundary information, that is, the boundary defined based on the multi-scale information of the structure tensor; e is a natural constant, and its value is 2.718.

[0111] The adaptive index can be adaptively adjusted according to the boundary information.

[0112] Furthermore, the adaptive index satisfies the following condition: 1≤p(S(ω))≤2. Moreover, the boundary region (ie, the edge region) satisfies the condition p(S(ω))→1, while the smooth region satisfies the condition p(S(ω))→2. The symbol “→” indicates a tendency toward.

[0113] S52. Construct the adaptive exponential variational method based on the adaptive exponential and total variation method.

[0114] Specifically, the total variation method is a method for describing the variation of a function or signal within its domain. In image processing, the total variation method is usually defined as the sum of the gray value differences between adjacent pixels in an image, which can be in the horizontal, vertical or diagonal direction. The total variation method achieves image denoising, deblurring and other purposes by minimizing the total variation value of the image.

[0115] The calculation formula of the total variation method is:

[0116]

[0117] Among them, u * represents the final denoised image; Y represents the noisy image; u represents the possible denoised image; U represents the set of all possible denoised images; E(u) represents the energy function; represents a regularization term, which is related to the structural information in the image (i.e., the noisy image or the possible denoised image); represents the gradient of u, u x represents the partial derivative of u in the x direction, u y represents the partial derivative of u in the y direction; γ is a parameter used to balance the regularization term and the fidelity term; represents the fidelity term, which is used to guide the minimization process using the noise image Y; Represents the spatial extent of the noisy image.

[0118] It should be understood that the regularization term is related to the structural information in the image. Because noise increases the pseudo structural information in the image, the denoised image can be obtained by minimizing the energy function E(u). In addition, the process of minimizing the structural information is guided by the intrinsic information of the image to be restored, rather than blindly minimizing it.

[0119] The regularization term of the total variation method Used L 1 Regularization flow. The purpose of image denoising is to remove noise in smooth areas as much as possible while accurately restoring the boundary information of the image. However, from a formal point of view, the total variation method minimizes In the process of smoothing the smooth area and the boundary area, it is not possible to distinguish the smooth area from the boundary area well. Therefore, it is easy to cause the loss of image boundary information when the noise in the smooth area is well smoothed. In order to solve this defect, the present application makes innovative modifications based on the total variation method and proposes an adaptive index

[0120] Variation method.

[0121] The calculation formula of the adaptive exponential variation method is:

[0122]

[0123] Among them, u * represents the target denoised image; arg min represents the minimum value finding function; E p (u) represents an energy function; u represents a possible denoised image, i.e., the optimized image; U represents a set of all possible denoised images, i.e., the set of optimized images; represents the gradient of u; represents the regularization term; p(S(ω)) represents the adaptive index; Y represents the image to be processed, that is, the noisy image; represents the fidelity term; γ represents the regularization parameter; Represents the spatial range of the image pointed to by the image variable, that is, the pixel space of the image pointed to by the image variable or the set of pixel points of the image pointed to by the image variable.

[0124] The target denoised image is the final denoised image.

[0125] The adaptive index is an index item of spatial adaptation. The adaptive index can be reasonably calculated according to whether it is a boundary area or a smooth area.

[0126] is an exponential regularization term, which can automatically adjust the degree of structure restoration and noise smoothing according to the local structure of the image, and can adaptively process edges and flat areas to improve filtering performance. In order to obtain an acceptable restored image, it is ensured that the filtering method can smooth flat areas but retain edge information as much as possible.

[0127] The fidelity term is used to ensure that the optimized u is as close to the original observed data Y as possible.

[0128] The regularization parameter is a weight that balances the regularization term and the fidelity term.

[0129] The design of the adaptive exponential variation method can effectively distinguish the smooth area and the boundary area in the image, and reasonably process the two areas according to the boundary information, thereby achieving effective smoothing of the noise in the smooth area and reasonable preservation of the boundary information, so as to improve the denoising effect of the image.

[0130] S6. Use the adaptive exponential variation method to calculate the image pointed to by the image variable to obtain an optimized image, and modify the image pointed to by the image variable to the optimized image.

[0131] In the embodiment of the present application, the gradient descent method is used to minimize the energy in the adaptive exponential variation method, and E p The necessary condition for (u) to take an extreme value is to satisfy the Euler-Lagrange equation.

[0132] Specifically, the energy function satisfies the following Euler-Lagrange equation:

[0133]

[0134] Wherein, u represents a possible denoised image, that is, the optimized image; E p (u) represents the energy function; Represents E p (u) partial derivative with respect to u; u x represents the first-order partial derivative of u in the x direction; Represents E p (u) About u x The partial derivative of represent Partial derivative in the x direction; u y represents the first-order partial derivative of u in the y direction; Represents E p (u) About u y The partial derivative of represent Partial derivative in the y direction.

[0135] In the embodiment of the present application, the Euler-Lagrange equation is equivalent to Wherein, γ represents the regularization parameter; u represents the possible denoised image, that is, the optimized image; Y represents the image to be processed, that is, the noisy image; div represents the divergence acting on the vector; Represents the gradient of u.

[0136] Next, introduce the time variable t to solve E p The calculation formula of the gradient descent flow of (u) with the introduction of time variable t is:

[0137]

[0138]

[0139]

[0140] Among them, u xx is the second-order partial derivative of u in the x direction; u yy is the second-order partial derivative of u in the y direction; is the square of the first-order partial derivative of u in the x direction; is the square of the first-order partial derivative of u in the y direction; u x is the first-order partial derivative of u in the x direction; u y is the first-order partial derivative of u in the y direction; γ is the regularization parameter; y(x,y,t) is the pixel with coordinates (x,u) on image u at time t; Y(x,y,t) is the pixel with coordinates (x,y) on image Y at time t; is the gradient of image u; p is the regularization term The calculation formula for introducing the time variable t can be understood as Figure 2 The steps of "calculating the second-order partial derivatives of the image" and "updating according to the energy iteration formula to obtain the next iterative image" are as follows.

[0141] It should be understood that in the iterative process of the calculation formula introducing the time variable t, the image boundary area satisfies u(x, y, t) = 0, and the image at the initial moment satisfies u(x, y, 00 = Y(x, y).

[0142] S7. When the number of cycles is equal to the predetermined value, determine the optimized image as a target denoised image, and output the target denoised image.

[0143] S8. When the predetermined value is not 1, the number of cycles is incremented by 1.

[0144] In the above technical scheme, the embodiment of the present application can take into account both the smooth areas and the boundary areas in the image. Compared with the prior art, the embodiment of the present application can not only reduce the noise in the smooth areas of the image, but also avoid information blurring and information loss in the boundary areas, thereby efficiently realizing the image denoising task.

[0145] In other embodiments, the numerical difference between the maximum eigenvalue and the minimum eigenvalue in the embodiment of the present application is It can also be used as another way to describe boundary information in the structure tensor.

[0146] In actual application, the effect of the image denoising method can be referred to the following example:

[0147] refer to Figure 4 , is an ideal image; Figure 3 is a noise image (i.e., the image to be processed); if the predetermined value is set to 7, after 7 iterations, the target denoised image reference Figure 5 After 7 iterations, the removed elements are Figure 5 After 7 iterations, the present application can effectively remove the noise in the image, which is mainly manifested in that the noise in the smooth area is well removed and the boundary information is well preserved.

[0148] If the predetermined value is set to 12, after 12 iterations, the target denoised image reference Figure 6 After 12 iterations, the removed elements are Figure 6 . After 12 iterations, the noise in the smooth area is better removed. Although some boundaries are blurred, the final denoising result is acceptable. In addition, the difference image shows that the present application removes a lot of noise in the image, and the structural information (such as boundaries) in the image is not mistakenly removed.

[0149] In summary, the embodiments of the present application can indeed achieve the image denoising task, and the denoising effect is obvious.

[0150] <Image Denoising Device>

[0151] like Figure 7 , which is a principle block diagram of an image denoising device provided by the second aspect of the embodiment of the present application. Figure 7 In the embodiment, the image denoising device 800 comprises:

[0152] The to-be-processed image acquisition module 801 is used to acquire the to-be-processed image and set the image variable to point to the to-be-processed image;

[0153] A setting module 802 is used to set a predetermined value and a number of cycles, and initialize the number of cycles to 1;

[0154] The loop module 803 is used to perform the following steps when the number of loops is less than or equal to the predetermined value: extract boundary information corresponding to the target point using a multi-scale image boundary extraction algorithm based on a structure tensor, wherein the target point is any pixel point in the image pointed to by the image variable; construct an adaptive exponential variation method based on the boundary information; use the adaptive exponential variation method to calculate the image pointed to by the image variable to obtain an optimized image, and modify the image pointed to by the image variable to the optimized image; when the number of loops is equal to the predetermined value, determine the optimized image as the target denoised image, and output the target denoised image; when the predetermined value is not 1, the number of loops performs a self-increment operation.

[0155] <Terminal Equipment>

[0156] In a third aspect of the embodiments of the present application, a terminal device is provided, wherein the principle block diagram of the terminal device can be as follows: Figure 8 As shown. The terminal device includes a processor, a memory, a network interface, a display screen and a temperature sensor connected via a system bus. The processor is used to provide computing and control capabilities. The memory includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The network interface of the terminal device is used to communicate with an external terminal via a network connection. When the computer program is executed by the processor, an image denoising method is implemented. The display screen can be a liquid crystal display screen or an electronic ink display screen, and the temperature sensor is pre-set inside the terminal device to detect the operating temperature of the internal device.

[0157] Those skilled in the art will understand that Figure 8 The principle block diagram shown in the figure is only a block diagram of a partial structure related to the scheme of the present application, and does not constitute a limitation on the terminal device to which the scheme of the present application is applied. The specific terminal device may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.

[0158] In some embodiments, the terminal device includes a processor and a memory, the memory is used to store a computer program, and the processor is used to call and run the computer program stored in the memory to execute the steps of the image denoising method provided in the first aspect of the above-mentioned embodiment of the present application.

[0159] <Computer Readable Storage Medium>

[0160] According to a fourth aspect of an embodiment of the present application, a computer-readable storage medium is provided, wherein the computer-readable storage medium is used to store a computer program, wherein the computer program enables a computer to execute the steps of the image denoising method described in the first aspect of the embodiment of the present application.

[0161] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0162] The technical features of the above embodiments can be combined without changing the basic principles of the present application. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0163] The above embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the patent application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the scope of patent protection of the present application shall be subject to the attached claims.

Claims

1. An image denoising method, characterized in that: include: Obtain an image to be processed, and set an image variable to point to the image to be processed; Set a predetermined value and a number of cycles, and initialize the number of cycles to 1; When the number of cycles is less than or equal to the predetermined value, the following steps are performed: A multi-scale image boundary extraction algorithm based on structure tensor is used to extract boundary information corresponding to a target point, wherein the target point is any pixel point in the image pointed to by the image variable; Constructing an adaptive exponential variational method based on the boundary information; The adaptive exponential variation method is used to calculate the image pointed to by the image variable to obtain an optimized image, and the image pointed to by the image variable is modified to be the optimized image; When the number of cycles is equal to the predetermined value, determining the optimized image as a target denoised image, and outputting the target denoised image; When the predetermined value is not 1, the number of cycles is incremented by 1.

2. The image denoising method according to claim 1, characterized in that: The step of extracting boundary information corresponding to the target point using a multi-scale image boundary extraction algorithm based on a structure tensor comprises: Calculating the structure tensor corresponding to the target point; Performing eigenvalue decomposition on the structure tensor to obtain a first eigenvalue and a second eigenvalue; Set the first parameter, the second parameter and the third parameter; The boundary information is obtained based on the first eigenvalue, the second eigenvalue, the first parameter, the second parameter, and the third parameter.

3. The image denoising method according to claim 2, characterized in that: The step of calculating the structure tensor corresponding to the target point comprises: Calculate the first-order partial derivative of the target point in the x direction; Calculate the first-order partial derivative of the target point in the y direction; Calculate the Gaussian kernel corresponding to the target point; A structure tensor corresponding to the target point is calculated based on the first-order partial derivative of the target point in the x-direction, the first-order partial derivative of the target point in the y-direction, and the Gaussian kernel.

4. The image denoising method according to claim 3, characterized in that: The calculation formula of the structure tensor is: Among them, K σ is the Gaussian kernel; I is a single-channel image, that is, the image pointed to by the image variable; ω is the target point, is the spatial range of the image pointed to by the image variable, that is, the pixel space of the image pointed to by the image variable or the set of pixel points of the image pointed to by the image variable; x (ω) is the first-order partial derivative of the target point in the x direction; I y (ω) is the first-order partial derivative of the target point in the y direction; is the square of the first-order partial derivative of the target point in the x direction; is the square of the first-order partial derivative of the target point in the y direction; is the gradient of the target point in the image pointed to by the image variable; for The transpose of ; * is the convolution operation.

5. The image denoising method according to claim 1, characterized in that: The step of constructing an adaptive exponential variational method based on the boundary information comprises: Calculating an adaptive index of the target point using the boundary information; The adaptive exponential variational method is constructed based on the adaptive exponential and the total variation method.

6. The image denoising method according to claim 5, characterized in that: The calculation formula of the adaptive exponential variation method is: Among them, u * represents the target denoised image; arg min represents the minimum value finding function; E p (u) represents an energy function; u represents a possible denoised image, i.e., the optimized image; U represents a set of all possible denoised images, i.e., the set of optimized images; represents the gradient of u; represents the regularization term; p(S(ω)) represents the adaptive index; Y represents the image to be processed, that is, the noisy image; represents the data fidelity term; γ represents the regularization parameter; Represents the spatial range of the image pointed to by the image variable, that is, the pixel space of the image pointed to by the image variable or the set of pixel points of the image pointed to by the image variable.

7. The image denoising method according to claim 6, characterized in that: The energy function satisfies the following Euler-Lagrange equation: Wherein, u represents a possible denoised image, that is, the optimized image; E p (u) represents the energy function; Represents E p (u) partial derivative with respect to u; u x represents the first-order partial derivative of u in the x direction; Represents E p (u) About u x The partial derivative of represent Partial derivative in the x direction; u y represents the first-order partial derivative of u in the y direction; Represents E p (u) About u y The partial derivative of represent Partial derivative in the y direction.

8. An image denoising device, characterized in that: include: The image acquisition module to be processed is used to acquire the image to be processed and set the image variable to point to the image to be processed; A setting module, used for setting a predetermined value and a number of cycles, and initializing the number of cycles to 1; A loop module, used to perform the following steps when the number of loops is less than or equal to the predetermined value: extract boundary information corresponding to a target point using a multi-scale image boundary extraction algorithm based on a structural tensor, wherein the target point is any pixel point in the image pointed to by the image variable; construct an adaptive exponential variation method based on the boundary information; use the adaptive exponential variation method to calculate the image pointed to by the image variable to obtain an optimized image, and modify the image pointed to by the image variable to the optimized image; when the number of loops is equal to the predetermined value, determine the optimized image as a target denoised image, and output the target denoised image; when the predetermined value is not 1, the number of loops performs a self-increment operation.

9. A terminal device, characterized in that: include: A processor and a memory, the memory being used to store a computer program, the processor being used to call and run the computer program stored in the memory to execute the steps of the image denoising method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: Used to store a computer program, wherein the computer program enables a computer to execute the steps of the image denoising method according to any one of claims 1 to 7.

Images