Image denoising method, device, storage medium and terminal

Abstract

The present invention provides an image denoising method and device, belonging to the field of image processing technology. The method comprises: dividing an external noise-free image and an internal noisy image into multiple image blocks; performing a block matching search on each image block of the external noise-free image to obtain the most similar block group as the noise-free image similar block group; performing a block matching search on each image block of the internal noisy image to obtain the most similar block group as the noisy image similar block group; learning the noise-free image similar block group based on a Gaussian mixture process to construct a Gaussian mixture prior model of the external noise-free image; guiding the internal noisy image to learn the internal noise prior based on the Gaussian mixture prior model, and supplementing the internal dictionary with an external dictionary; implementing denoising based on the internal and external mixed prior dictionaries and structured sparse coding, and aggregating the reconstructed blocks in all block groups to obtain a target image corresponding to the image to be processed. The present invention can improve the image denoising effect and thus improve image quality.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to an image denoising method, device, storage medium and terminal. Background Art

[0002] During the process of data acquisition, quantification, transmission, and storage, various factors, such as the environment, transmission channels, and the equipment itself, inevitably introduce various types of noise into images, severely impacting the accuracy, reliability, and efficiency of subsequent pattern recognition and image interpretation. Image denoising is an essential and critical step in improving image and visual quality, and has broad and meaningful real-world applications in fields such as information retrieval and public safety.

[0003] Most current image denoising algorithms assume that noise follows a uniform white Gaussian noise distribution with known intensity. In reality, due to differences in imaging methods and conditions, image noise is composed of a mixture of various types (such as salt and pepper, Poisson, Gaussian, and multiplicative noise), making it far more complex than uniform white Gaussian noise.

[0004] Furthermore, image noise is dependent on factors such as pixel distribution, object scale, spatial structure, usage frequency, and polarization. It is difficult to describe each type of implicit noise using a simple distribution function, making modeling complex. Inappropriate denoising methods can weaken image features, further complicating image interpretation. Summary of the Invention

[0005] The present invention provides an image denoising method, device, storage medium and terminal, which are used to solve the defects of the existing image denoising technology, such as complex background and difficulty in accurate modeling, and achieve image quality enhancement.

[0006] In a first aspect, the present invention provides an image denoising method, comprising: dividing an external noise-free image and an internal noisy image into multiple image blocks respectively; the internal noisy image is an image to be processed; performing a block matching search on each image block of the external noise-free image to obtain the most similar block group as a noise-free image similar block group; performing a block matching search on each image block of the internal noisy image to obtain the most similar block group as a noisy image similar block group; learning the noise-free image similar block group based on a Gaussian mixture process to construct a Gaussian mixture prior model of the external noise-free image; guiding the internal noisy image to learn the internal noisy prior based on the Gaussian mixture prior model, and supplementing the internal dictionary with an external dictionary; wherein the guidance includes: guiding the Gaussian mixture model element selection and orthogonal dictionary learning of the noise-free image similar block group; realizing denoising based on the internal and external mixed prior dictionaries and structured sparse coding, and aggregating the reconstructed blocks in all block groups to obtain the target image corresponding to the image to be processed.

[0007] According to the image denoising method provided by the present invention, a block matching search is performed on each image block to obtain the most similar block group, including: determining a searchable position of the current reference frame according to a search box with the position of the current image block as the center; wherein the search box is determined according to the gradient of the current image block; obtaining all reference blocks of the same size as the current image block according to the searchable positions that can be traversed according to a preset search box step size; calculating the similarity between all reference blocks and the current image block, and determining a matching block corresponding to the current reference frame from all reference blocks of the current reference frame based on the similarity, so as to respectively determine a matching block group corresponding to the current image block in multiple reference frames.

[0008] According to the image denoising method provided by the present invention, a matching block corresponding to the current reference frame is determined from all reference blocks of the current reference frame based on similarity, including: at least one reference block with the highest similarity among all reference blocks is used as the matching block of the current image block in the current reference frame.

[0009] The image denoising method provided by the present invention further includes: if no matching block exists within the searchable position of the current reference frame, determining the matching block according to an image block supplementation method; wherein the image block supplementation method includes: using a matching block of another reference frame or the current image block as the matching block of the current reference frame; and translating a motion vector of the matching block in the current reference frame based on the adjacent image block closest to the current image block to determine the matching block of the current image block in the current reference frame; and using a matching block whose motion vector is a preset value as the matching block of the current image block.

[0010] According to the image denoising method provided by the present invention, a Gaussian mixture process is used to learn a group of similar blocks of a noise-free image and to construct a Gaussian mixture prior model of an external noise-free image. The method includes: learning external noise-free image blocks based on the Gaussian mixture process, capturing the statistical characteristics of similar blocks in the image, and exploring the model hierarchy, component structure, and parameter relationships; generating shared weights based on a reference matrix and associated parameters; in the process of constructing the Gaussian mixture model, using a low-rank representation to transform a high-dimensional problem into a low-dimensional problem, and using a trace norm prior to construct low-rank characteristics; and using the characteristic matrix of the Gaussian mixture prior model as an external sub-dictionary to guide the learning of an internal sub-dictionary in the next stage.

[0011] According to the image denoising method provided by the present invention, an internal noisy image is guided by a Gaussian mixture prior model to learn an internal noisy prior, and an external dictionary is used to supplement the internal dictionary. The method includes: performing a de-meaning operation on similar block groups of the noisy image; guiding the selection of Gaussian mixture model elements for similar block groups of the noisy image based on the Gaussian mixture prior model, mapping the similar block groups of the noisy image to a Gaussian mixture prior model subspace, and selecting the most appropriate Gaussian mixture model element by estimating the posterior probability; guiding the learning of an orthogonal dictionary for similar block groups of the noisy image based on the Gaussian mixture prior model, assigning all similar block groups of the noisy image to corresponding Gaussian mixture model elements in the Gaussian mixture prior model, learning a mixed orthogonal dictionary from the similar block groups of the noisy image, characterizing the noisy image similar block group prior under the guidance of the corresponding external orthogonal sub-dictionary, and learning an internal sub-dictionary to supplement the external sub-dictionary.

[0012] According to the image denoising method provided by the present invention, denoising is achieved based on an internal and external mixed prior dictionary and structured sparse coding, and reconstructed blocks in all block groups are aggregated to obtain a target image corresponding to the image to be processed. The method includes: constructing an internal and external mixed orthogonal dictionary based on an external sub-dictionary and an internal sub-dictionary, the external sub-dictionary is pre-trained from external noise-free data, and learning the mixed orthogonal dictionary through structured weighted sparse coding to obtain the internal sub-dictionary; alternately updating the sparse coding vector and the internal sub-dictionary until the number of iterations exceeds a preset iteration threshold, and the internal and external mixed orthogonal dictionary is learned; denoising is performed based on the internal and external mixed prior dictionary and structured sparse coding, and multiple iterative optimization is performed using an iterative regularization strategy; similar block groups of the noisy image and the current block group are aggregated to obtain an aggregation result, until all image blocks of the image to be processed are aggregated, and the target image is determined based on the aggregation result.

[0013] In a second aspect, the present invention further provides an image denoising device, comprising:

[0014] An image block module is used to divide the external noise-free image and the internal noisy image into multiple image blocks respectively; the internal noisy image is the image to be processed;

[0015] A similar block group determination module is used to perform a block matching search on each image block of the external noise-free image to obtain the most similar block group as the noise-free image similar block group; and to perform a block matching search on each image block of the internal noisy image to obtain the most similar block group as the noisy image similar block group;

[0016] An external prior learning module is used to learn similar block groups of noise-free images based on a Gaussian mixture process and construct a Gaussian mixture prior model of external noise-free images;

[0017] An internal prior learning module is configured to guide the learning of an internal noisy prior for an internal noisy image based on a Gaussian mixture prior model, and to supplement the internal dictionary with an external dictionary; wherein the guidance includes guiding the selection of Gaussian mixture model components for similar block groups of the noise-free image and orthogonal dictionary learning;

[0018] The image denoising module is used to implement denoising based on internal and external mixed prior dictionaries and structured sparse coding, and aggregate the reconstructed blocks in all block groups to obtain the target image corresponding to the image to be processed.

[0019] In a third aspect, the present invention provides a terminal comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of any of the above-described image denoising methods are implemented.

[0020] In a fourth aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any of the above-described image denoising methods.

[0021] The image denoising method, device, storage medium and terminal provided by the present invention convert the denoising problem into a learning problem to be solved. It does not require the use of the complex relationship between noise and image features. Instead, it is based on a low-rank Gaussian mixture process, hybrid dictionary learning and structured weighted sparse coding. It simultaneously utilizes external noise-free image and internal noisy image information, uses noise-free prior to guide noisy prior learning, and uses an external dictionary to supplement the internal dictionary, thereby achieving effective image denoising without the need to predict the noise type and intensity. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions in the present invention or the prior art, a brief introduction is given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0023] Figure 1 1 is a flow chart of the image denoising method provided by the present invention;

[0024] Figure 2 is a schematic diagram of block matching provided by the present invention;

[0025] Figure 3 Schematic diagram of the structure of the image denoising device provided by the present invention;

[0026] Figure 4 1 is a schematic diagram comparing denoising results of different image denoising algorithms provided by the present invention;

[0027] Figure 5Schematic diagram of PSNR comparison results of different denoising algorithms provided by the present invention;

[0028] Figure 6 2 is a schematic diagram showing the comparison results of the running time of different denoising algorithms provided by the present invention;

[0029] Figure 7 It is a structural diagram of the terminal provided by the present invention. DETAILED DESCRIPTION

[0030] To make the objectives, technical solutions, and advantages of the present invention more clear, the technical solutions of the present invention will be clearly and completely described below in conjunction with the accompanying drawings. Obviously, the embodiments described are only some of the embodiments of the present invention, not all of them. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0031] It should be noted that, in the description of the embodiments of the present invention, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "include a ..." do not exclude the presence of other identical elements in the process, method, article or device comprising the elements. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to the specific circumstances.

[0032] The terms "first," "second," and the like in this application are used to distinguish similar objects, and are not used to describe a particular order or precedence. It should be understood that the terms used in this manner are interchangeable where appropriate, so that the embodiments of this application can be implemented in orders other than those illustrated or described herein. Furthermore, the objects distinguished by "first," "second," and the like generally refer to a class of objects and do not limit the number of objects. For example, the first object may be one or more.

[0033] The following combination Figure 1-Figure 7 The image denoising method and apparatus provided by the embodiments of the present invention are described.

[0034] Figure 1 FIG. 1 is a flow chart of the image denoising method provided by the present invention, as shown in FIG. Figure 1 As shown, including but not limited to the following steps:

[0035] Step 101: Divide the external noise-free image and the internal noisy image into multiple image blocks respectively.

[0036] In the method proposed in the present invention, the internal and external mixed prior is composed of an internal noisy prior and an external noise-free prior. The external noise-free image and the internal noisy image have no special meanings and refer to a noise-free image and a noisy image, respectively.

[0037] Among them, the internal noisy image is the image to be processed; the image to be processed can be an image obtained by cameras or other image acquisition devices installed in various places such as roads, communities, airports / train stations, large venues, etc.

[0038] Step 102: Perform a block matching search on each image block of the external noise-free image to obtain the most similar block group as the noise-free image similar block group; perform a block matching search on each image block of the internal noisy image to obtain the most similar block group as the noisy image similar block group.

[0039] It can be understood that the matching search process for the external noise-free image and the internal noisy image is the same.

[0040] Optionally, step 102 specifically includes the following steps:

[0041] Taking the position of the current image block as the center, a searchable position of the current reference frame is determined according to a search box; wherein the search box is determined according to the gradient of the current image block.

[0042] Obtain all reference blocks of the same size as the current image block from the searchable positions that can be traversed according to the preset search box step size;

[0043] Calculating similarities between all reference blocks and the current image block, and determining a matching block corresponding to the current reference frame from all reference blocks of the current reference frame according to the similarities, so as to respectively determine a matching block group corresponding to the current image block in multiple reference frames;

[0044] At least one reference block with the highest similarity among all reference blocks is used as a matching block of the current image block in the current reference frame.

[0045] Optionally, if there is no matching block within the searchable position of the current reference frame, the matching block is determined according to an image block supplementation method.

[0046] The image block supplementation method includes: using a matching block of another reference frame or a current image block as a matching block of the current reference frame; and

[0047] According to the adjacent image block closest to the current image block, the motion vector of the matching block is translated in the current reference frame to determine the matching block of the current image block in the current reference frame; and the matching block whose motion vector is a preset value is used as the matching block of the current image block.

[0048] The following is a further explanation of the process of searching for similar image blocks by block matching:

[0049] Assume that the image has N pixels, the search box size is W×W, the neighborhood window size is p×p, the neighborhood window slides in the search box, and the pixel weight is determined according to the similarity between neighbors. Figure 2 is a schematic diagram of block matching provided by the present invention, such as Figure 2 As shown, the large window (purple border) is i(x i ,y i ) is the center of the search box, and the two green windows (red borders) represent the search results of i(x i ,y i ), j(x j ,y j ) is the center of the neighborhood window N i 、N j , where j(x j ,y j ) is the center of the neighborhood window N j Slide in the search box and calculate the neighborhood window N i 、N j The similarity between them is weighted value w(i,j).

[0050] Let the noisy image be I:

[0051] I = {I(x,y)|(x,y)∈Ω};

[0052] Where x and y are the pixel coordinates on I.

[0053] Use the weighted average of all pixels in image I as the pixel i(x i ,y i ) at the valuation I'(x i ,y i ):

[0054]

[0055] Among them, i(x i ,y i ) and j(x j ,y j ) depends on the similarity between them, which is determined by i(x i ,y i )、j(x j ,y j ) as the center of the rectangular neighborhood N i 、N j Distance between || N i -N j || 2Decision. Use Euclidean distance to measure the two image blocks N i 、N j The similarity between w(i,j):

[0056]

[0057] Among them, Z(i) represents the normalization factor, i(x i ,y i ) and j(x j ,y j ) represent the rectangular neighborhood N i 、N j The center pixel of , h is the smoothing parameter, S(i,j) is the pixel neighborhood N i and N j The weighted Euclidean distance between them is calculated as follows:

[0058]

[0059] Where a is the standard deviation of the Gaussian kernel.

[0060] Optionally, the Euclidean distance S(i,j) is used to compare the pixel neighborhood N i and N j Similarity between:

[0061]

[0062] Among them, I i (g,m) and I j (g,m) represent the pixel neighborhood N i and N j The corresponding pixels in , g and m represent the positions of the image block moving up, down, left and right in the search box, respectively. The search box size is W×W.

[0063] Optionally, the present invention further explains image block group extraction and block group mean removal.

[0064] There are often many repeated local patterns and similar image patches in an image. A patch group (PG) is formed by grouping similar local image patches.

[0065] For example, a noise-free image is divided into overlapping image blocks (size is p×p). For any image block, a search box of size W×W is used to search for the image block and its M most similar blocks, which is called block matching. The image block group composed of these M similar blocks is represented as in, Represents the block vector, and the mean vector of the image block group is To highlight individual characteristics and differences, the mean of the block vector is removed. The image block group after removing the mean is defined as:

[0066]

[0067] Step 103: Based on the Gaussian mixture process, similar block groups of the noise-free image are learned to construct a Gaussian mixture prior model of the external noise-free image.

[0068] Optionally, the group of similar blocks of the noise-free image searched in step 102 is modeled, and a Gaussian mixture prior model of the external noise-free image is constructed based on a Gaussian mixture process to capture the statistical characteristics of similar blocks in the image and explore the model hierarchy, component structure, and parameter relationship;

[0069] Optionally, a shared weight parameter is generated according to a reference matrix, an associated parameter, and a const normalization term, and a conjugate prior of a Gaussian distribution is introduced as a reference distribution;

[0070] w t ={μ t ,A t} represents the parameters of subspace t, where μ t and A t Denote the mean vector and dictionary matrix of subspace t respectively. Set the parameters {μ t ,A t}:

[0071] μ t ~G(·|μ0,Σ0)

[0072]

[0073] Among them, G represents the Gaussian mixture process, and const represents the normalization term. t,k ={μ t,k ,∑ t,k}Determine the Gaussian mixture model elements of subspace t, μ t,k and ∑ t,k Represent the mean vector and covariance matrix of each element respectively, and D represents μ t,k The conjugate prior of the Gaussian distribution is introduced as the reference distribution as follows:

[0074] μ t,k ~G(·|ε0,Ω0)

[0075] ∑ t,k ~iWishart(v0,B0)

[0076] Where d∈{1,...,D}, iWishart(v0,B0) represents the inverse gamma distribution, v0 and B0 represent the shape parameter and scale matrix, respectively.

[0077] Optionally, to reduce the computational complexity of the model, during the construction of the Gaussian mixture model, a low-rank representation is used to transform the high-dimensional problem into a low-dimensional problem, and the trace norm prior is used to construct the low-rank feature:

[0078]

[0079] Where Tr represents the trace of the matrix, A is a d×d dimensional matrix,

[0080] Optionally, the Gaussian mixture process includes one or more of a Chinese restaurant process, a folded stick process, and a Pollyon scheme.

[0081] Assume that L block groups are extracted from a set of noise-free images, and the lth block group is:

[0082]

[0083] Among them, M represents the number of similar image blocks. There is abundant non-local similarity information in , so the problem is transformed into how to We learn a Gaussian mixture model to model the external priors of the block groups.

[0084] Optionally, based on a Gaussian mixture model, we aim to extract the L noise-free image blocks from the image. Learning the prior model {N(μ k ,∑ k )}, while requiring block groups M similar blocks Belong to the same Gaussian mixture model element. Calculated by the following formula Likelihood:

[0085]

[0086] Assuming that all block groups are sampled independently, the total target likelihood function is Take its logarithm and maximize the objective function of Gaussian mixture learning:

[0087]

[0088] In Gaussian mixture learning, the implicit variable {Δ lk |l=1,...,L;k=1,...,K}Optimization formula If the block group Belongs to the kth element, then Δ lk =1, otherwise, Δ lk =0.

[0089] Optionally, use the Expectation Maximization (EM) algorithm to optimize the formula The EM algorithm mainly includes: expectation step (E step) and maximization step (M step).

[0090] The EM algorithm alternates between the E step and the M step:

[0091] 1. Step E, introduce the posterior probability γ lk , indicating that the kth component generates a block Probability of:

[0092]

[0093] 2. Randomly initialize γ lk ;

[0094] 3.M step: based on the current γ lk Estimated parameters. Each block group in M ​​steps Therefore:

[0095]

[0096] in,

[0097] 4.E step: Based on the current parameter values ​​and formula Re-estimate γ lk ;

[0098] 5. Calculate the log-likelihood of the image (Eq. ) and check the convergence of the log-likelihood. If it does not converge, go back to step 3.

[0099] Step 104: Guide the internal noisy image to learn the internal noise prior based on the Gaussian mixture prior model, and use the external dictionary to supplement the internal dictionary. The guidance includes guiding the selection of Gaussian mixture model components and orthogonal dictionary learning for similar blocks of the noise-free image.

[0100] Optionally, step 104 in the present invention specifically includes the following steps:

[0101] Perform a de-meaning operation on similar block groups of noisy images;

[0102] The Gaussian mixture model components of the noisy image similar block group are selected according to the Gaussian mixture prior model, the noisy image similar block group is mapped to the Gaussian mixture prior model subspace, and the most appropriate Gaussian mixture model components are selected by estimating the posterior probability;

[0103] The orthogonal dictionary learning of noisy image similar block groups is guided by the Gaussian mixture prior model. All noisy image similar block groups are assigned to the corresponding Gaussian mixture model elements in the Gaussian mixture prior model. A mixed orthogonal dictionary is learned from the noisy image similar block groups. The noisy image similar block group prior is characterized under the guidance of the corresponding external orthogonal sub-dictionary, and the internal sub-dictionary is learned to supplement the external sub-dictionary.

[0104] The term "Gaussian mixture model components" generally refers to the individual Gaussian distributions that make up the entire mixture model. Each such Gaussian distribution is called an "element" or "component." These components work together to describe the overall distribution of the data.

[0105] The above steps are further explained below.

[0106] Optionally, the following process is used to select Gaussian mixture model components:

[0107] Given a noisy image y, extract N image blocks (noisy blocks). Similar to the external prior learning stage, for the nth noisy block (n = 1, 2, ..., N), search for the M most similar image blocks through block matching to construct a noisy image similar block group Y n ={y n1 ,...,y nM}. The mean vector of this block group is μ n , the block vector after removing the mean is The similar block group of the noisy image with the mean removed is remember in is the corresponding noise-free block group, and V is the noise. At this point, the problem is transformed into how to use the learned external Gaussian mixture prior to Mid-term recovery Because the mean vector of the noise V is close to the zero vector, The mean μ y is very close to The mean of .

[0108] Gaussian mixture external prior model The subspace of the noise-free block group can be characterized. Select appropriate Gaussian mixture model components from the Gaussian mixture. Therefore, each noisy block group Mapping to Gaussian mixture model In the subspace, the posterior probability is estimated as Select the most appropriate Gaussian mixture model component:

[0109]

[0110] Wherein, k=1,...,K, k is the number of Gaussian mixture model elements.

[0111] Pair Taking the logarithm, we get:

[0112]

[0113] Among them, C is expressed as The denominator of is the same for all elements and subspaces.

[0114] The Gaussian mixture model component covariance matrix ∑ k For further explanation:

[0115] Through singular value decomposition, ∑ k It can be further decomposed into:

[0116]

[0117] Among them, U k Represents ∑ k The orthogonal characteristic matrix, S k represents the diagonal eigenvalue matrix. It serves as an external sub-dictionary to guide the next stage of internal sub-dictionary learning.

[0118] Finally, we have the maximum posterior probability of are assigned to Gaussian mixture model elements.

[0119] Optionally, the following process is used for orthogonal dictionary learning:

[0120] Group all similar blocks of noisy images Assign to external prior model The corresponding Gaussian mixture model element in . The noisy block group assigned to the kth element is expressed as in, From each block group Middle school to obtain an orthogonal dictionary D k , in the corresponding external orthogonal dictionary U k Under the guidance of , we characterize the similarity block group prior of noisy images. k represents the internal and external mixed dictionary obtained by learning the internal noisy prior based on the external noiseless prior.

[0121] Let the orthogonal matrix D k for:

[0122]

[0123] in, Represents an external sub-dictionary, represented by U k The first r eigenvectors are composed of the first r eigenvectors, adaptively selected from the noisy block group Internal sub-dictionary

[0124] D k The basic principle of designing a hybrid dictionary is as follows: the external sub-dictionary D kE Pre-trained from external noise-free data, it represents the kth Gaussian mixture model component in the image, which helps to reconstruct the common structure of the image. However, D kE It is generally applicable to noise-free images, but cannot adapt to a given noisy image and cannot well describe the details of a given noisy image. Therefore, the internal sub-dictionary D is learned. kI To supplement D kE .

[0125] The hybrid orthogonal dictionary D is implemented by the following structured weighted sparse coding: k Learning:

[0126]

[0127] Among them, I k For p 2 dimensional identity matrix, α nmk represents the nth block group in the kth Dirichlet element The mth block The sparse coding vector of α nmkj is α nmk The jth element in the vector. j represents the jth regularization parameter, which is defined as follows:

[0128]

[0129] Among them, S k (j) represents the diagonal singular value matrix S k The jth singular value of , ε is a small positive number to avoid the denominator being zero. Note that if r = p 2 , then D kE =U k , if r = 0, then

[0130] Optionally, in the dictionary learning model (Formula ), the l2 norm is used to model the residuals of the block groups.

[0131] Step 105: Denoising is performed based on the internal and external mixed prior dictionary and structured sparse coding, and the reconstructed blocks in all block groups are aggregated to obtain a target image corresponding to the image to be processed.

[0132] Optionally, step 105 in the present invention specifically includes the following steps:

[0133] An internal and external hybrid orthogonal dictionary is constructed based on the external sub-dictionary and the internal sub-dictionary. The external sub-dictionary is pre-trained from external noise-free data. The internal sub-dictionary is learned by hybrid orthogonal dictionary learning through structured weighted sparse coding.

[0134] Alternately update the sparse coding vector and the internal sub-dictionary until the number of iterations exceeds the preset iteration threshold, and learn the internal and external mixed orthogonal dictionary;

[0135] Denoising is performed according to the internal and external mixed prior dictionary and structured sparse coding, and multiple iterative optimization is performed using an iterative regularization strategy;

[0136] The similar block group of the noisy image and the current block group are aggregated to obtain an aggregation result, until all image blocks of the image to be processed are aggregated, and the target image is determined according to the aggregation result.

[0137] The above steps are further explained below.

[0138] Optionally, given an orthogonal dictionary Update the sparse coding vector by:

[0139]

[0140] in, is the mth block in the nth block group in the kth Gaussian mixture model component. represents the regularization parameter vector, λ j >0,j=1,...,p 2 , represents the inner product. sgn(·) represents the sign function.

[0141] Optionally, given a sparse coding vector Update the internal sub-dictionary D by kI :

[0142]

[0143] in, For (p 2 -r)-dimensional identity matrix. The sparse coding coefficient matrix can be written as The external part and the inner part Respectively In the external sub-dictionary D kE and the internal sub-dictionary D kI The coding coefficient of . Checking internal sub-dictionaries orthogonality.

[0144] Repeat the above alternating update steps until the number of iterations exceeds the preset threshold.

[0145] Optionally, denoising is implemented based on sparse coding vectors and orthogonal dictionaries:

[0146] The denoising of the noisy image y is carried out simultaneously with the process of the external sub-dictionary guiding the learning of the internal sub-dictionary. Japanese style Get the sparse coding vectors respectively and orthogonal dictionary Image block denoising is achieved by the following formula:

[0147]

[0148] Among them, μ nk Y nk Then, the noise-free image is restored by aggregating the reconstructed blocks in all block groups.

[0149] Optionally, in order to obtain a better denoising effect, the above denoising process is iterated multiple times. At the tth iteration, the reconstructed image is reconstructed using the iterative regularization strategy. The residual of is added to the t-1th iteration. The noise standard deviation in the tth iteration is adjusted to Where η is a constant.

[0150] Figure 3 Schematic diagram of the structure of the image denoising device provided by the present invention, such as Figure 3 As shown, the apparatus includes: an image segmentation module 301 , a similar block group determination module 302 , an external priori learning module 303 , an internal priori learning module 304 and an image denoising module 305 .

[0151] The image block module 301 is used to divide the external noise-free image and the internal noisy image into multiple image blocks respectively; the internal noisy image is the image to be processed;

[0152] The similar block group determination module 302 is configured to perform a block matching search on each image block of the external noise-free image to obtain the most similar block group as the noise-free image similar block group; and perform a block matching search on each image block of the internal noisy image to obtain the most similar block group as the noisy image similar block group;

[0153] External prior learning module 303, used for learning similar block groups of noise-free images based on Gaussian mixture process and constructing Gaussian mixture prior model of external noise-free images;

[0154] Internal prior learning module 304 is used to guide the internal noisy image to learn the internal noisy prior based on the Gaussian mixture prior model, and supplement the internal dictionary with the external dictionary; wherein the guidance includes guiding the Gaussian mixture model element selection and orthogonal dictionary learning of the similar block group of the noise-free image;

[0155] The image denoising module 305 is used to implement denoising based on the internal and external mixed prior dictionary and structured sparse coding, and aggregate the reconstructed blocks in all block groups to obtain a target image corresponding to the image to be processed.

[0156] It should be noted that the image denoising device provided by the embodiment of the present invention can execute the image denoising method described in any of the above embodiments during specific operation, which will not be described in detail in this embodiment.

[0157] In order to verify the technical advantages of the image denoising method and device provided by the present invention, further analysis and explanation are given below in conjunction with experimental simulation examples.

[0158] 1. Example conditions

[0159] This example was run in the Ubuntu 18.04 operating system, using Matlab / Python language, PC Intel i7-8750H CPU (2.20GHz) and NVIDIA GeForce RTX 2060 GPU for experiments.

[0160] 2. Example Content

[0161] The experiments in this example used two datasets: Dataset-1 and Dataset-2. Dataset-1 consists of noisy images generated by adding uniform white Gaussian noise of varying intensities to a noise-free image. The noise intensities are 20, 35, 50, 65, 80, and 95, respectively. A noise intensity of 20 indicates a Gaussian noise standard deviation of 20. Dataset-2 consists of images generated by randomly adding salt and pepper, Poisson, Gaussian, multiplicative, and mixed noise to a noise-free image. The noise type and intensity are all random.

[0162] Figure 4 This figure compares the denoising results of different image denoising algorithms provided by the present invention. The dataset used in this paper is relatively complex, with varying viewpoints, scales, and poses, varying degrees of noise, distortion, and low resolution, and lacks clear and discontinuous geometric features. The figure shows that the denoising results of the present method have a good response.

[0163] Figure 5 : is a schematic diagram of the PSNR comparison results of different denoising algorithms provided by the present invention, Figure 6This figure shows a comparison of the runtime results of different denoising algorithms provided by the present invention. The comparison methods used include EPLL, NL-Means, BM3D, NL-Bayes, BLS-GSM, DCT, and MSDCT. A higher PSNR value indicates better denoising performance, while a shorter runtime indicates faster execution. The figure shows that both the PSNR value and runtime of the present invention outperform the other methods, demonstrating that the present method has superior denoising performance.

[0164] Figure 7 It is a schematic diagram of the structure of the terminal provided by the present invention, such as Figure 7 As shown, the terminal may include: a processor 710 , a communication interface 720 , a memory 730 and a communication bus 740 , wherein the processor 710 , the communication interface 720 and the memory 730 communicate with each other via the communication bus 740 . The processor 710 can call the logic instructions in the memory 730 to execute the image denoising method, which includes: dividing the external noise-free image and the internal noisy image into multiple image blocks respectively; the internal noisy image is the image to be processed; performing block matching search on each image block of the external noise-free image to obtain the most similar block group as the noise-free image similar block group; performing block matching search on each image block of the internal noisy image to obtain the most similar block group as the noisy image similar block group; learning the noise-free image similar block group based on a Gaussian mixture process to construct a Gaussian mixture prior model of the external noise-free image; guiding the internal noisy image to learn the internal noisy prior based on the Gaussian mixture prior model, and supplementing the internal dictionary with an external dictionary; wherein the guidance includes: guiding the Gaussian mixture model element selection and orthogonal dictionary learning of the noise-free image similar block group; realizing denoising based on the internal and external mixed prior dictionaries and structured sparse coding, and aggregating the reconstructed blocks in all block groups to obtain the target image corresponding to the image to be processed.

[0165] On the other hand, the present invention also provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer, the computer can execute the image denoising method provided by the above embodiments, the method including: dividing an external noise-free image and an internal noisy image into multiple image blocks respectively; the internal noisy image is the image to be processed; performing a block matching search on each image block of the external noise-free image to obtain the most similar block group as the noise-free image similar block group; performing a block matching search on each image block of the internal noisy image to obtain the most similar block group as the noisy image similar block group; learning the noise-free image similar block group based on a Gaussian mixture process to construct a Gaussian mixture prior model of the external noise-free image; guiding the internal noisy image to learn the internal noise prior based on the Gaussian mixture prior model, and supplementing the internal dictionary with an external dictionary; wherein the guidance includes: guiding the Gaussian mixture model element selection and orthogonal dictionary learning of the noise-free image similar block group; realizing denoising based on the internal and external mixed prior dictionaries and structured sparse coding, and aggregating the reconstructed blocks in all block groups to obtain the target image corresponding to the image to be processed.

[0166] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to perform the image denoising method provided by the above-mentioned embodiments, the method comprising: dividing an external noise-free image and an internal noisy image into multiple image blocks respectively; the internal noisy image is the image to be processed; performing a block matching search on each image block of the external noise-free image to obtain the most similar block group as the noise-free image similar block group; performing a block matching search on each image block of the internal noisy image to obtain the most similar block group as the noisy image similar block group; learning the noise-free image similar block group based on a Gaussian mixture process to construct a Gaussian mixture prior model of the external noise-free image; guiding the internal noisy image to learn the internal noisy prior based on the Gaussian mixture prior model, and supplementing the internal dictionary with an external dictionary; wherein the guidance comprises: guiding the Gaussian mixture model element selection and orthogonal dictionary learning of the noise-free image similar block group; implementing denoising based on the internal and external mixed prior dictionaries and structured sparse coding, and aggregating the reconstructed blocks in all block groups to obtain the target image corresponding to the image to be processed.

[0167] Through the description of the above embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus a necessary general hardware platform, or of course, by hardware. Based on this understanding, the essence of the above technical solution or the part that contributes to the existing technology can be embodied in the form of a software product. The computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a magnetic disk, an optical disk, etc., and includes a number of instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or certain parts of the embodiments.

[0168] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. An image denoising method, characterized in that: include: The external noise-free image and the internal noisy image are divided into multiple image blocks respectively; the internal noisy image is the image to be processed; Perform block matching search on each image block of the external noise-free image to obtain the most similar block group as the noise-free image similar block group; Perform block matching search on each image block of the internal noisy image to obtain the most similar block group as the similar block group of the noisy image; Based on the Gaussian mixture process, similar blocks of noise-free images are learned and a Gaussian mixture prior model of external noise-free images is constructed. The internal noisy image is guided to learn the internal noise prior based on the Gaussian mixture prior model, and the internal dictionary is supplemented with the external dictionary; wherein the guidance includes guiding the selection of Gaussian mixture model components and orthogonal dictionary learning for similar blocks of the noise-free image; Denoising is achieved based on internal and external mixed prior dictionaries and structured sparse coding, and the reconstructed blocks in all block groups are aggregated to obtain the target image corresponding to the image to be processed; Among them, based on the Gaussian mixture process, similar blocks of noise-free images are learned and a Gaussian mixture prior model of the external noise-free image is constructed, including: Learning external noise-free image patches based on Gaussian mixture processes captures the statistical characteristics of similar patches in the image and explores the model hierarchy, component structure, and parameter relationships; Generate shared weights based on the reference matrix and associated parameters. In the process of building the Gaussian mixture model, a low-rank representation is used to transform the high-dimensional problem into a low-dimensional problem, and the trace norm prior is used to construct the low-rank feature. The feature matrix of the Gaussian mixture prior model is used as an external sub-dictionary to guide the learning of the internal sub-dictionary in the next stage; The internal noisy image is guided by the Gaussian mixture prior model to learn the internal noisy prior, and the internal dictionary is supplemented with an external dictionary, including: Perform a de-meaning operation on similar block groups of noisy images; The Gaussian mixture model components of the noisy image similar block group are selected according to the Gaussian mixture prior model, the noisy image similar block group is mapped to the Gaussian mixture prior model subspace, and the most appropriate Gaussian mixture model components are selected by estimating the posterior probability; Based on the Gaussian mixture prior model, the orthogonal dictionary learning of similar block groups in noisy images is guided. All similar block groups in noisy images are assigned to corresponding Gaussian mixture model components in the Gaussian mixture prior model. A mixed orthogonal dictionary is learned from the similar block groups in noisy images. The prior of similar block groups in noisy images is characterized under the guidance of the corresponding external orthogonal sub-dictionary. The internal sub-dictionary is learned to supplement the external sub-dictionary. Denoising is achieved based on internal and external mixed prior dictionaries and structured sparse coding, and the reconstructed blocks in all block groups are aggregated to obtain the target image corresponding to the image to be processed, including: An internal and external hybrid orthogonal dictionary is constructed based on the external sub-dictionary and the internal sub-dictionary. The external sub-dictionary is pre-trained from external noise-free data. The internal sub-dictionary is learned by hybrid orthogonal dictionary learning through structured weighted sparse coding. Alternately update the sparse coding vector and the internal sub-dictionary until the number of iterations exceeds the preset iteration threshold, and learn the internal and external mixed orthogonal dictionary; Denoising is performed according to the internal and external mixed prior dictionary and structured sparse coding, and multiple iterative optimization is performed using an iterative regularization strategy; The similar block group of the noisy image and the current block group are aggregated to obtain an aggregation result, until all image blocks of the image to be processed are aggregated, and the target image is determined according to the aggregation result.

2. The image denoising method according to claim 1, wherein: Perform block matching search on each image block to obtain the most similar block group, including: Determining a searchable position of the current reference frame based on a search box centered at the position of the current image block; wherein the search box is determined based on the gradient of the current image block; Obtain all reference blocks of the same size as the current image block from the searchable positions that can be traversed according to the preset search box step size; The similarities between all reference blocks and the current image block are calculated, and matching blocks corresponding to the current reference frame are determined from all reference blocks of the current reference frame according to the similarities, so as to respectively determine matching block groups corresponding to the current image block in multiple reference frames.

3. The image denoising method according to claim 2, wherein: Determine the matching block corresponding to the current reference frame from all reference blocks of the current reference frame based on similarity, including: At least one reference block with the highest similarity among all reference blocks is used as a matching block of the current image block in the current reference frame.

4. The image denoising method according to claim 3, wherein: Also includes: If there is no matching block within the searchable position of the current reference frame, the matching block is determined according to the image block supplement method; The image block supplementation method includes: using a matching block of another reference frame or a current image block as a matching block of the current reference frame; and translating a motion vector of a matching block in the current reference frame according to a neighboring image block closest to the current image block to determine a matching block of the current image block in the current reference frame; The matching block whose motion vector is a preset value is used as the matching block of the current image block.

5. An image denoising device, characterized in that: The method for implementing the image denoising method according to any one of claims 1 to 4 comprises: An image block module is used to divide the external noise-free image and the internal noisy image into multiple image blocks respectively; the internal noisy image is the image to be processed; A similar block group determination module is used to perform a block matching search on each image block of the external noise-free image to obtain the most similar block group as the noise-free image similar block group; and to perform a block matching search on each image block of the internal noisy image to obtain the most similar block group as the noisy image similar block group; An external prior learning module is used to learn similar block groups of noise-free images based on a Gaussian mixture process and construct a Gaussian mixture prior model of external noise-free images; An internal prior learning module is configured to guide the learning of an internal noisy prior for an internal noisy image based on a Gaussian mixture prior model, and to supplement the internal dictionary with an external dictionary; wherein the guidance includes guiding the selection of Gaussian mixture model components for similar block groups of the noise-free image and orthogonal dictionary learning; The image denoising module is used to implement denoising based on internal and external mixed prior dictionaries and structured sparse coding, and aggregate the reconstructed blocks in all block groups to obtain the target image corresponding to the image to be processed.

6. A terminal comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the steps of the image denoising method according to any one of claims 1 to 4 are implemented.

7. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the image denoising method according to any one of claims 1 to 4 are implemented.

Images