Image denoising method and device, storage medium and terminal

Abstract

The invention provides an image denoising method and device, and belongs to the technical field of image processing, and the method comprises the steps: dividing an external noise-free image and an internal noise-containing image into a plurality of image blocks; performing block matching search on each image block of the external noise-free image to obtain a most similar block group as a noise-free image similar block group; block matching search is carried out on each image block of the internal noisy image, and a most similar block group is obtained and serves as a noisy image similar block group; learning a noise-free image similar block group based on a Gaussian mixture process, and constructing a Gaussian mixture prior model of an external noise-free image; guiding an internal noisy image to learn internal noisy priori based on a Gaussian mixture priori model, and supplementing an internal dictionary with an external dictionary; and denoising is realized based on the internal and external mixed prior dictionaries and structured sparse coding, and reconstruction blocks in all block groups are aggregated to obtain a target image corresponding to the to-be-processed image. According to the invention, the image denoising effect can be improved, and the image quality is improved.

Description

Technical Field

[0001] The present invention relates to the field of image processing technology, and in particular to an image denoising method, device, storage medium and terminal. Background Art

[0002] In the process of data collection, quantification, transmission and storage, due to the influence of various factors such as the environment, transmission channel and the equipment itself, images will inevitably introduce various types of noise, which seriously affects the accuracy, reliability and efficiency of pattern recognition and image interpretation in the subsequent stage. Image denoising is an indispensable key link to improve image and visual quality, and has extensive and meaningful practical applications in information retrieval, public safety and other fields.

[0003] In the current image denoising algorithms, most of them need to assume that the noise follows a uniform white Gaussian noise distribution with known intensity. In fact, due to the differences in imaging methods and imaging conditions, image noise is composed of a mixture of multiple types (such as salt and pepper / Poisson / Gaussian / multiplicative noise), which is far more complex than uniform white Gaussian noise.

[0004] In addition, image noise is related to factors such as pixel distribution, target scale, spatial structure, usage frequency and polarization mode. It is difficult to describe each type of hidden noise with a simple distribution function, and modeling is very complicated. If the denoising method is inappropriate, the image features will be weakened, further increasing the difficulty of image interpretation. Summary of the invention

[0005] The present invention provides an image denoising method, device, storage medium and terminal, which are used to solve the defects of the prior art that the image denoising background is complex and difficult to accurately model, and realize the enhancement of image quality.

[0006] In a first aspect, the present invention provides an image denoising method, comprising: dividing an external noise-free image and an internal noisy image into a plurality of image blocks respectively; the internal noisy image is an image to be processed; performing a block matching search on each image block of the external noise-free image to obtain the most similar block group as a noise-free image similar block group; performing a block matching search on each image block of the internal noisy image to obtain the most similar block group as a noisy image similar block group; learning the noise-free image similar block group based on a Gaussian mixture process to construct a Gaussian mixture prior model of the external noise-free image; guiding the internal noisy image to learn the internal noisy prior based on the Gaussian mixture prior model, and supplementing the internal dictionary with an external dictionary; wherein the guidance comprises: guiding the Gaussian mixture model element selection and orthogonal dictionary learning of the noise-free image similar block group; realizing denoising based on internal and external mixed prior dictionaries and structured sparse coding, and aggregating the reconstructed blocks in all block groups to obtain a target image corresponding to the image to be processed.

[0007] According to the image denoising method provided by the present invention, a block matching search is performed on each image block to obtain the most similar block group, including: taking the position of the current image block as the center, determining the searchable position of the current reference frame according to the search box; wherein the search box is determined according to the gradient of the current image block; obtaining all reference blocks with the same size as the current image block according to the searchable positions that can be traversed according to the preset search box step size; calculating the similarity between all reference blocks and the current image block, and determining the matching block corresponding to the current reference frame from all reference blocks of the current reference frame according to the similarity to determine the matching block group corresponding to the current image block in multiple reference frames respectively.

[0008] According to the image denoising method provided by the present invention, a matching block corresponding to the current reference frame is determined from all reference blocks of the current reference frame according to similarity, including: at least one reference block with the highest similarity among all reference blocks is used as a matching block of the current image block in the current reference frame.

[0009] The image denoising method provided by the present invention further includes: if there is no matching block in the searchable position of the current reference frame, determining the matching block according to the image block supplement method; wherein the image block supplement method includes: using the matching block of other reference frames or the current image block as the matching block of the current reference frame; and translating the motion vector of the matching block in the current reference frame according to the adjacent image block closest to the current image block to determine the matching block of the current image block in the current reference frame; using the matching block with a motion vector of a preset value as the matching block of the current image block.

[0010] According to the image denoising method provided by the present invention, a Gaussian mixture process is used to learn a group of similar blocks of a noise-free image, and a Gaussian mixture prior model of an external noise-free image is constructed, including: learning external noise-free image blocks based on a Gaussian mixture process, capturing the statistical characteristics of similar blocks in the image, and exploring the model hierarchy, component structure, and parameter relationship; generating shared weights according to a reference matrix and associated parameters, and in the process of constructing the Gaussian mixture model, using a low-rank representation to transform a high-dimensional problem into a low-dimensional problem, and using a trace norm prior to construct a low-rank characteristic; and using a characteristic matrix of the Gaussian mixture prior model as an external sub-dictionary to guide the learning of an internal sub-dictionary in the next stage.

[0011] According to the image denoising method provided by the present invention, an internal noisy image is guided to learn an internal noisy prior based on a Gaussian mixture prior model, and an internal dictionary is supplemented with an external dictionary, including: performing a de-meaning operation on a noisy image similar block group; guiding the selection of Gaussian mixture model elements of the noisy image similar block group according to the Gaussian mixture prior model, mapping the noisy image similar block group to the Gaussian mixture prior model subspace, and selecting the most appropriate Gaussian mixture model element by estimating the posterior probability; guiding the orthogonal dictionary learning of the noisy image similar block group based on the Gaussian mixture prior model, assigning all noisy image similar block groups to corresponding Gaussian mixture model elements in the Gaussian mixture prior model, learning a mixed orthogonal dictionary from the noisy image similar block group, characterizing the noisy image similar block group prior under the guidance of the corresponding external orthogonal sub-dictionary, and learning an internal sub-dictionary to supplement the external sub-dictionary.

[0012] According to the image denoising method provided by the present invention, denoising is achieved based on an internal and external mixed prior dictionary and structured sparse coding, and reconstructed blocks in all block groups are aggregated to obtain a target image corresponding to the image to be processed, including: constructing an internal and external mixed orthogonal dictionary according to an external sub-dictionary and an internal sub-dictionary, the external sub-dictionary is pre-trained from external noise-free data, and hybrid orthogonal dictionary learning is achieved through structured weighted sparse coding to learn the internal sub-dictionary; alternately updating the sparse coding vector and the internal sub-dictionary until the number of iterations exceeds a preset iteration number threshold, and the internal and external mixed orthogonal dictionary is learned; denoising is performed according to the internal and external mixed prior dictionary and structured sparse coding, and multiple iterative optimization is performed using an iterative regularization strategy; similar block groups of noisy images and the current block group are aggregated to obtain an aggregation result, until all image blocks of the image to be processed are aggregated, and the target image is determined according to the aggregation result.

[0013] In a second aspect, the present invention further provides an image denoising device, comprising:

[0014] An image block division module is used to divide the external noise-free image and the internal noisy image into a plurality of image blocks respectively; the internal noisy image is the image to be processed;

[0015] The similar block group determination module is used to perform block matching search on each image block of the external noise-free image to obtain the most similar block group as the noise-free image similar block group; and perform block matching search on each image block of the internal noisy image to obtain the most similar block group as the noisy image similar block group;

[0016] An external prior learning module is used to learn similar block groups of noise-free images based on a Gaussian mixture process and to construct a Gaussian mixture prior model of external noise-free images;

[0017] An internal prior learning module is used to guide the internal noisy image to learn the internal noisy prior based on the Gaussian mixture prior model, and to supplement the internal dictionary with the external dictionary; wherein the guidance includes: guiding the Gaussian mixture model element selection and orthogonal dictionary learning of the noise-free image similar block group;

[0018] The image denoising module is used to implement denoising based on internal and external mixed prior dictionaries and structured sparse coding, and aggregate the reconstructed blocks in all block groups to obtain the target image corresponding to the image to be processed.

[0019] In a third aspect, the present invention provides a terminal, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of any of the above-mentioned image denoising methods are implemented.

[0020] In a fourth aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of any of the above-described image denoising methods.

[0021] The image denoising method, device, storage medium and terminal provided by the present invention convert the denoising problem into a learning problem to be solved. It does not need to utilize the complex relationship between noise and image features, but is based on a low-rank Gaussian mixture process, hybrid dictionary learning and structured weighted sparse coding. It also utilizes external noise-free image and internal noisy image information, uses noise-free priors to guide noisy prior learning, and uses an external dictionary to supplement the internal dictionary, thereby achieving effective image denoising without the need to predict the type and intensity of noise. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0023] Figure 1 It is a flow chart of the image denoising method provided by the present invention;

[0024] Figure 2 is a schematic diagram of block matching provided by the present invention;

[0025] Figure 3 It is a structural schematic diagram of the image denoising device provided by the present invention;

[0026] Figure 4 It is a schematic diagram comparing denoising results of different image denoising algorithms provided by the present invention;

[0027] Figure 5Schematic diagram of PSNR comparison results of different denoising algorithms provided by the present invention;

[0028] Figure 6 is a schematic diagram of the comparison results of the running time of different denoising algorithms provided by the present invention;

[0029] Figure 7 It is a schematic diagram of the structure of the terminal provided by the present invention. DETAILED DESCRIPTION

[0030] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0031] It should be noted that, in the description of the embodiments of the present invention, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "include one..." do not exclude the existence of other identical elements in the process, method, article or device including the elements. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to the specific circumstances.

[0032] The terms "first", "second", etc. in this application are used to distinguish similar objects, and are not used to describe a specific order or sequence. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, so that the embodiments of the present application can be implemented in an order other than those illustrated or described here, and the objects distinguished by "first", "second", etc. are generally of the same type, and the number of objects is not limited. For example, the first object can be one or more.

[0033] Combine the following Figure 1-Figure 7 The image denoising method and device provided by the embodiments of the present invention are described.

[0034] Figure 1 It is a flow chart of the image denoising method provided by the present invention, such as Figure 1 As shown, including but not limited to the following steps:

[0035] Step 101: Divide the external noise-free image and the internal noisy image into a plurality of image blocks respectively.

[0036] In the method proposed in the present invention, the internal and external mixed prior is composed of an internal noisy prior and an external noise-free prior. The external noise-free image and the internal noisy image have no special meanings, and refer to a noise-free image and a noisy image, respectively.

[0037] Among them, the internal noisy image is the image to be processed; the image to be processed can be an image obtained by cameras or other image acquisition devices installed in various places such as roads, communities, airports / train stations, large venues, etc.

[0038] Step 102: Perform block matching search on each image block of the external noise-free image to obtain the most similar block group as the noise-free image similar block group; perform block matching search on each image block of the internal noisy image to obtain the most similar block group as the noisy image similar block group.

[0039] It can be understood that the matching search process for the external noise-free image and the internal noisy image is the same.

[0040] Optionally, step 102 specifically includes the following steps:

[0041] Taking the position of the current image block as the center, the searchable position of the current reference frame is determined according to a search box; wherein the search box is determined according to the gradient of the current image block.

[0042] According to the searchable positions that can be traversed by the preset search box step length, all reference blocks with the same size as the current image block are obtained;

[0043] Calculating the similarity between all reference blocks and the current image block, and determining a matching block corresponding to the current reference frame from all reference blocks of the current reference frame according to the similarity, so as to respectively determine a matching block group corresponding to the current image block in multiple reference frames;

[0044] Among them, at least one reference block with the highest similarity among all the reference blocks is used as a matching block of the current image block in the current reference frame.

[0045] Optionally, if there is no matching block in the searchable position of the current reference frame, the matching block is determined according to an image block supplementation method.

[0046] The image block supplementation method includes: using a matching block of another reference frame or a current image block as a matching block of the current reference frame; and,

[0047] According to the adjacent image block closest to the current image block, the motion vector of the matching block is translated in the current reference frame to determine the matching block of the current image block in the current reference frame; and the matching block with a motion vector of a preset value is used as the matching block of the current image block.

[0048] The following is a further explanation of the process of block matching to search for similar image blocks:

[0049] Assume that the image has N pixels in total, the search box size is W×W, the neighborhood window size is p×p, the neighborhood window slides in the search box, and the pixel weight is determined according to the similarity between neighbors. Figure 2 It is a schematic diagram of block matching provided by the present invention, such as Figure 2 As shown, the large window (purple border) is i(x i ,y i ) is the center of the search box, and the two green windows (with red borders) represent the search results with i(x i ,y i ), j(x j ,y j ) is the center of the neighborhood window N i 、N j , where j(x j ,y j ) is the center of the neighborhood window N j Slide in the search box and calculate the neighborhood window N i 、N j The similarity between them is assigned a weighted value w(i,j).

[0050] Assume the noisy image is I:

[0051] I = {I(x,y)|(x,y)∈Ω};

[0052] Where x, y are the pixel coordinates on I.

[0053] Use the weighted average of all pixels in image I as the pixel i (x i ,y i ) at the estimated value I'(x i ,y i ):

[0054]

[0055] Among them, i(x i ,y i ) and j(x j ,y j ) depends on the similarity between them, which is calculated by i ,y i )、j(x j ,y j ) is the center of the rectangular neighborhood N i 、N j The distance between ||N i -N j || 2Use the Euclidean distance to measure the two image blocks N i 、N j The similarity between them w(i,j):

[0056]

[0057] Among them, Z(i) represents the normalization factor, i(x i ,y i ) and j(x j ,y j ) represent the rectangular neighborhood N i 、N j The central pixel of , h is the smoothing parameter, S(i,j) is the pixel neighborhood N i and N j The weighted Euclidean distance between is calculated as follows:

[0058]

[0059] Here, a is the standard deviation of the Gaussian kernel.

[0060] Optionally, the Euclidean distance S(i,j) is used to compare the pixel neighborhood N i and N j Similarity between:

[0061]

[0062] Among them, I i (g,m) and I j (g,m) represent the pixel neighborhood N i and N j The corresponding pixels in , g and m represent the positions of the image block moving up, down, left, and right in the search box, respectively. The search box size is W×W.

[0063] Optionally, the present invention further explains image block group extraction and block group mean removal.

[0064] There are often many repeated local patterns and similar image patches in an image. A patch group (PG) is formed by grouping similar local image patches.

[0065] For example, a noise-free image is divided into overlapping image blocks (size is p×p). For any image block, a search box of size W×W is used to search for the image block and the M blocks that are most similar to it, which is called block matching. The image block group composed of these M similar blocks is represented as in, represents the block vector, and the mean vector of the image block group is To highlight individual characteristics and differences, remove the mean of the block vector, that is, The image block group after removing the mean is defined as:

[0066]

[0067] Step 103: Based on the Gaussian mixture process, similar block groups of the noise-free image are learned to construct a Gaussian mixture prior model of the external noise-free image.

[0068] Optionally, the noise-free image similar block group searched in step 102 is modeled, and a Gaussian mixture prior model of the external noise-free image is constructed based on a Gaussian mixture process to capture the statistical characteristics of similar blocks in the image and explore the model hierarchy, component structure and parameter relationship;

[0069] Optionally, a shared weight parameter is generated according to a reference matrix, an associated parameter and a const normalization term, and a conjugate prior of a Gaussian distribution is introduced as a reference distribution;

[0070] w t ={μ t ,A t} represents the parameters of subspace t, where μ t and A t Denote the mean vector and dictionary matrix of subspace t respectively. Set the parameters {μ t ,A t}:

[0071] μ t ~G(·|μ0,Σ0)

[0072]

[0073] Where G represents the Gaussian mixture process and const represents the normalization term. t,k ={μ t,k ,∑ t,k}Determine the Gaussian mixture model element of subspace t, μ t,k and∑ t,k Represent the mean vector and covariance matrix of each element respectively, and D represents μ t,k The conjugate prior of the Gaussian distribution is introduced as the reference distribution as follows:

[0074] μ t,k ~G(·|ε0,Ω0)

[0075] ∑ t,k ~iWishart(v0,B0)

[0076] Where d∈{1,...,D}, iWishart(v0,B0) represents the inverse gamma distribution, v0 and B0 represent the shape parameter and scale matrix, respectively.

[0077] Optionally, in order to reduce the computational complexity of the model, in the process of building the Gaussian mixture model, a low-rank representation is used to transform the high-dimensional problem into a low-dimensional problem, and the trace norm prior is used to construct the low-rank feature:

[0078]

[0079] Where Tr represents the trace of the matrix, A is a d×d dimensional matrix,

[0080] Optionally, the Gaussian mixture process includes one or more of a Chinese restaurant process, a folded stick process, and a Polyaeon scheme.

[0081] Assume that L block groups are extracted from a set of noise-free images, and the lth block group is:

[0082]

[0083] Where M represents the number of similar image blocks. There is abundant non-local similarity information in , so the problem is transformed into how to A Gaussian mixture model is learned to model the external priors of the block groups.

[0084] Optionally, based on a Gaussian mixture model, we aim to extract the L noise-free image blocks from the L noise-free image blocks. The prior model {N(μ k ,∑ k )}, and requires block groups M similar blocks belong to the same Gaussian mixture model element. It is calculated by the following formula The likelihood:

[0085]

[0086] Assuming that all block groups are sampled independently, the total target likelihood function is Take its logarithm and maximize the objective function of Gaussian mixture learning:

[0087]

[0088] In Gaussian mixture learning, the implicit variable {Δ lk |l=1,...,L;k=1,...,K}Optimization formula If the block group belongs to the kth element, then Δ lk =1, otherwise, Δ lk =0.

[0089] Optionally, use the Expectation Maximization (EM) algorithm to optimize The EM algorithm mainly includes: expectation step (E step) and maximization step (M step).

[0090] The EM algorithm alternately executes the E step and the M step:

[0091] 1. Step E, introduce the posterior probability γ lk , indicating that the kth component generates a block Probability:

[0092]

[0093] 2. Randomly initialize γ lk ;

[0094] 3.M step: based on the current γ lk Estimate parameters. Each block group in M ​​steps Therefore:

[0095]

[0096] in,

[0097] 4.E Step: Based on the current parameter values ​​and formula Re-estimate γ lk ;

[0098] 5. Calculate the log-likelihood of the image (Formula ) and check the convergence of the log-likelihood. If it does not converge, go back to step 3.

[0099] Step 104: guiding the internal noisy image to learn the internal noisy prior based on the Gaussian mixture prior model, and supplementing the internal dictionary with the external dictionary. The guidance includes guiding the Gaussian mixture model element selection and orthogonal dictionary learning of the noise-free image similar block group.

[0100] Optionally, step 104 in the present invention specifically includes the following steps:

[0101] Perform a de-meaning operation on similar block groups of noisy images;

[0102] The Gaussian mixture model element selection of the noisy image similar block group is guided by the Gaussian mixture prior model, the noisy image similar block group is mapped to the Gaussian mixture prior model subspace, and the most appropriate Gaussian mixture model element is selected by estimating the posterior probability;

[0103] The orthogonal dictionary learning of similar block groups of noisy images is guided by the Gaussian mixture prior model. All similar block groups of noisy images are assigned to the corresponding Gaussian mixture model elements in the Gaussian mixture prior model. A mixed orthogonal dictionary is learned from the similar block groups of noisy images. The prior of similar block groups of noisy images is characterized under the guidance of the corresponding external orthogonal sub-dictionary, and the internal sub-dictionary is learned to supplement the external sub-dictionary.

[0104] Here, "Gaussian mixture model elements" usually refer to the individual Gaussian distributions that make up the entire mixture model. Each such Gaussian distribution is called an "element" or "component". These elements work together to describe the overall distribution of the data.

[0105] The above steps are further explained below.

[0106] Optionally, the following process is used to select Gaussian mixture model components:

[0107] Given a noisy image y, extract N image blocks (noisy blocks). Similar to the external prior learning stage, for the nth noisy block (n = 1, 2, ..., N), search for the M most similar image blocks through block matching to construct a noisy image similar block group Y n ={y n1 ,...,y nM}. The mean vector of this block group is μ n , the block vector after removing the mean is The similar block group of noisy image with the mean removed is remember in is the corresponding noise-free block group, and V is the noise. At this point, the problem is transformed into how to use the learned external Gaussian mixture prior to Recovery Because the mean vector of the noise V is close to the zero vector, The mean μ y is very close to The mean of .

[0108] Gaussian Mixture External Prior Model The subspace of the noise-free block group can be characterized. Select appropriate Gaussian mixture model components from the Gaussian mixture. Therefore, each noisy block group Mapping to Gaussian Mixture Model In the subspace, the posterior probability is estimated as Select the most appropriate Gaussian mixture model component:

[0109]

[0110] Wherein, k=1,...,K, k is the number of Gaussian mixture model elements.

[0111] Pair Taking the logarithm, we get:

[0112]

[0113] Among them, C is expressed as The denominator of is the same for all elements and subspaces.

[0114] The Gaussian mixture model component covariance matrix ∑ k For further explanation:

[0115] Through singular value decomposition, ∑ k It can be further decomposed into:

[0116]

[0117] Among them, U k Represents ∑ k The orthogonal characteristic matrix, S k represents the diagonal eigenvalue matrix. It serves as an external sub-dictionary to guide the next stage of internal sub-dictionary learning.

[0118] Finally, we have the maximum posterior probability of are assigned to Gaussian mixture model elements.

[0119] Optionally, the following process is used for orthogonal dictionary learning:

[0120] Group all similar blocks of noisy images Assignment to external prior model The corresponding Gaussian mixture model element in . The noisy block group assigned to the kth element is expressed as in, From each block group Learn an orthogonal dictionary D k , in the corresponding external orthogonal dictionary U k Under the guidance of , we characterize the similar block group prior of noisy images. k represents the internal and external mixed dictionary obtained by learning the internal noisy prior based on the external noise-free prior.

[0121] Let the orthogonal matrix D k for:

[0122]

[0123] in, Represents an external sub-dictionary, represented by U k The first r feature vectors are composed of Internal sub-dictionary

[0124] D k The basic principle of designing a hybrid dictionary is as follows: the external sub-dictionary D kE Pre-trained from external noise-free data, it represents the kth Gaussian mixture model component in the image, which helps to reconstruct the common structure of the image. However, D kE It is generally applicable to noise-free images, but cannot adapt to a given noisy image and cannot well describe the details of a given noisy image. Therefore, the internal sub-dictionary D is learned. kI To supplement D kE .

[0125] The hybrid orthogonal dictionary D is implemented by the following structured weighted sparse coding: k Learning:

[0126]

[0127] Among them, I k For p 2 dimensional identity matrix, α nmk represents the nth block group in the kth Dirichlet element The mth block in The sparse coding vector of nmkj is α nmk The jth element in the vector. j represents the jth regularization parameter and is defined as follows:

[0128]

[0129] Among them, S k (j) represents the diagonal singular value matrix S k The jth singular value of , ε is a small positive number to avoid the denominator being zero. Note that if r = p 2 , then D kE =U k , if r = 0, then

[0130] Optionally, in the dictionary learning model (Formula ), the l2 norm is used to model the residuals of the block groups.

[0131] Step 105: Denoising is performed based on the internal and external mixed prior dictionary and structured sparse coding, and the reconstructed blocks in all block groups are aggregated to obtain a target image corresponding to the image to be processed.

[0132] Optionally, step 105 in the present invention specifically includes the following steps:

[0133] An internal and external hybrid orthogonal dictionary is constructed based on the external sub-dictionary and the internal sub-dictionary. The external sub-dictionary is pre-trained from external noise-free data, and the hybrid orthogonal dictionary learning is realized through structured weighted sparse coding to learn the internal sub-dictionary.

[0134] Alternately update the sparse coding vector and the internal sub-dictionary until the number of iterations exceeds a preset iteration threshold, and learn an internal and external mixed orthogonal dictionary;

[0135] Denoising is performed according to the internal and external mixed prior dictionary and structured sparse coding, and multiple iterative optimization is performed using an iterative regularization strategy;

[0136] The similar block groups of the noisy image and the current block group are aggregated to obtain an aggregation result, until all image blocks of the image to be processed are aggregated, and the target image is determined according to the aggregation result.

[0137] The above steps are further explained below.

[0138] Optionally, given an orthogonal dictionary Update the sparse coding vector by:

[0139]

[0140] in, is the mth block in the nth block group in the kth Gaussian mixture model element. represents the regularization parameter vector, λ j >0,j=1,...,p 2 , represents the inner product. sgn(·) represents the sign function.

[0141] Optionally, given a sparse coding vector Update the internal sub-dictionary D by kI :

[0142]

[0143] in, for (p 2 -r)-dimensional identity matrix. The sparse coding coefficient matrix can be written as The external part and the internal part Respectively In the external sub-dictionary D kE and the internal sub-dictionary D kI The coding coefficient of . Checking the internal sub-dictionary orthogonality.

[0144] Repeat the above alternating update steps until the number of iterations exceeds the preset threshold.

[0145] Optionally, denoising is implemented based on sparse coding vectors and orthogonal dictionaries:

[0146] The denoising of the noisy image y is carried out simultaneously with the process of the external sub-dictionary guiding the learning of the internal sub-dictionary. Japanese style Get the sparse coding vectors and orthogonal dictionary Image block denoising is achieved by the following formula:

[0147]

[0148] Among them, μ nk Y nk Then, the noise-free image is restored by aggregating the reconstructed blocks in all block groups.

[0149] Optionally, in order to obtain a better denoising effect, the above denoising process is iterated multiple times. In the tth iteration, the reconstructed image is reconstructed using the iterative regularization strategy. The residual is added to the t-1th iteration. The noise standard deviation in the tth iteration is adjusted to Where η is a constant.

[0150] Figure 3 is a schematic diagram of the structure of the image denoising device provided by the present invention, such as Figure 3 As shown, the device includes: an image segmentation module 301 , a similar block group determination module 302 , an external prior learning module 303 , an internal prior learning module 304 and an image denoising module 305 .

[0151] The image block division module 301 is used to divide the external noise-free image and the internal noisy image into a plurality of image blocks respectively; the internal noisy image is the image to be processed;

[0152] The similar block group determination module 302 is used to perform a block matching search on each image block of the external noise-free image to obtain the most similar block group as the noise-free image similar block group; and perform a block matching search on each image block of the internal noisy image to obtain the most similar block group as the noisy image similar block group;

[0153] An external prior learning module 303 is used to learn similar block groups of noise-free images based on a Gaussian mixture process and construct a Gaussian mixture prior model of external noise-free images;

[0154] Internal prior learning module 304, used for guiding the internal noisy image to learn the internal noisy prior based on the Gaussian mixture prior model, and supplementing the internal dictionary with the external dictionary; wherein the guidance includes: guiding the Gaussian mixture model element selection and orthogonal dictionary learning of the noise-free image similar block group;

[0155] The image denoising module 305 is used to implement denoising based on the internal and external mixed priori dictionary and structured sparse coding, and aggregate the reconstructed blocks in all block groups to obtain a target image corresponding to the image to be processed.

[0156] It should be noted that the image denoising device provided in the embodiment of the present invention can execute the image denoising method described in any of the above embodiments during specific operation, which will not be described in detail in this embodiment.

[0157] In order to verify the technical advantages of the image denoising method and device provided by the present invention, further analysis and description are given below in conjunction with an experimental simulation embodiment.

[0158] 1. Example conditions

[0159] This embodiment is run in the Ubuntu 18.04 operating system, using Matlab / Python language, PC Intel i7-8750H CPU (2.20GHz) and NVIDIA GeForce RTX 2060 GPU for experiments.

[0160] 2. Example Content

[0161] Two data sets are used for experiments in the embodiment, namely, data set-1 and data set-2. Data set-1 is a noisy image obtained by adding uniform white Gaussian noise of different intensities to a noise-free image, and the noise intensities are 20, 35, 50, 65, 80 and 95 respectively. The noise intensity of 20 means that the standard deviation of the Gaussian noise is 20. Data set-2 is an image obtained by randomly adding salt and pepper / Poisson / Gaussian / multiplicative noise and their mixed noises to a noise-free image, and the noise type and intensity are random.

[0162] Figure 4 This is a schematic diagram of the denoising results of different image denoising algorithms provided by the present invention. The data set used in the present invention is relatively complex, with different viewing angles, scales, postures, different degrees of noise, deformation and low resolution, and unclear and discontinuous geometric features. It can be seen from the figure that the denoising result of the method of the present invention has a better response.

[0163] Figure 5 : is a schematic diagram of the PSNR comparison results of different denoising algorithms provided by the present invention, Figure 6It is a schematic diagram of the running time comparison results of different denoising algorithms provided by the present invention. Among them, the comparison methods used are EPLL, NL-Means, BM3D, NL-Bayes, BLS-GSM, DCT and MSDCT. The larger the PSNR value, the better the denoising effect of the method, and the shorter the running time, the faster the running speed of the method. It can be seen from the figure that the PSNR value and running time of the present invention are better than those of other methods. It shows that the present method has good denoising performance.

[0164] Figure 7 is a schematic diagram of the structure of the terminal provided by the present invention, such as Figure 7 As shown, the terminal may include: a processor 710 , a communication interface 720 , a memory 730 and a communication bus 740 , wherein the processor 710 , the communication interface 720 , and the memory 730 communicate with each other via the communication bus 740 . The processor 710 can call the logic instructions in the memory 730 to execute the image denoising method, which includes: dividing the external noise-free image and the internal noisy image into multiple image blocks respectively; the internal noisy image is the image to be processed; performing block matching search on each image block of the external noise-free image to obtain the most similar block group as the noise-free image similar block group; performing block matching search on each image block of the internal noisy image to obtain the most similar block group as the noisy image similar block group; learning the noise-free image similar block group based on the Gaussian mixture process to construct a Gaussian mixture prior model of the external noise-free image; guiding the internal noisy image to learn the internal noisy prior based on the Gaussian mixture prior model, and supplementing the internal dictionary with an external dictionary; wherein the guidance includes: guiding the Gaussian mixture model element selection and orthogonal dictionary learning of the noise-free image similar block group; realizing denoising based on the internal and external mixed prior dictionaries and structured sparse coding, and aggregating the reconstructed blocks in all block groups to obtain the target image corresponding to the image to be processed.

[0165] On the other hand, the present invention also provides a computer program product, which includes a computer program stored on a non-transitory computer-readable storage medium, and the computer program includes program instructions. When the program instructions are executed by a computer, the computer can execute the image denoising method provided by the above embodiments, the method including: dividing an external noise-free image and an internal noisy image into multiple image blocks respectively; the internal noisy image is the image to be processed; performing block matching search on each image block of the external noise-free image to obtain the most similar block group as the noise-free image similar block group; performing block matching search on each image block of the internal noisy image to obtain the most similar block group as the noisy image similar block group; learning the noise-free image similar block group based on a Gaussian mixture process to construct a Gaussian mixture prior model of the external noise-free image; guiding the internal noisy image to learn the internal noisy prior based on the Gaussian mixture prior model, and supplementing the internal dictionary with an external dictionary; wherein the guidance includes: guiding the Gaussian mixture model element selection and orthogonal dictionary learning of the noise-free image similar block group; realizing denoising based on internal and external mixed prior dictionaries and structured sparse coding, and aggregating the reconstructed blocks in all block groups to obtain the target image corresponding to the image to be processed.

[0166] On the other hand, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which is implemented when the processor executes the image denoising method provided by the above-mentioned embodiments, the method comprising: dividing an external noise-free image and an internal noisy image into a plurality of image blocks respectively; the internal noisy image is the image to be processed; performing a block matching search on each image block of the external noise-free image to obtain the most similar block group as the noise-free image similar block group; performing a block matching search on each image block of the internal noisy image to obtain the most similar block group as the noisy image similar block group; learning the noise-free image similar block group based on a Gaussian mixture process to construct a Gaussian mixture prior model of the external noise-free image; guiding the internal noisy image to learn the internal noisy prior based on the Gaussian mixture prior model, and supplementing the internal dictionary with an external dictionary; wherein the guidance comprises: guiding the Gaussian mixture model element selection and orthogonal dictionary learning of the noise-free image similar block group; realizing denoising based on internal and external mixed prior dictionaries and structured sparse coding, and aggregating the reconstructed blocks in all block groups to obtain the target image corresponding to the image to be processed.

[0167] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0168] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An image denoising method, characterized in that: include: The external noise-free image and the internal noisy image are divided into a plurality of image blocks respectively; the internal noisy image is the image to be processed; Perform block matching search on each image block of the external noise-free image to obtain the most similar block group as the noise-free image similar block group; Perform block matching search on each image block of the internal noisy image to obtain the most similar block group as the similar block group of the noisy image; Based on the Gaussian mixture process, similar blocks of noise-free images are learned to construct a Gaussian mixture prior model of external noise-free images. Based on the Gaussian mixture prior model, the internal noisy image is guided to learn the internal noisy prior, and the internal dictionary is supplemented with the external dictionary; wherein the guidance includes: guiding the Gaussian mixture model element selection and orthogonal dictionary learning of the noise-free image similar block group; Denoising is achieved based on internal and external mixed prior dictionaries and structured sparse coding, and the reconstructed blocks in all block groups are aggregated to obtain the target image corresponding to the image to be processed.

2. The image denoising method according to claim 1, characterized in that: Perform block matching search on each image block to obtain the most similar block group, including: Taking the position of the current image block as the center, determining the searchable position of the current reference frame according to a search box; wherein the search box is determined according to the gradient of the current image block; According to the searchable positions that can be traversed by the preset search box step length, all reference blocks with the same size as the current image block are obtained; The similarities between all reference blocks and the current image block are calculated, and matching blocks corresponding to the current reference frame are determined from all reference blocks of the current reference frame according to the similarities, so as to respectively determine matching block groups corresponding to the current image block in multiple reference frames.

3. The image denoising method according to claim 2, characterized in that: Determine the matching block corresponding to the current reference frame from all reference blocks of the current reference frame according to the similarity, including: At least one reference block with the highest similarity among all the reference blocks is used as a matching block of the current image block in the current reference frame.

4. The image denoising method according to claim 3, characterized in that: Also includes: If there is no matching block in the searchable position of the current reference frame, the matching block is determined according to the image block supplementation method; The image block supplementation method includes: using a matching block of another reference frame or a current image block as a matching block of the current reference frame; and, According to the adjacent image block closest to the current image block, the motion vector of the matching block is translated in the current reference frame to determine the matching block of the current image block in the current reference frame; A matching block whose motion vector is a preset value is used as a matching block of the current image block.

5. The image denoising method according to claim 4, characterized in that: Based on the Gaussian mixture process, similar blocks of noise-free images are learned and a Gaussian mixture prior model of external noise-free images is constructed, including: Learning external noise-free image patches based on Gaussian mixture processes captures the statistical characteristics of similar patches in the image and explores model hierarchy, component structure, and parameter relationships; Generate shared weights based on the reference matrix and associated parameters. In the process of building the Gaussian mixture model, use low-rank representation to transform high-dimensional problems into low-dimensional problems, and use trace norm priors to construct low-rank features. The feature matrix of the Gaussian mixture prior model is used as an external sub-dictionary to guide the next stage of internal sub-dictionary learning.

6. The image denoising method according to claim 5, characterized in that: Based on the Gaussian mixture prior model, the internal noisy image is guided to learn the internal noisy prior, and the internal dictionary is supplemented with an external dictionary, including: Perform a de-meaning operation on similar block groups of noisy images; The Gaussian mixture model element selection of the noisy image similar block group is guided by the Gaussian mixture prior model, the noisy image similar block group is mapped to the Gaussian mixture prior model subspace, and the most appropriate Gaussian mixture model element is selected by estimating the posterior probability; The orthogonal dictionary learning of similar block groups of noisy images is guided by the Gaussian mixture prior model. All similar block groups of noisy images are assigned to the corresponding Gaussian mixture model elements in the Gaussian mixture prior model. A mixed orthogonal dictionary is learned from the similar block groups of noisy images. The prior of similar block groups of noisy images is characterized under the guidance of the corresponding external orthogonal sub-dictionary, and the internal sub-dictionary is learned to supplement the external sub-dictionary.

7. The image denoising method according to claim 6, characterized in that: Denoising is achieved based on internal and external mixed prior dictionaries and structured sparse coding, and the reconstructed blocks in all block groups are aggregated to obtain the target image corresponding to the image to be processed, including: An internal and external hybrid orthogonal dictionary is constructed based on the external sub-dictionary and the internal sub-dictionary. The external sub-dictionary is pre-trained from external noise-free data, and the hybrid orthogonal dictionary learning is realized through structured weighted sparse coding to learn the internal sub-dictionary. Alternately update the sparse coding vector and the internal sub-dictionary until the number of iterations exceeds a preset iteration threshold, and learn an internal and external mixed orthogonal dictionary; Denoising is performed according to the internal and external mixed prior dictionary and structured sparse coding, and multiple iterative optimization is performed using an iterative regularization strategy; The similar block groups of the noisy image and the current block group are aggregated to obtain an aggregation result, until all image blocks of the image to be processed are aggregated, and the target image is determined according to the aggregation result.

8. An image denoising device, characterized in that: include: An image block division module is used to divide the external noise-free image and the internal noisy image into a plurality of image blocks respectively; the internal noisy image is the image to be processed; A similar block group determination module is used to perform block matching search on each image block of the external noise-free image to obtain the most similar block group as the noise-free image similar block group; Perform block matching search on each image block of the internal noisy image to obtain the most similar block group as the similar block group of the noisy image; An external prior learning module is used to learn similar block groups of noise-free images based on a Gaussian mixture process and to construct a Gaussian mixture prior model of external noise-free images; An internal prior learning module is used to guide the internal noisy image to learn the internal noisy prior based on the Gaussian mixture prior model, and to supplement the internal dictionary with the external dictionary; wherein the guidance includes: guiding the Gaussian mixture model element selection and orthogonal dictionary learning of the noise-free image similar block group; The image denoising module is used to implement denoising based on internal and external mixed prior dictionaries and structured sparse coding, and aggregate the reconstructed blocks in all block groups to obtain the target image corresponding to the image to be processed.

9. A terminal comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the image denoising method according to any one of claims 1 to 7 are implemented.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the image denoising method according to any one of claims 1 to 7 are implemented.

Images