An image denoising method, device, equipment and medium

Abstract

The present invention discloses an image denoising method, apparatus, device and medium. In the present invention, after determining a first column vector composed of first weight values of N candidate image blocks with respect to a reference image block, and a first matrix composed of pixel values of pixel points in the N candidate image blocks, the non-local means denoising result of the reference image block is determined according to the first matrix and the first column vector, and the non-local means denoising result is corrected using the variance and gradient information of the reference image block, thereby realizing the retention of image edge information and the removal of noise in flat areas, and thus obtaining a better image denoising effect.

Description

Technical Field

[0001] The present invention relates to the technical field of image denoising, and in particular, to an image denoising method, device, equipment and medium. Background Art

[0002] Images are widely applied in fields such as biomedicine, military, traffic security, and machine vision. The quality of images directly affects their effectiveness in these applications. However, during the acquisition, processing, and transmission of images, they are inevitably interfered by noise. Therefore, filtering the noise in images is of extremely important significance.

[0003] The existing mainstream image denoising techniques mainly include spatial domain image denoising, frequency domain image denoising, hybrid spatial and frequency domain image denoising, and learning-based image denoising.

[0004] Simple spatial domain image denoising algorithms include mean filtering, Gaussian filtering, median filtering, etc. These denoising algorithms do not distinguish between boundary regions and flat regions, so a lot of useful image information will be lost during denoising. Slightly more complex algorithms include bilateral filtering and non-local mean algorithms; however, the disadvantage of spatial domain algorithms is that they do not protect textures well.

[0005] The main representatives of frequency domain image denoising are wavelet, DCT and other denoising algorithms. These algorithms transform the image into a frequency domain image and perform denoising on the frequency domain image. The disadvantage of frequency domain image denoising is that suppressing large noise is not as good as spatial domain denoising.

[0006] Hybrid spatial and frequency domain image denoising mainly includes algorithms such as BM3D and WNNM. These algorithms combine spatial domain and frequency domain denoising, greatly improving the image denoising ability.

[0007] Learning-based image denoising mainly includes deep learning and dictionary learning-based denoising algorithms. These algorithms use pre-collected samples to train the denoising algorithm and achieve relatively good results, but the computational complexity is huge and it is difficult to apply to actual products.

[0008] In existing image denoising systems, methods and media, the design key point is to decompose the image, perform non-local averaging denoising on the decomposed sub-images, and finally reconstruct the denoised image; however, non-local mean is affected by the size of the matching block, and the calculated weights are often sub-optimal, resulting in poor denoising effects. Summary of the Invention

[0009] The present invention provides an image denoising method, device, equipment and medium to solve the problem of poor denoising effect in the prior art.

[0010] The present invention provides an image denoising method, the method comprising:

[0011] Divide the image to be denoised into M reference image blocks to be denoised;

[0012] Search for N candidate image blocks around each of the reference image blocks in sequence, where there is an overlapping area between adjacent candidate image blocks;

[0013] For each of the reference image blocks, determine the first weight value of the N candidate image blocks for the reference image block, and form the first weight values of the N candidate image blocks into a first column vector. According to the second column vector composed of the pixel values of the pixel points in the N candidate image blocks, determine the first matrix of the image to be denoised. According to the first matrix and the first column vector, determine the third column vector composed of the pixel values of the denoised reference image block. According to the pixel values of the pixel points in the reference image block, determine the variance of the reference image block. According to the variance and the first gradient matrix in the horizontal direction and the second gradient matrix in the vertical direction of the reference image block pre-stored, correct the third column vector to obtain the corrected third column vector;

[0014] According to the corrected third column vectors corresponding to each target reference image block where the pixel points in the image to be denoised are located, perform direct averaging to obtain the denoised target image.

[0015] Further, the determining the first weight value of the N candidate image blocks for the reference image block includes:

[0016] According to Determine the first weight value of the output N candidate image blocks for the reference image block, where ref i represents the pixel value of the i-th pixel point in the reference image block, j is the candidate image block identifier, and A j,i represents the pixel value of the i-th pixel point in the j-th candidate image block, N represents the number of pixel points included in the reference image block, and Sigma is a preset value configured in advance.

[0017] Further, after determining the first weight value of the N candidate image blocks for the reference image block and before forming the first weight values of the N candidate image blocks into a first column vector, the method further includes:

[0018] For the N candidate image blocks, determine the ratio of the first weight value of the N candidate image blocks for the reference image block and the sum of the weights of the first weight values of the N candidate image blocks for the reference image block, and use the ratio as the updated first weight value of the N candidate image blocks for the reference image block.

[0019] Further, the determination of the third column vector composed of the pixel values of the denoised reference image block according to the first matrix and the first column vector includes:

[0020] Determine the third column vector composed of the pixel values of the denoised reference image block according to deRef1 = AW, where A represents the first matrix, W represents the first column vector, and deRef1 represents the third column vector;

[0021] The correction of the third column vector according to the variance and the pre - saved first gradient matrix in the horizontal direction and the second gradient matrix in the vertical direction of the reference image block to obtain the corrected third column vector includes:

[0022] According to Determine the third column vector composed of the pixel values of the output denoised reference image block, where D x represents the first gradient matrix, D y represents the second gradient matrix, AW represents the third column vector, D(ref) represents the variance, I represents the identity matrix, deRef2 represents the corrected third column vector, and α represents a preset value.

[0023] Further, the direct averaging according to the corrected third column vector corresponding to each target reference image block where the pixel points in the image to be denoised are located to obtain the denoised target image includes:

[0024] For each pixel point in the image to be denoised, according to each target reference image block where the pixel point is located and the corrected third column vector corresponding to each denoised target reference image block, determine each element value of the element points in the corrected third column vector corresponding to the position of the pixel point in each target reference image block, and determine the average value of the determined each element value as the pixel value of the pixel point after denoising.

[0025] Correspondingly, the present invention provides an image denoising device, and the device includes:

[0026] A division module, configured to divide the image to be denoised into M reference image blocks to be denoised;

[0027] A search module, configured to sequentially search for N candidate image blocks around each of the reference image blocks, where there is an overlapping area between adjacent candidate image blocks;

[0028] A noise reduction module, which is used to determine, for each of the reference image blocks, a first weight value of the N candidate image blocks with respect to the reference image block, and form the first weight values of the N candidate image blocks into a first column vector. According to a second column vector composed of pixel values of pixel points in the N candidate image blocks, a first matrix of the image to be denoised is determined. According to the first matrix and the first column vector, a third column vector composed of pixel values of the denoised reference image block is determined. According to the pixel values of pixel points in the reference image block, the variance of the reference image block is determined. According to the variance and the first gradient matrix in the horizontal direction and the second gradient matrix in the vertical direction of the reference image block pre-stored, the third column vector is corrected to obtain a corrected third column vector;

[0029] A determination module, which is used to directly average according to the corrected third column vectors corresponding to each target reference image block where the pixel points in the image to be denoised are located, to obtain a denoised target image.

[0030] Further, the noise reduction module is specifically used to according to determine the first weight value of the output N candidate image blocks with respect to the reference image block, where ref i represents the pixel value of the i-th pixel point in the reference image block, j is the candidate image block identifier, A j,i represents the pixel value of the i-th pixel point in the j-th candidate image block, N represents the number of pixel points included in the reference image block, and Sigma is a preset value configured in advance.

[0031] Further, after the noise reduction module determines the first weight value of the N candidate image blocks with respect to the reference image block and before forming the first weight values of the N candidate image blocks into a first column vector, for the N candidate image blocks, it determines the first weight value of the N candidate image blocks with respect to the reference image block and the ratio of the sum of weights of the first weight values of the N candidate image blocks with respect to the reference image block, and uses the ratio as the updated first weight value of the N candidate image blocks with respect to the reference image block.

[0032] Further, the noise reduction module is specifically used to determine, according to deRef1 = AW, a third column vector composed of pixel values of the denoised reference image block, where A represents the first matrix, W represents the first column vector, and deRef1 represents the third column vector; according to determine a third column vector composed of pixel values of the output denoised reference image block, where D x represents the first gradient matrix, D yIt represents the second gradient matrix, AW represents the third column vector, D(ref) represents the variance, I represents the identity matrix, deRef2 represents the corrected third column vector, and α represents a preset value.

[0033] Further, the determining module is specifically configured to, for each pixel point in the image to be denoised, determine each element value of the element points in the corrected third column vector corresponding to the position of the pixel point in each target reference image block according to each target reference image block where the pixel point is located and the corrected third column vector corresponding to each denoised target reference image block, and determine the mean value of the determined each element value as the pixel value of the pixel point after denoising.

[0034] Correspondingly, the present invention provides an electronic device, including: a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory complete communication with each other through the communication bus;

[0035] A computer program is stored in the memory, and when the program is executed by the processor, the processor is caused to implement the steps of any one of the above image denoising methods.

[0036] Correspondingly, the present invention provides a computer-readable storage medium, which stores a computer program, and when the computer program is executed by a processor, the steps of any one of the above image denoising methods are implemented.

[0037] The present invention provides an image denoising method, apparatus, device and medium. In this method, the denoised image is divided into M reference image blocks to be denoised, and N candidate image blocks are searched around each reference image block in turn, where there is an overlapping area between adjacent candidate image blocks; for each reference image block, determine the first weight value of the N candidate image blocks with respect to the reference image block, and form the first column vector with the first weight values of the N candidate image blocks. Determine the first matrix of the image to be denoised according to the second column vector composed of the pixel values of the pixel points in the N candidate image blocks. According to the first matrix and the first column vector, determine the third column vector composed of the pixel values of the denoised reference image block. According to the pixel values of the pixel points in the reference image block, determine the variance of the reference image block. Correct the third column vector according to the variance and the pre-stored first gradient matrix in the horizontal direction and the second gradient matrix in the vertical direction of the reference image block to obtain the corrected third column vector; perform direct averaging according to the corrected third column vector corresponding to each target reference image block where the pixel points in the image to be denoised are located to obtain the denoised target image; since in the present invention, after determining the first column vector composed of the first weight values of the N candidate image blocks with respect to the reference image block and the first matrix composed of the pixel values of the pixel points in the N candidate image blocks, determine the non-local mean denoising result of the reference image block according to the first matrix and the first column vector, and use the variance and gradient information of the reference image block to correct the non-local mean denoising result, so as to retain the edge information of the image and remove the noise in the flat area, thereby obtaining a better image denoising effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0039] Figure 1 It is a schematic diagram of the process of an image denoising method provided by an embodiment of the present invention;

[0040] Figure 2 It is a schematic diagram of dividing an image to be denoised into M reference image blocks to be denoised provided by an embodiment of the present invention;

[0041] Figure 3 It is a schematic diagram of searching for a reference image block provided by an embodiment of the present invention;

[0042] Figure 4 It is a schematic diagram of candidate image blocks searched around a reference image provided by an embodiment of the present invention;

[0043] Figure 5 Schematic diagram of a reference image block ref provided by an embodiment of the present invention;

[0044] Figure 6 Schematic diagram of a candidate image block A provided by an embodiment of the present invention;

[0045] Figure 7 Schematic structural diagram of an image noise reduction device provided by an embodiment of the present invention;

[0046] Figure 8 Schematic structural diagram of an electronic device provided by an embodiment of the present invention. Specific embodiments

[0047] In order to make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the scope of protection of the present invention.

[0048] In order to improve the image noise reduction effect, an embodiment of the present invention provides an image noise reduction method, device, equipment, and medium.

[0049] Embodiment 1:

[0050] Figure 1 Schematic process diagram of an image noise reduction method provided by an embodiment of the present invention. The process includes the following steps:

[0051] S101: Divide the image to be denoised into M reference image blocks to be denoised.

[0052] In order to improve the image noise reduction effect, an image noise reduction method provided by an embodiment of the present invention is applied to an electronic device. The electronic device may be an intelligent terminal device such as a host, a tablet computer, a notebook computer, or a smart phone, or may be a server. The server may be a local server or a cloud server. The embodiments of the present invention do not limit this.

[0053] The electronic device acquires the image to be denoised. Specifically, the image to be denoised may be acquired by the image acquisition device of the electronic device itself, or may be received from other devices connected to the electronic device, or may be the image to be denoised pre-stored in the electronic device itself. The embodiments of the present invention do not limit this;

[0054] The denoising image can be divided by moving an image frame of a preset size one pixel at a time in the denoising image to determine M reference image blocks to be denoised, where the size of M is determined according to the size of the denoising image and the preset size; alternatively, a preset value M can be set in advance, and the target size of each divided reference image block is determined according to the size of M and the size of the denoising image, and an image frame of the target size is moved one pixel at a time in the denoising image to determine M reference image blocks to be denoised.

[0055] Figure 2 FIG. is a schematic diagram of dividing a denoising image into M reference image blocks to be denoised provided by an embodiment of the present invention. As Figure 2 shown, the size of the denoising image is 5×5, and the denoising image is divided into 9 reference image blocks to be denoised, and the size of each reference image block is 3×3.

[0056] S102: Search for N candidate image blocks around each of the reference image blocks in sequence, where there is an overlapping area between adjacent candidate image blocks.

[0057] After dividing the denoising image into M reference image blocks to be denoised, for each reference image block in sequence, search around the reference image block and search for N candidate image blocks, where N is a preset value, the size of the candidate image block is the same as the size of the reference image block, and there is an overlapping area between adjacent candidate image blocks; specifically, search within a preset range of the reference image, and the reference image block is located at the center of the preset range.

[0058] Figure 3 FIG. is a schematic diagram of searching for a reference image block provided by an embodiment of the present invention. As Figure 3 shown, the black area in the figure is the reference image block, the search range is 5×5, and the size of the reference image block is 3×3. Therefore, a total of 25 candidate image blocks are searched.

[0059] Figure 4 FIG. is a schematic diagram of candidate image blocks searched around a reference image provided by an embodiment of the present invention. As Figure 4 shown, 25 3x3 candidate image blocks are searched within a 5×5 range. Figure 4 The dark area of each sub-image in it represents a candidate image block.

[0060] S103: For each of the reference image blocks, determine the first weight value of the N candidate image blocks for the reference image block, and form the first weight values of the N candidate image blocks into a first column vector. According to the second column vector composed of the pixel values of the pixel points in the N candidate image blocks, determine the first matrix of the image to be denoised. According to the first matrix and the first column vector, determine the third column vector composed of the pixel values of the denoised reference image block. According to the pixel values of the pixel points in the reference image block, determine the variance of the reference image block. According to the variance and the pre-stored first gradient matrix in the horizontal direction and the second gradient matrix in the vertical direction of the reference image block, correct the third column vector to obtain a corrected third column vector.

[0061] For each reference image block, in order to determine the first weight value of the N candidate image blocks for the reference image block, the electronic device pre-stores a weight value determination function. For each candidate image block, according to the pixel values of each pixel point in the reference image block and the pixel values of each pixel point in the candidate image block, input the pixel values of each pixel point in the reference image block and the pixel values of each pixel point in the candidate image block into the weight value determination function to determine the output first weight value of the candidate image block; and according to the first weight values of the N candidate image blocks, form the first weight values of each candidate image block into a first column vector in sequence.

[0062] According to the pixel values of the pixel points in the N candidate images, form the pixel values of the pixel points in the N candidate image blocks into N second column vectors, and according to the N second column vectors of the N candidate image blocks, sort them in the row direction to form the first matrix of the image to be denoised.

[0063] According to the first matrix and the first column vector, multiply the first matrix by the first column vector to obtain a third column vector. The third column vector is the non-local mean denoising result of the reference image block, which is composed of the pixel values of the denoised reference image block. Each element value in the third column vector is the pixel value of each pixel point of the denoised reference image block.

[0064] According to the pixel values of the pixel points in the reference image block, determine the variance D(ref) of the reference image block, where D(ref) = E(ref 2 ) - E(ref) × E(ref). For example, when the reference image block is an image block of size 3×3, E(ref) represents the mean of the pixel values of each pixel point in the reference image block before denoising, E(ref 2 ) represents the mean of the squared pixel values of each pixel point in the reference image block before denoising.

[0065] According to the variance, the first gradient matrix in the horizontal direction of the pre - saved reference image block, and the second gradient matrix in the vertical direction, the electronic device corrects the third column vector. Specifically, the electronic device pre - saves a noise reduction optimization ability function, inputs the first gradient matrix in the horizontal direction of the pre - saved reference image block, the second gradient matrix in the vertical direction, the variance of the reference image block, and the third column vector into the noise reduction optimization ability function, and determines the output corrected third column vector, where each element value in the corrected third column vector is each pixel value after correcting the non - local matrix noise reduction result of the reference image block.

[0066] Among them, the first gradient matrix in the horizontal direction of the reference image block and the second gradient matrix in the vertical direction are only related to the size of the reference image block and have nothing to do with the pixel values of the pixel points in the reference image block.

[0067] For example, when the size of the reference image block is 3×3, the first gradient matrix The second gradient matrix The values of the remaining elements in the first gradient matrix and the second gradient matrix except 1 are all 0.

[0068] S104: According to the corrected third column vectors corresponding to each target reference image block where the pixel points in the image to be denoised are located, perform direct averaging to obtain the denoised target image.

[0069] For each pixel point in the image to be denoised, the electronic device determines each target reference image block where the pixel point is located according to each reference image block in the image to be denoised, and determines the corrected third column vectors corresponding to each target reference image block according to the corrected third column vectors corresponding to each reference image block.

[0070] According to each target reference image block where the pixel point is located and the corrected third column vectors corresponding to each denoised target reference image block, perform direct averaging to obtain the denoised target image.

[0071] In the embodiments of the present invention, the noise-reduced image is divided into M reference image blocks to be denoised, and N candidate image blocks are searched around each reference image block in sequence. There is an overlapping area between adjacent candidate image blocks. For each reference image block, the first weight value of the N candidate image blocks with respect to the reference image block is determined, and the first weight values of the N candidate image blocks are formed into a first column vector. According to the second column vector composed of the pixel values of the pixel points in the N candidate image blocks, the first matrix of the image to be denoised is determined. According to the first matrix and the first column vector, the third column vector composed of the pixel values of the denoised reference image block is determined. According to the pixel values of the pixel points in the reference image block, the variance of the reference image block is determined. The third column vector is corrected according to the variance and the first gradient matrix in the horizontal direction and the second gradient matrix in the vertical direction of the reference image block pre-stored to obtain the corrected third column vector. According to the corrected third column vector corresponding to each target reference image block where the pixel points in the image to be denoised are located, direct averaging is performed to obtain the denoised target image. Since in the present invention, after determining the first column vector composed of the first weight values of the N candidate image blocks with respect to the reference image block and the first matrix composed of the pixel values of the pixel points in the N candidate image blocks, the non-local mean denoising result of the reference image block is determined according to the first matrix and the first column vector, and the non-local mean denoising result is corrected using the variance and gradient information of the reference image block, the edge information of the image is retained and the noise in the flat area is removed, thereby obtaining a better image denoising effect.

[0072] Embodiment 2:

[0073] In order to determine the first weight value of the candidate image block with respect to the reference image block, based on the above embodiments, in the embodiments of the present invention, the determining the first weight value of the N candidate image blocks with respect to the reference image block includes:

[0074] According to determine the first weight value of the output N candidate image blocks with respect to the reference image block, where ref i represents the pixel value of the i-th pixel point in the reference image block, j is the candidate image block identifier, A j,i represents the pixel value of the i-th pixel point in the j-th candidate image block, N represents the number of pixel points included in the reference image block, and Sigma is a preset value configured in advance.

[0075] The weight value determination function is ref i represents the pixel value of the i-th pixel point in the reference image block, j is the candidate image block identifier, A j,irepresents the pixel value of the i-th pixel in the j-th candidate image block, N represents the number of pixels in the reference image block, Sigma is a configuration parameter that is adjusted according to the noise level; for example, for a 3x3 reference image block, N represents 9, and for a 5x5 search range, the range of j is from 1 to 25. According to the weight value determination function, the first weight value of the N output candidate image blocks with respect to the reference image block is determined.

[0076] Figure 5 is a schematic diagram of a reference image block ref provided by an embodiment of the present invention, as Figure 5 shown, the reference image contains a total of 9 pixels; Figure 6 is a schematic diagram of candidate image block A provided by an embodiment of the present invention, as Figure 6 shown, the candidate image block is located in the upper left corner of the image to be denoised.

[0077] That is, for each candidate image block, the electronic device determines, according to the pixel values of the pixels in the reference image block and the squared value of the absolute difference between the pixel values of the corresponding positions in the candidate image block, the sum value of the squared values corresponding to each pixel and the product value of the number of pixels in the reference image block and the squared value of the configuration parameter sigma, determines the first negative number of the ratio of the sum value to the squared value, and calculates the result value of the negative power of the natural constant e to the first negative number, and determines the result value as the first weight value of the candidate image block.

[0078] Embodiment 3:

[0079] To improve the image denoising effect, based on the above embodiment, in an embodiment of the present invention, after determining the first weight value of the N candidate image blocks with respect to the reference image block and before forming the first weight values of the N candidate image blocks into a second column vector, the method further includes:

[0080] For the N candidate image blocks, determine the ratio of the first weight value of the N candidate image blocks with respect to the reference image block and the sum value of the weights of the first weight values of the N candidate image blocks with respect to the reference image block, and use the ratio as the updated first weight value of the N candidate image blocks with respect to the reference image block.

[0081] To improve the image denoising effect, the electronic device normalizes the first weight values of the N candidate image blocks. Specifically, according to the first weight values of the N candidate image blocks with respect to the reference image block, the sum value of the weights of each first weight value is determined. For each candidate image block, the electronic device determines the ratio of the first weight value of the candidate image block with respect to the reference image block to the sum value of the weights, and determines the ratio as the updated first weight value of the candidate image block with respect to the reference image block, thereby realizing the normalization of the first weight values of the N candidate image blocks.

[0082] For example, the electronic device determines, according to a pre-stored function, a first weight value after updating each candidate image block for a reference image block, where weight on the left side of the equal sign j represents the first weight value after updating the j-th candidate image block, and weight on the right side of the equal sign j represents the first weight value of the j-th candidate image block. M is the number of candidate image blocks. For example, in a search range of 5x5, M is 25.

[0083] Embodiment 4:

[0084] To determine a third column vector composed of pixel values of the denoised reference image block, based on the above embodiments, in an embodiment of the present invention, determining the third column vector composed of pixel values of the denoised reference image block according to the first matrix and the second column vector includes:

[0085] Determining the third column vector composed of pixel values of the denoised reference image block according to deRef1 = AW, where A represents the first matrix, W represents the second column vector, and deRef1 represents the third column vector;

[0086] Correcting the third column vector according to the variance and the pre-stored first gradient matrix in the horizontal direction and second gradient matrix in the vertical direction of the reference image block to obtain a corrected third column vector, which includes:

[0087] According to determining the third column vector composed of pixel values of the output denoised reference image block, where D x represents the first gradient matrix, D y represents the second gradient matrix, AW represents the third column vector, D(ref) represents the variance, I represents the identity matrix, deRef2 represents the corrected third column vector, and α represents a preset value.

[0088] Calculating the pixel value of the denoised reference image block according to the non-local mean weight of each candidate image block. Specifically, according to determining the third column vector composed of pixel values of the denoised reference image block, where j represents the identification information of the candidate image block, i represents the position identification information of the pixel point in the image block, and weight j represents the first weight value corresponding to the j-th candidate image block, and A j,i represents the pixel value of the pixel point at the i-th position of the j-th candidate image block. Simplifying the above formula into matrix and vector form is expressed as deRef1 = AW, where A represents the first matrix, W represents the second column vector, and deRef1 represents the third column vector.

[0089] Since the deRef denoising result obtained by calculating the non-local mean weight has problems such as incomplete denoising and blurred edges; the electronic device uses the pre-saved denoising optimization energy function

[0090]

[0091] , the two-dimensional reference image block is expanded into a one-dimensional column vector. The right side of the equal sign represents deRef2 when the value calculated within the curly brackets is minimized. Here, deRef represents the third column vector composed of preset pixel values in the preset denoised reference image block, represents the square value of the vector bisector of the difference column vector. α is a preset value configured externally. When α increases, the influence of the gradient and variance information of the reference image block on the denoising result increases. D x and D y are respectively used to represent the first gradient matrix in the x direction and the second gradient matrix in the y direction of the reference image block. D(ref) is the variance of the reference image block before denoising.

[0092] The denoising optimization energy function is simplified and represented as a matrix and vector as

[0093] A is the first matrix of N×M. N represents the number of pixel points included in each candidate image block, and M represents the number of candidate image blocks. Each column of A represents a candidate image block; W is the first column vector of M×1, and each element value in W is the first weight value of each candidate image block.

[0094] After expanding the above simplified denoising optimization energy function, we can get To determine deRef2 when the value within the curly brackets on the right side of the equal sign is minimized, we also take the derivative of deRef2 in the above formula, After simplification, we get That is Therefore When the derivative function calculated based on deRef is zero, the value within the curly brackets on the right side of the equal sign is the smallest.

[0095] According to the finally obtained denoising optimization energy function The first gradient matrix in the horizontal direction, the second gradient matrix in the vertical direction, the first matrix, the first column vector composed of the weight values of each candidate image block, and the variance of the pre-stored reference image block are input into the noise reduction optimization energy function to obtain the third column vector composed of the pixel values of the denoised reference image block as the output; AW represents the third column vector, D(ref) represents the variance, I represents the identity matrix, and deRef2 represents the corrected third column vector.

[0096] Embodiment 5:

[0097] In order to determine the pixel value of each pixel point after denoising, based on the above embodiments, in the embodiment of the present invention, the direct averaging of the corrected third column vectors corresponding to each target reference image block where the pixel point in the image to be denoised is located to obtain the denoised target image includes:

[0098] For each pixel point in the image to be denoised, according to each target reference image block where the pixel point is located, and the corrected third column vectors corresponding to each denoised target reference image block, determine each element value of the element points in the corrected third column vector corresponding to the position of the pixel point in each target reference image block, and determine the average value of the determined each element value as the pixel value of the pixel point after denoising.

[0099] For each pixel point in the image to be denoised, the electronic device determines each element value of the element points in the corrected third column vector corresponding to the position of the pixel point in each target reference image block according to each target reference image block where the pixel point is located, and the corrected third column vectors corresponding to each denoised target reference image block, and determines each element value as each pixel value of the pixel point after denoising in each target reference image block.

[0100] According to each pixel value of the pixel point after denoising in each target reference image block, determine the average value of each pixel value, and determine the average value as the pixel value of the pixel point after denoising.

[0101] As Figure 2 shown, the pixel point at the third row and third column in the image to be denoised exists in 9 reference image blocks. Therefore, the direct averaging method is used to obtain the final denoised target image. Assume that the overlapping blocks involved in the coordinate point i are deRef1, deRef2,..., deRefM. Therefore, the pixel value of the coordinate point i in the denoised target image where M is the number of overlapping target reference image blocks involved in the coordinate point i, and deRefj i represents the pixel value of the pixel point of the coordinate point i in the jth reference image block.

[0102] Embodiment 6:

[0103] Figure 7 The following is a schematic structural diagram of an image noise reduction device provided by an embodiment of the present invention. As Figure 7 shown, the device includes:

[0104] A partitioning module 701, configured to partition the image to be noise-reduced into M reference image blocks to be noise-reduced;

[0105] A searching module 702, configured to sequentially search for N candidate image blocks around each of the reference image blocks, where there is an overlapping area between adjacent candidate image blocks;

[0106] A noise reduction module 703, configured to, for each of the reference image blocks, determine a first weight value of the N candidate image blocks for the reference image block, and form a first column vector with the first weight values of the N candidate image blocks. According to a second column vector composed of pixel values of pixel points in the N candidate image blocks, determine a first matrix of the image to be noise-reduced. According to the first matrix and the first column vector, determine a third column vector composed of pixel values of the noise-reduced reference image block. According to the pixel values of pixel points in the reference image block, determine the variance of the reference image block. According to the variance and a first gradient matrix in the horizontal direction and a second gradient matrix in the vertical direction of the reference image block pre-stored, correct the third column vector to obtain a corrected third column vector;

[0107] A determination module 704, configured to directly average according to the corrected third column vectors corresponding to each target reference image block where the pixel points in the image to be noise-reduced are located to obtain a noise-reduced target image.

[0108] Further, the noise reduction module 703 is specifically configured to determine, according to the first weight value of the output N candidate image blocks for the reference image block, where ref i represents the pixel value of the i-th pixel point in the reference image block, j is the candidate image block identifier, and A j,i represents the pixel value of the i-th pixel point in the j-th candidate image block, N represents the number of pixel points included in the reference image block, and Sigma is a preset value pre-configured.

[0109] Further, after the noise reduction module 703 determines the first weight values of the N candidate image blocks with respect to the reference image block, and before forming the first weight values of the N candidate image blocks into a first column vector, for the N candidate image blocks, it determines the ratio of the first weight values of the N candidate image blocks with respect to the reference image block and the sum of the weights of the first weight values of the N candidate image blocks with respect to the reference image block, and uses the ratio as the updated first weight values of the N candidate image blocks with respect to the reference image block.

[0110] Further, the noise reduction module 703 is specifically configured to determine a third column vector composed of the pixel values of the denoised reference image block according to deRef1 = AW, where A represents the first matrix, W represents the first column vector, and deRef1 represents the third column vector; according to determine a third column vector composed of the pixel values of the output denoised reference image block, where D x represents the first gradient matrix, D y represents the second gradient matrix, AW represents the third column vector, D(ref) represents the variance, I represents the identity matrix, deRef2 represents the corrected third column vector, and α represents a preset value.

[0111] Further, the determination module 704 is specifically configured to, for each pixel point in the image to be denoised, determine each element value of the element points in the corrected third column vector corresponding to the position of the pixel point in each target reference image block according to each target reference image block where the pixel point is located and the corrected third column vector corresponding to each denoised target reference image block, and determine the mean value of the determined each element value as the pixel value of the pixel point after denoising.

[0112] Embodiment 7:

[0113] Figure 8 The following is a schematic structural diagram of an electronic device provided by an embodiment of the present invention. On the basis of the above embodiments, the present application further provides an electronic device, as Figure 8 shown, including: a processor 801, a communication interface 802, a memory 803, and a communication bus 804. Among them, the processor 801, the communication interface 802, and the memory 803 complete communication with each other through the communication bus 804.

[0114] A computer program is stored in the memory 803. When the program is executed by the processor 801, the processor 801 is caused to execute the following steps:

[0115] Divide the image to be denoised into M reference image blocks to be denoised;

[0116] Search for N candidate image patches around each of the reference image patches in sequence, where there is an overlapping area between adjacent candidate image patches;

[0117] For each of the reference image patches, determine the first weight value of the N candidate image patches with respect to the reference image patch, and form the first weight values of the N candidate image patches into a first column vector. According to the second column vector composed of the pixel values of the pixel points in the N candidate image patches, determine the first matrix of the image to be denoised. According to the first matrix and the first column vector, determine the third column vector composed of the pixel values of the denoised reference image patch. According to the pixel values of the pixel points in the reference image patch, determine the variance of the reference image patch. According to the variance and the first gradient matrix in the horizontal direction and the second gradient matrix in the vertical direction of the reference image patch pre-stored, correct the third column vector to obtain a corrected third column vector;

[0118] According to the corrected third column vectors corresponding to each target reference image patch where the pixel points in the image to be denoised are located, perform direct averaging to obtain the denoised target image.

[0119] Further, the processor 801 is specifically used for the determination of the first weight value of the N candidate image patches with respect to the reference image patch, including:

[0120] According to Determine the first weight value of the output N candidate image patches with respect to the reference image patch, where ref i Represents the pixel value of the i-th pixel point in the reference image patch, j is the candidate image patch identifier, and A j,i Represents the pixel value of the i-th pixel point in the j-th candidate image patch, N represents the number of pixel points included in the reference image patch, and Sigma is a preset value configured in advance.

[0121] Further, after the processor 801 is further used for the determination of the first weight value of the N candidate image patches with respect to the reference image patch and before the first weight values of the N candidate image patches are formed into a first column vector, the method further includes:

[0122] For the N candidate image patches, determine the first weight value of the N candidate image patches with respect to the reference image patch and the ratio of the sum of the weights of the first weight values of the N candidate image patches with respect to the reference image patch, and use the ratio as the updated first weight value of the N candidate image patches with respect to the reference image patch.

[0123] Further, the processor 801 is specifically configured to determine that the third column vector composed of the pixel values of the denoised reference image block according to the first matrix and the first column vector includes:

[0124] Determine the third column vector composed of the pixel values of the denoised reference image block according to deRef1 = AW, where A represents the first matrix, W represents the first column vector, and deRef1 represents the third column vector;

[0125] The correction of the third column vector according to the variance and the pre-stored first gradient matrix in the horizontal direction and the second gradient matrix in the vertical direction of the reference image block to obtain the corrected third column vector includes:

[0126] According to Determine the third column vector composed of the pixel values of the output denoised reference image block, where D x Represents the first gradient matrix, D y Represents the second gradient matrix, AW represents the third column vector, D(ref) represents the variance, I represents the identity matrix, deRef2 represents the corrected third column vector, and α represents a preset value.

[0127] Further, the processor 801 is specifically configured to directly average according to the corrected third column vector corresponding to each target reference image block where the pixel points in the image to be denoised are located to obtain the denoised target image, including:

[0128] For each pixel point in the image to be denoised, according to each target reference image block where the pixel point is located, and the corrected third column vector corresponding to each denoised target reference image block, determine each element value of the element points in the corrected third column vector corresponding to the position of the pixel point in each target reference image block, and determine the average value of the determined each element value as the pixel value of the pixel point after denoising.

[0129] The communication bus mentioned in the above electronic device may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into an address bus, a data bus, a control bus, etc. For the sake of representation, only a thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0130] The communication interface 802 is used for communication between the above electronic device and other devices.

[0131] The memory may include a Random Access Memory (RAM), or may also include a Non-Volatile Memory (NVM), such as at least one disk memory. Optionally, the memory may also be at least one storage device located far from the aforementioned processor.

[0132] The aforementioned processor may be a general-purpose processor, including a central processing unit, a Network Processor (NP), etc.; it may also be a Digital Signal Processing (DSP), an application-specific integrated circuit, a field-programmable gate array, or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0133] Embodiment 8:

[0134] Based on the above embodiments, the present application also provides a computer-readable storage medium, in which a computer program executable by a processor is stored. When the program runs on the processor, the processor is caused to perform the following steps when executing:

[0135] Divide the image to be denoised into M reference image blocks to be denoised;

[0136] Search for N candidate image blocks around each of the reference image blocks in sequence, where there is an overlapping area between adjacent candidate image blocks;

[0137] For each of the reference image blocks, determine the first weight value of the N candidate image blocks for the reference image block, and form the first weight values of the N candidate image blocks into a first column vector. According to the second column vector composed of the pixel values of the pixel points in the N candidate image blocks, determine the first matrix of the image to be denoised. According to the first matrix and the first column vector, determine the third column vector composed of the pixel values of the denoised reference image block. According to the pixel values of the pixel points in the reference image block, determine the variance of the reference image block. According to the variance and the pre-stored first gradient matrix in the horizontal direction and the second gradient matrix in the vertical direction of the reference image block, correct the third column vector to obtain a corrected third column vector;

[0138] Perform direct averaging according to the corrected third column vectors corresponding to each target reference image block where the pixel points in the image to be denoised are located to obtain a denoised target image.

[0139] Further, the determining the first weight value of the N candidate image blocks for the reference image block includes:

[0140] According to Determine the first weight value of the N candidate image patches with respect to the reference image patch, where ref i represents the pixel value of the i-th pixel point in the reference image patch, j is the candidate image patch identifier, and A j,i represents the pixel value of the i-th pixel point in the j-th candidate image patch, N represents the number of pixel points included in the reference image patch, and Sigma is a pre-configured preset value.

[0141] Further, after determining the first weight value of the N candidate image patches with respect to the reference image patch and before forming the first column vector of the first weight values of the N candidate image patches, the method further includes:

[0142] For the N candidate image patches, determine the ratio of the first weight value of the N candidate image patches with respect to the reference image patch and the sum of the weights of the first weight values of the N candidate image patches with respect to the reference image patch, and use the ratio as the updated first weight value of the N candidate image patches with respect to the reference image patch.

[0143] Further, the determining the third column vector composed of the pixel values of the denoised reference image patch according to the first matrix and the first column vector includes:

[0144] According to deRef1 = AW, determine the third column vector composed of the pixel values of the denoised reference image patch, where A represents the first matrix, W represents the first column vector, and deRef1 represents the third column vector;

[0145] The correcting the third column vector according to the variance and the pre-stored first gradient matrix in the horizontal direction and the second gradient matrix in the vertical direction of the reference image patch to obtain the corrected third column vector includes:

[0146] According to Determine the third column vector composed of the pixel values of the output denoised reference image patch, where D x represents the first gradient matrix, D y represents the second gradient matrix, AW represents the third column vector, D(ref) represents the variance, I represents the identity matrix, deRef2 represents the corrected third column vector, and α represents a preset value.

[0147] Further, the directly averaging according to the corrected third column vector corresponding to each target reference image patch where the pixel points in the image to be denoised are located to obtain the denoised target image includes:

[0148] For each pixel point in the image to be denoised, according to each target reference image block where the pixel point is located, and the corrected third column vector corresponding to each target reference image block after denoising, determine each element value of the element points in the corrected third column vector corresponding to the position of the pixel point in each target reference image block, and determine the mean value of the determined each element value as the pixel value of the pixel point after denoising.

[0149] Those skilled in the art should understand that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0150] The present application is described with reference to the flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to the present application. It should be understood that each process and / or block in the flowchart and / or block diagram, and the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing devices generate means for implementing the specified functions in Figure 1 one process or multiple processes and / or blocks Figure 1 one block or multiple blocks.

[0151] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory generate a manufactured article including an instruction device, and the instruction device implements the specified functions in Figure 1 one process or multiple processes and / or blocks Figure 1 one block or multiple blocks.

[0152] These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to generate a computer-implemented process, and thus the instructions executed on the computer or other programmable device provide steps for implementing the specified functions in Figure 1 one process or multiple processes and / or blocks Figure 1 one block or multiple blocks.

[0153] Obviously, those skilled in the art can make various changes and modifications to this application without departing from the spirit and scope of this application. Thus, if these modifications and variations of this application fall within the scope of the claims of this application and their equivalent technologies, this application is also intended to include these modifications and variations.

Claims

1. An image noise reduction method, characterized in that, The method includes: Dividing the image to be denoised into M reference image blocks to be denoised; Sequentially searching for N candidate image blocks around each of the reference image blocks, where there is an overlapping area between adjacent candidate image blocks; For each of the reference image blocks, determining a first weight value of the N candidate image blocks for the reference image block, and forming the first weight values of the N candidate image blocks into a first column vector. According to a second column vector composed of pixel values of pixel points in the N candidate image blocks, determining a first matrix of the image to be denoised. According to the first matrix and the first column vector, determining a third column vector composed of pixel values of the denoised reference image block. According to the pixel values of pixel points in the reference image block, determining the variance of the reference image block. According to the variance and a first gradient matrix in the horizontal direction and a second gradient matrix in the vertical direction of the reference image block pre - stored, correcting the third column vector to obtain a corrected third column vector; According to the corrected third column vectors corresponding to each target reference image block where the pixel points in the image to be denoised are located, directly averaging to obtain a denoised target image.

2. The method according to claim 1, wherein The determining the first weight value of the N candidate image blocks for the reference image block includes: According to determine a first weight value of the N candidate image patches for the reference image patch, where ref i represents the pixel value of the i-th pixel point in the reference image patch, j is the candidate image patch identifier, A j,i represents the pixel value of the i-th pixel point in the j-th candidate image patch, N represents the number of pixel points included in the reference image patch, and Sigma is a preset value configured in advance.

3. The method according to claim 1, characterized in that, After determining the first weight value of the N candidate image blocks for the reference image block and before forming the first weight values of the N candidate image blocks into a first column vector, the method further includes: For the N candidate image blocks, determining the ratio of the first weight value of the N candidate image blocks for the reference image block and the sum of weights of the first weight values of the N candidate image blocks for the reference image block, and using the ratio as the updated first weight value of the N candidate image blocks for the reference image block.

4. The method according to claim 1, characterized in that, The determining the third column vector composed of pixel values of the denoised reference image block according to the first matrix and the first column vector includes: Determining the third column vector composed of pixel values of the denoised reference image block according to deRef1 = AW, where A represents the first matrix, W represents the first column vector, and deRef1 represents the third column vector; The correcting the third column vector according to the variance and a first gradient matrix in the horizontal direction and a second gradient matrix in the vertical direction of the reference image block pre - stored to obtain a corrected third column vector includes: According to Determine the third column vector composed of the pixel values of the denoised reference image block of the output, where D x represents the first gradient matrix, D y represents the second gradient matrix, AW represents the third column vector, D(ref) represents the variance, I represents the identity matrix, deRef2 represents the corrected third column vector, and α represents a preset value.

5. The method according to claim 1, wherein The directly averaging according to the corrected third column vectors corresponding to each target reference image block where the pixel points in the image to be denoised are located to obtain a denoised target image includes: For each pixel point in the image to be denoised, according to each target reference image block where the pixel point is located and the corrected third column vectors corresponding to each denoised target reference image block, determining each element value of the element points in the corrected third column vectors corresponding to the position of the pixel point in each target reference image block, and determining the mean value of the determined each element value as the pixel value of the denoised pixel point.

6. An image noise reduction device, characterized in that, The device includes: A partitioning module, configured to partition an image to be denoised into M reference image blocks to be denoised; A searching module, configured to sequentially search for N candidate image blocks around each of the reference image blocks, where there is an overlapping area between adjacent candidate image blocks; A denoising module, configured to, for each of the reference image blocks, determine a first weight value of the N candidate image blocks for the reference image block, and form the first weight values of the N candidate image blocks into a first column vector, determine a first matrix of the image to be denoised according to a second column vector formed by pixel values of pixel points in the N candidate image blocks, determine a third column vector formed by pixel values of the denoised reference image block according to the first matrix and the first column vector, determine the variance of the reference image block according to the pixel values of pixel points in the reference image block, and correct the third column vector according to the variance and a first gradient matrix in the horizontal direction and a second gradient matrix in the vertical direction of the reference image block pre-stored to obtain a corrected third column vector; A determining module, configured to directly average according to the corrected third column vectors corresponding to each target reference image block where the pixel points in the image to be denoised are located to obtain a denoised target image.

7. The device according to claim 6, characterized in that, The noise reduction module is specifically configured to, according to determine a first weight value of the N candidate image patches output with respect to the reference image patch, where ref i represents the pixel value of the i-th pixel point in the reference image patch, j is the candidate image patch identifier, and A j,i represents the pixel value of the i-th pixel point in the j-th candidate image patch, N represents the number of pixel points included in the reference image patch, and Sigma is a preset value configured in advance.

8. The device according to claim 6, characterized in that, The denoising module is further configured to, after determining the first weight value of the N candidate image blocks for the reference image block and before forming the first weight values of the N candidate image blocks into a first column vector, for the N candidate image blocks, determine the ratio of the first weight value of the N candidate image blocks for the reference image block and the weighted sum value of the first weight values of the N candidate image blocks for the reference image block, and use the ratio as the updated first weight value of the N candidate image blocks for the reference image block.

9. The device according to claim 6, wherein The noise reduction module is specifically configured to determine a third column vector composed of pixel values of the denoised reference image block according to deRef1 = AW, where A represents the first matrix, W represents the first column vector, and deRef1 represents the third column vector; according to determine a third column vector composed of pixel values of the output denoised reference image block, where D x represents the first gradient matrix, D y represents the second gradient matrix, AW represents the third column vector, D(ref) represents the variance, I represents the identity matrix, deRef2 represents the corrected third column vector, and α represents a preset value.

10. The device according to claim 6, characterized in that, The determining module is specifically configured to, for each pixel point in the image to be denoised, according to each target reference image block where the pixel point is located and the corrected third column vector corresponding to each denoised target reference image block, determine each element value of an element point in the corrected third column vector corresponding to the position of the pixel point in each target reference image block, and determine the average value of the determined each element value as the pixel value of the pixel point after denoising.

11. An electronic device, characterized in that, Comprising: A processor, a communication interface, a memory, and a communication bus, where the processor, the communication interface, and the memory complete communication with each other through the communication bus; The memory stores a computer program, and when the program is executed by the processor, the processor is caused to execute the steps of the image denoising method according to any one of claims 1-5.

12. A computer-readable storage medium, characterized in that, It stores a computer program executable by a processor, and when the program runs on the processor, the processor is caused to execute the steps of the image denoising method according to any one of claims 1-5.

Images