Low-illumination image denoising method based on phase correlation

Abstract

The application discloses a low-illumination image denoising method based on phase correlation, which comprises the following steps: extracting image edge information, performing edge filtering on each frame of image to extract effective edge information of the image; performing edge filtering on two adjacent frames of image, performing phase correlation operation on the two frames of image after filtering to obtain displacement of the two frames of image; performing homologous signal constraint and global correlation constraint on the multiple frames of image after registration to construct cross-correlation equation; solving the cross-correlation equation according to cross-correlation maximization optimization equation and gradient maximization equation, and retaining the solution of the current frame to realize iterative update of the denoised image in the cross-correlation equation of the next frame. The application can effectively remove various noises of the low-illumination image under the low-illumination scene while considering the calculation speed and algorithm stability, restores the real scene, makes the scene clear, and enhances the details.

Description

Technical Field

[0001] This invention belongs to the field of image denoising and enhancement technology, specifically a low-light image denoising method based on phase correlation. Background Technology

[0002] Noise is a significant cause of image interference. Especially in low-light scenes, camera capture can introduce various types of noise that degrade image quality. This noise can be generated during transmission or during processing such as quantization. These noises include additive noise, additive noise, and so on. Common image denoising methods include:

[0003] (1) Filtering-based methods: Filtering can be broadly classified into spatial domain filtering, time domain filtering, and transform domain filtering. Median filtering and mean filtering are the most common spatial domain filters. Their basic principle is to replace the median or mean of the filtering window with the median or mean, which can eliminate isolated noise points. Classical time domain filtering suppresses time-correlated noise across multiple frames. Transform domain filtering is a method that transforms the spatial domain to the frequency domain and then uses a frequency domain filter to remove noise.

[0004] (2) Model-based methods: Model-based methods model the distribution of captured images and noise separately, using the model distribution as a prior to obtain clear images and optimize algorithms. Relatively successful image prior models include: Non-Local Self-Similarity (NSS) models, sparse models, gradient models, and Markov Random Field (MRF) models. They possess excellent denoising capabilities, but these methods involve significant computational costs and difficulties in manually selecting some parameters.

[0005] (3) Learning-based methods: Learning-based methods focus on learning the latent mapping from noisy images to clean images. They can be divided into traditional learning-based methods and deep network-based learning methods. In recent years, deep network-based methods have become the mainstream methods because they have achieved more promising denoising results than filtering-based, model-based, and traditional learning-based methods.

[0006] Other methods for image denoising, such as using the BM3D algorithm based on block matching for a set of image blocks that can be represented as many similar blocks in the same image, are computationally too complex and cannot adapt to images with stronger noise or cause distortion due to improper parameter settings.

[0007] In summary, the problems with existing technologies can be summarized as follows: poor noise reduction effect; blurred edges; complex computation; and too many parameters. Summary of the Invention

[0008] The present application aims to provide a low-illumination image denoising method based on phase correlation, which can balance the denoising effect and the complexity of calculation, quickly denoises in a low-illumination strong-noise scene, restores the details lost by the image and enhances the details.

[0009] The technical solution of the present application is as follows: a low-illumination image denoising method based on phase correlation, comprising the following steps:

[0010] (10) extracting image edge information: performing edge filtering on each frame of image to extract the effective edge information of the image;

[0011] (20) phase correlation of edge images: performing phase correlation operation on the adjacent two frames of edge images to calculate the registration points of the image sequence;

[0012] (30) multi-frame image correlation constraint analysis: performing homologous signal constraint and global correlation constraint on the multi-frame images after registration to construct a cross-correlation equation;

[0013] (40) solving the cross-correlation equation and iteratively updating the noise image: solving the cross-correlation equation according to the cross-correlation maximization optimization equation and the gradient maximization equation, retaining the solution of the current frame and bringing it into the cross-correlation equation of the next frame to realize the iterative update of the denoised image.

[0014] Preferably, the specific steps of the step (10) of extracting image edge information are as follows:

[0015] (11) directional filtering:

[0016] The input original image is respectively subjected to directional filtering in 0°, 45°, 90° and 135° directions to obtain four directional filtering images I0, I 45 , I 90 and I 135 .

[0017] (12) calculating an edge image:

[0018] According to the four directional filtering images, an edge image is calculated, which is specifically represented as:

[0019] I s = (|I0|+|I 45 |+|I 90 |+|I 135 |) / 4

[0020] (13) threshold segmentation:

[0021] The edge image I s is compared with a global binary threshold Th, and the pixels in the edge image I s that are less than Th are set to 0.

[0022] Preferably, the step (20) edge image phase correlation, the specific steps of calculating the registration point include:

[0023] (21) image spatial domain translation:

[0024] The adjacent two edge images are f2(x,y), f1(x,y), and f2(x,y) is obtained by translating f1(x,y) by (dx, dy), that is, it satisfies the following relationship:

[0025] f2(x,y) = f1(x-dx, y-dy)

[0026] (22) spatial domain to frequency domain:

[0027] Reflecting the two edge images into the frequency domain, specifically:

[0028] F2(u,v) = F1(u,v)*e -i*2π*(u*dx+v*dy)

[0029] Where F2(u,v) is the frequency domain image of the edge image f2(x,y), and F1(u,v) is the frequency domain image of the edge image f1(x,y);

[0030] (23) calculate the mutual power spectrum to get the registration point:

[0031] Divide the left side of the frequency domain form of the image by the right side to get the mutual power spectrum, which is:

[0032]

[0033] By taking the inverse Fourier transform of the mutual power spectrum, a Dirac function is obtained, and then the coordinates corresponding to the peak value of the function are found, that is, the required registration point is obtained;

[0034] (24) registration point global signal time correlation:

[0035] According to the registration point obtained in step (23), the registration between the adjacent two frames is realized, and the image sequence is represented as:

[0036] I(t) = I(t')

[0037] Where I represents the image sequence, t represents time, and t' represents different times.

[0038] Preferably, step (30) performs global correlation constraint on the multiple images after registration, and the step of constructing a cross-correlation equation includes:

[0039] (31) constructing a homologous signal constraint equation associated with the edge signal

[0040] In low-light scenes, the effective signal is submerged in noise, that is:

[0041] I = S + N

[0042] I is the image, S is the scene signal, which is specifically the edge signal obtained in step (12), and N is the noise;

[0043] The low-light image is characterized in spatial distribution as:

[0044] I(x, y) = S(x s ,y s ) + N(x, y)

[0045] I(x, y) represents the spatial distribution of a low-light image, (x s ,y s ) represents the spatial distribution of the edge signal, and N(x, y) represents the spatial distribution of the noise;

[0046] According to the time distribution of the global signal in the image sequence obtained in step (24), the homologous signal constraint equation of the edge signal is determined:

[0047] S(x s ,y s ,t) = S(x s ,y s ,t’)

[0048] The noise is represented in time correlation as:

[0049] N(x, y, t) = N(x’, y’, t’)

[0050] (32) Constructing a global correlation constraint equation to solve the real coordinates of pixels

[0051] According to the homologous signal constraint equation of the edge signal obtained in step (31), the global signal constraint equation of the edge signal is constructed according to the correlation:

[0052]

[0053] Proj maps each pixel of the effective signal to an equation with the homologous signal as a variable, i(t), j(t) representing the time distribution of the pixel position;

[0054] (33) The spatial distribution of the low-light image is represented as:

[0055] I(x, y) = S(x s ,y s ) + N(x, y)

[0056] According to the constructed homologous signal constraint equation, the spatial distribution of the low-light image is further represented as:

[0057] I(x,y,t') = S(x s ,y s ,t) + N(x',y',t')

[0058] Solve S(x s ,y s ,t) according to the global correlation constraint equation constructed in step (32), the above formula is expressed as:

[0059]

[0060] In the formula, The global correlation constraint equation of the edge signal is expressed as I(x,y,t'), which is expressed as a low-illumination image.

[0061] Preferably, the cross-correlation equation is optimized and solved according to the cross-correlation maximization optimization equation and the gradient maximization equation:

[0062] (41) Construct a cross-correlation maximization optimization equation:

[0063] The cross-correlation maximization is expressed as:

[0064]

[0065] (42) Construct a gradient maximization equation to enhance the image:

[0066] The gradient maximization is expressed as:

[0067] max2 <- grad{I(x,y)}

[0068] (43) Solve the cross-correlation equation to reconstruct the denoised image:

[0069] According to the cross-correlation maximization optimization equation in step (41) and the gradient maximization equation in step (42), the cross-correlation equation in step (33) is solved, and a clear and noise-free image is reconstructed.

[0070] Preferably, the cross-correlation equation and the optimization equation are solved simultaneously:

[0071]

[0072] Solve i, j represent the position of the pixel of the final low-illumination image after denoising, and the noise-free image is expressed as:

[0073] I(i(t),j(t))

[0074] Compared with the prior art, the present application has the following advantages:

[0075] 1. The denoising effect is obvious: the application can cope with various types of noise, compared with the condition that a filter window of a traditional denoising algorithm can only cope with one type of noise, the application can cope with various types of noise at the same time.

[0076] 2. For motion scenes: the existing algorithm can only process single-frame images in most cases, and the performance is poor if continuous frame images are used. The application estimates camera motion parameters by calculating the displacement of multiple frames of images through phase correlation, and ensures denoising without being affected by the camera motion state.

[0077] 3. Small information loss: the traditional denoising algorithm will lose some edge information to some extent during denoising, and the stronger the noise, the more serious the loss. The application has a certain protective effect on the edge, and the detail loss is small. BRIEF DESCRIPTION OF DRAWINGS

[0078] Figure 1 It is a flow chart of a low-illumination image denoising method based on phase correlation.

[0079] Figure 2 It is a noisy original image.

[0080] Figure 3 It is an edge image extracted from noise.

[0081] Figure 4 It is a registration point of any two frames of images.

[0082] Figure 5 It is the position information of the global signal registration point of each frame.

[0083] Figure 6 It is a preliminary solution of the cross-correlation equation.

[0084] Figure 7 It is an optimized solution of the cross-correlation equation.

[0085] Figure 8 It is a denoised image. DETAILED DESCRIPTION

[0086] As shown in Figure 1 , a low-illumination image denoising method based on phase correlation comprises the following steps:

[0087] (10) Extracting image edge information: edge filtering is performed on each frame of image to extract effective edge information of the image, and the specific steps are as follows:

[0088] The image edge is useful information of the image, and even in the noise, the image edge has certain invariance, and through this property, the edge of the scene in the noise image can be extracted.

[0089] (11) Direction filtering:

[0090] The input original image is directionally filtered in four directions of 0°, 45°, 90° and 135°, to obtain four directional filter images I0, I 45 , 90 , 135 ,

[0091] (12) Calculate the edge image:

[0092] According to the four directional filter images calculated in the last step, the edge image is represented as:

[0093] I s = (|I0| + |I 45 | + |I 90 | + |I 135 |) / 4

[0094] (13) Threshold segmentation:

[0095] Calculate the global binary threshold Th for the edge image I s , and set the pixels less than Th to 0 to reduce the impact of noise on subsequent calculations.

[0096] The noise image and the edge extraction image are shown in Figure 2 and Figure 3 .

[0097] In Figure 2 , the original image contains a large amount of noise, and the scene information is almost indistinguishable.

[0098] In Figure 3 , the original image is calculated according to (10) to obtain the image edge, and it can be seen that in Figure 3 , the edge of the signal is well extracted, and the noise is suppressed.

[0099] (20) Phase correlation of edge image: Perform phase correlation operation on the adjacent two edge images to calculate the registration points of the image sequence, the specific steps include:

[0100] (21) Image spatial domain translation:

[0101] Suppose two images are represented as f2(x,y), f1(x,y), and f2(x,y) is obtained by translating f1(x,y) by (dx, dy), that is, it satisfies the following relationship:

[0102] f2(x,y) = f1(x-dx,y-dy)

[0103] (22) Spatial domain to frequency domain:

[0104] Reflect the spatial domain image to the frequency domain, which is in the form of:

[0105] F2(u,v)=F1(u,v)*e -i*2π*(u*dx+v*dy)

[0106] (23) Calculate the cross power spectrum:

[0107] Dividing the left side of the image by the right side in its frequency domain form yields the cross-power spectrum, which has the following form:

[0108]

[0109] By obtaining the inverse Fourier transform of the cross-power spectrum, a Dirac function (pulse function) is obtained. Then, by finding the coordinates corresponding to the peak point of the function, the registration point we need can be obtained.

[0110] Figure 4 These are the registration points we are looking for.

[0111] (24) Global signal time correlation at registration points:

[0112] Based on the registration points obtained in step 23, registration is achieved between two adjacent frames, and the image sequence is then represented as follows:

[0113] I(t)=I(t')

[0114] In the formula, I represents the image sequence, t represents time, and t' represents different times. This formula links a series of isolated image sequences into a whole, and the image sequences have temporal correlation.

[0115] (30) Multi-frame image correlation constraint analysis: After registration, multiple frames of images are subjected to source signal constraints and global correlation constraints, and cross-correlation equations are constructed. The specific steps are as follows:

[0116] (31) Construct constraint equations for co-originating signals to correlate edge signals

[0117] In low-light scenarios, the effective signal is submerged in noise, that is:

[0118]

[0119] I represents the image, S represents the scene signal (which can be represented as the edge signal obtained in step 12), and N represents noise; spatially, the low-light image can be characterized as...

[0120] I(x,y)=S(x s ,y s )+N(x,y)

[0121] I(x,y) represents the spatial distribution of a low-light image, (x... s ,y s ) represents the spatial distribution of the edge signal, and N(x,y) represents the spatial distribution of the noise.

[0122] S as an important part of the global signal, has stability in spatial distribution, step 24 gets the time distribution of the global signal in the image sequence, so that the time correlation of the edge signal can be derived:

[0123] S(x s ,y s ,t)=S(x s ,y s ,t’)

[0124] This formula shows that the edge signal, its spatial distribution remains unchanged, the edge signal has time correlation. That is, the edge signal has a homology constraint, and this equation is the homology signal constraint equation of the edge signal.

[0125] For noise, noise has randomness, noise is randomly distributed in space, that is:

[0126] N(x,y)=N(x’,y’)

[0127] And because of its spatial randomness, noise has no correlation in time, the noise of the current frame may change in the next frame, so the time correlation of the noise can be expressed as:

[0128] N(x,y,t)=N(x’,y’,t’)

[0129] Thus, it can be seen that noise does not have homology signal constraints and cannot construct homology signal constraint equations, but the spatial and temporal distribution of noise can be expressed by the above formula.

[0130] Figure 5 The position information of the homologous signal registration points of each frame is obtained.

[0131] (32) Constructing a global correlation constraint equation to solve the real coordinates of pixels

[0132] According to the homology signal constraint equation of the edge signal obtained in step 31, the spatial and temporal distribution of the edge signal is analyzed, and the global signal constraint equation of the edge signal is constructed according to its correlation:

[0133]

[0134] The gray value of any pixel in the image corresponds to a complex physical process, and each edge signal corresponds to a pixel position (x s ,y s) can be mapped into an equation, and any related parameter in the physical process can be used as a variable in the equation. Proj maps each pixel of the effective signal to an equation with the homologous signal as a variable, uses the correlation of the homologous signal to solve the real position of the edge signal in the real scene, and accurately calculates the position relationship of each related pixel. i(t), j(t) is the time distribution of the pixel position.

[0135] (33) According to step 31, the spatial distribution of the low-illumination image can be represented as:

[0136] I(x,y)=S(x s ,y s )+N(x,y)

[0137] According to the homologous signal constraint equation constructed in step 31, the above formula can be represented as:

[0138] I(x,y,t’)=S(x s ,y s ,t)+N(x’,y’,t’)

[0139] According to the global correlation constraint equation constructed in step 32, the solution of S(x s ,y s ,t) is represented as:

[0140]

[0141] The equation is the cross-correlation equation of the real pixel spatial distribution and the time distribution, which represents the global correlation constraint equation of the edge signal, and the result is represented as which contains the time distribution of the real pixel, but the noise does not have correlation and cannot be brought into the global correlation constraint equation, so the final solution still contains the distribution of the noise. Therefore, the low-illumination image is represented as: I(x,y,t’).

[0142] Figure 6 The initial solution of the reconstructed signal equation can be seen that the distribution of the solution is quite different, and is not very accurate.

[0143] (40) Solve the cross-correlation equation and update the noise image iteratively: solve the cross-correlation equation according to the cross-correlation maximization optimization equation and the gradient maximization equation, and keep the solution of the current frame, and bring it into the cross-correlation equation of the next frame to realize the iterative update of the denoising image, and the specific steps are:

[0144] (41) Construct a cross-correlation maximization optimization equation to reduce noise

[0145] The cross-correlation maximization is to establish equations for the key areas, and the key areas are the edge signals. The total number of equations is reduced to ensure the best effect under the minimum calculation.

[0146] The cross-correlation maximization is represented as:

[0147]

[0148] In the cross-correlation maximization equation, the edge signal has correlation as a homologous signal, and the information redundancy it produces makes the number of constructed equations much larger than the number of variables, ensuring the accurate calculation of homologous signals in strong noise. However, noise has spatial and temporal randomness and does not have the correlation of homologous signals, so noise cannot be cross-correlated maximized in solving, and noise signals have no solution. But the noise contained in the edge of the scene cannot be removed because of the influence of the edge.

[0149] (42) Constructing a gradient maximization equation to enhance the image

[0150] Gradient maximization is to use the cross-correlation maximization solving result of the scene signal to solve other areas outside the scene and the edge of the scene again, to ensure the global calculation result and further eliminate the noise signal near the edge. The gradient maximization can be represented as:

[0151] max2←grad{I(x,y)}

[0152] The gradient distribution of the edge signal is obviously different from the noise, and the gradient distribution has consistency. The use of gradient maximization destroys the correlation of noise and directly suppresses it.

[0153] (43) Solving the cross-correlation equation to reconstruct the denoised image:

[0154] According to the cross-correlation maximization optimization equation in step (41) and the gradient maximization equation in step (42), the cross-correlation equation in step (33) is solved, and a clear noise-free image is reconstructed.

[0155] The equation in step (33) is the cross-correlation equation, which is represented as:

[0156]

[0157] In the reconstructed signal i(x,y), it is still not the optimal solution, but by using the two optimization equations of step (41) cross-correlation maximization optimization equation and step (42) gradient maximization equation, the cross-correlation equation in step (33) can be optimized and solved, and the cross-correlation equation and the optimization equation are solved together:

[0158]

[0159] Finally, the solution is i, j represent the position of the pixel of the last low-illumination image after denoising, This position is time-dependent, and the last noise-free image can be expressed as:

[0160] I(i(t), j(t))

[0161] The solution of this frame can still be brought into the cross-correlation equation of the next frame to achieve iterative updating of the denoised image.

[0162] Figure 7 The solution obtained by optimizing the cross-correlation equation through the cross-correlation maximum optimization equation and the gradient maximum optimization equation can be seen that the difference between the distribution of the solution is smaller, and the numerical value is more accurate.

[0163] Figure 8 The clear noise-free image obtained by iterative updating has basically achieved the denoising effect.

Claims

1. A low-light image denoising method based on phase correlation, characterized in that, Includes the following steps: (10) Extract image edge information: Perform edge filtering on each frame of the image to extract the effective edge information of the image; (20) Edge image phase correlation: Perform phase correlation operation on two adjacent edge images to calculate the registration points of the image sequence. The specific steps include: (21) Image spatial translation: The edge images of two adjacent frames are f2(x,y) and f1(x,y), respectively, and f2(x,y) is obtained by shifting f1(x,y) by (dx,dy), which satisfies the following relationship: f2(x,y)=f1(x-dx,y-dy) (22) Spatial domain to frequency domain conversion: Reflecting the two edge images in the frequency domain is as follows: F2(u,v)=F1(u,v)*e -i*2π*(u*dx+v*dy) Where F2(u,v) is the frequency domain image of edge image f2(x,y), and F1(u,v) is the frequency domain image of edge image f1(x,y); (23) The registration point is obtained by calculating the cross power spectrum: Dividing the left side of the image by the right side in its frequency domain form yields the cross-power spectrum, which has the following form: By obtaining the inverse Fourier transform of the cross-power spectrum, a Dirac function is obtained. Then, by finding the coordinates corresponding to the peak points of the function, the required registration points can be obtained. (24) Global signal time correlation at registration points: Based on the registration points obtained in step (23), registration is achieved between two adjacent frames, and the image sequence is then represented as follows: I(t=I(t)') In the formula, I represents the image sequence, t represents time, and t' represents different times; (30) Multi-frame image correlation constraint analysis: After registration, multiple frames of images are subjected to source signal constraints and global correlation constraints, and cross-correlation equations are constructed. The specific steps include: (31) Construct constraint equations for co-originating signals to correlate edge signals In low-light scenarios, the effective signal is submerged in noise, that is: I = S + N I represents the image, S represents the scene signal, the scene signal is specifically the edge signal obtained in step (12), and N represents noise; In terms of spatial distribution, low-light images are characterized as follows: I(x,y)=S(x s ,y s )+N(x,y) I(x,y) represents the spatial distribution of a low-light image, (x s ,y s N(x,y) represents the spatial distribution of the edge signal, and N(x,y) represents the spatial distribution of the noise. Based on the temporal distribution of the global signal in the image sequence obtained in step (24), determine the source signal constraint equation for the edge signal: S(x s ,y s ,t)=S(x s ,y s ,t’) The time correlation of noise is expressed as: N(x,y,t)=N(x',y',t') (32) Construct global correlation constraint equations to solve for pixel real coordinates Based on the source signal constraint equations of the edge signals obtained in step (31), the global signal constraint equations of the edge signals are constructed according to the correlation: Proj maps each pixel of the valid signal to an equation with the same source signal as the variable, where i(t) and j(t) represent the temporal distribution of the pixel position; (33) The spatial distribution of low-light images is represented as: I(x,y)=S(x s ,y s )+N(x,y) Based on the constructed homogeneous signal constraint equation, the spatial distribution of low-light images is further represented as: I(x,y,t’)=S(x s ,y s ,t)+N(x’,y’,t’) Based on the global correlation constraint equation constructed in step (32), for S(x s ,y s Solving for t), the above expression is: In the formula, The global correlation constraint equation representing the edge signal is I(x,y,t'), which represents the low-light image. (40) Solve the cross-correlation equation and iteratively update the noisy image: Solve the cross-correlation equation according to the cross-correlation maximization optimization equation and the gradient maximization equation, retain the solution of the current frame, and substitute it into the cross-correlation equation of the next frame to realize the iterative update of the denoised image. The specific process is as follows: (41) Construct the cross-correlation maximization optimization equation: The maximization of cross-correlation is characterized as: (42) Constructing a gradient maximization equation to enhance the image: Gradient maximization is characterized as: max2←grad{I(x,y)} (43) Solve the cross-correlation equation to reconstruct the denoised image: Based on the cross-correlation maximization optimization equation in step (41) and the gradient maximization equation in step (42), the cross-correlation equation in step (33) is solved to reconstruct a clear and noise-free image.

2. The low-light image denoising method based on phase correlation according to claim 1, characterized in that, The specific steps for extracting image edge information in step (10) are as follows: (11) Directional filtering: The input original image is subjected to directional filtering at four directions: 0°, 45°, 90°, and 135°, resulting in directional filtering maps I0 and I1 in the four directions. 45 I 90 I 135 ; (12) Calculate the edge image: Based on the directional filtering maps in four directions, the edge image is calculated, and the edge image is specifically represented as follows: I s =(|I0|+|I 45 |+|I 90 |+|I 135 |) / 4 (13) Threshold segmentation: edge image I s The edge image I is compared with the global binarization threshold Th. s Pixels smaller than h are set to 0.

3. The low-light image denoising method based on phase correlation according to claim 1, characterized in that, Simultaneous cross-correlation equations and optimization equations: Solve i and j represent the pixel positions in the denoised low-light image. The noise-free image is represented as: I(i(t),j(t)).

Images