Binocular vision system photon noise suppression method based on least square method

Abstract

The invention discloses a binocular vision system photon noise suppression method based on a least square method, and belongs to the technical field of image processing. The method solves the problems that an existing method is poor in photon noise suppression capacity, and detail information in the image cannot be fully reserved while the photon noise is suppressed. The method comprises the following steps: firstly, modeling photon noise by using a Poisson-Gaussian model containing two unknown parameters; aligning the two images collected by the binocular vision system to the same coordinate system through an image registration method based on SIFT, and constructing a linear equation about two unknown parameters for each pixel of the two registered images to form an overdetermined equation set; and constructing a cost function according to a least square criterion, and solving the cost function through a least square method so as to obtain the optimal estimation of two unknown parameters in the Poisson-Gaussian model, thereby obtaining the optimal estimation of the original image. The method can be applied to suppression of photon noise in the image.

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing, and in particular relates to a method for suppressing photon noise in a binocular vision system based on the least squares method. Background Art

[0002] Photon noise is a common form of image noise. The random motion of photons causes irregular brightness jumps and sudden changes in the image, significantly reducing image quality and severely limiting the human eye's ability to perceive and recognize image content. Because the intensity of photon noise is correlated with brightness, suppressing photon noise is much more difficult than suppressing conventional additive white noise. To mitigate the impact of photon noise on image quality, several photon noise suppression algorithms have been proposed. For example, Foi proposed a photon noise suppression algorithm based on heteroscedastic modeling, and Katkovnik proposed a non-local multi-model photon noise suppression algorithm.

[0003] However, the existing methods are still limited in their ability to suppress photon noise, and are unable to fully retain the detailed information in the image while suppressing photon noise. Therefore, it is very necessary to propose a new photon noise suppression method to solve the above problems. Summary of the Invention

[0004] The purpose of the present invention is to solve the problem that existing methods have poor suppression ability of photon noise and cannot fully retain the detail information in the image while suppressing photon noise, and propose a photon noise suppression method for binocular vision system based on least squares method.

[0005] The technical solution adopted by the present invention to solve the above technical problems is: a method for suppressing photon noise in a binocular vision system based on the least squares method, the method specifically comprising the following steps:

[0006] Step 1: Align the two images collected by the binocular vision system into a two-dimensional Cartesian coordinate system;

[0007] Step 2: Estimate the expression of the original image using linear least squares method;

[0008] Step 3: Construct an overdetermined set of equations based on the alignment result of step 1 and the original image expression of step 2;

[0009] Step 4: construct a cost function F according to the overdetermined equations, and obtain an image after suppressing photon noise based on the cost function F.

[0010] Furthermore, the two images collected by the binocular vision system are aligned to the two-dimensional Cartesian coordinates in space by using an image registration method based on SIFT.

[0011] Further, the specific process of the step two is:

[0012] Step two one, obtaining the degraded image d contaminated by Poisson-Gaussian noise:

[0013]

[0014] Wherein, f represents the original image not contaminated by noise, u represents white noise with Gaussian distribution whose mean is 0 and variance is 1, and k1 and k2 are unknown parameters;

[0015] Let the optimal linear estimation of the original image f be as shown in formula (2):

[0016]

[0017] Wherein, k and b are undetermined coefficients, is the linear optimal estimation of the original image f;

[0018] Step two two, obtaining the cost function h about the undetermined coefficients k and b according to the least square criterion:

[0019]

[0020] Wherein, E[·] represents the expectation of a random variable;

[0021] Substitute formula (2) into formula (3) and arrange to obtain:

[0022] h(k,b)=E[(f-k×d-b) 2 ] (4)

[0023] Take the partial derivative of formula (4) about the undetermined coefficients k and b respectively:

[0024]

[0025] Substitute formula (4) into formula (5):

[0026]

[0027] Wherein, E[d] represents the mean of the gray scale of all pixels in the degraded image d, and E[f] represents the mean of the gray scale of all pixels in the original image;

[0028] The solution of equation (6) is obtained by substitution method:

[0029]

[0030] Wherein, represents the variance of the gray scale of all pixels in the degraded image d;

[0031] Step two three, get the expectation of both sides of formula (1) :

[0032]

[0033] Expand and arrange formula (8) to get the expectation of f:

[0034] E[f] = E[d] (9)

[0035] Substitute formula (1) into E[f x d] to get:

[0036]

[0037] Square the left and right ends of formula (1) respectively, and then calculate the mean of the squared results to get:

[0038]

[0039] Substitute formula (9) into formula (11) and arrange to get:

[0040] E[f 2 ] = E[d 2 ]-k1 x E[d]-k2 (12)

[0041] Substitute formula (12) into formula (10) to get:

[0042] E[f x d] = E[d 2 ]-k1 x E[d]-k2 (13)

[0043] Substitute formula (13) and formula (9) into formula (7) to get the expressions of k and b:

[0044]

[0045] Substitute formula (14) into formula (2) and arrange to get the linear least squares estimation expression of f:

[0046]

[0047] Further, the specific process of step three is:

[0048] After aligning the two degraded images collected by the binocular vision system to the two-dimensional Cartesian coordinate system in space, according to formula (15), the gray values of the spatial pixel position (i, j) in the two original images are:

[0049]

[0050] Where d1 and d2 represent the two degraded images collected by the binocular vision system, represents the variance of the grayscale values ​​of all pixels in the observed degraded image d1, represents the variance of the grayscale values ​​of all pixels in the observed degraded image d2, E[d1] represents the mean grayscale value of all pixels in the degraded image d1, E[d2] represents the mean grayscale value of all pixels in the degraded image d2, d1(i,j) represents the grayscale value of the pixel (i,j) in the observed degraded image d1, d2(i,j) represents the grayscale value of the pixel (i,j) in the observed degraded image d2, Indicates the original image corresponding to d1 The grayscale value of the pixel (i, j) in Indicates the original image corresponding to d2 The gray value of the pixel (i, j);

[0051] Then formula (17) holds:

[0052]

[0053] Let the intermediate variables A1(i,j) and A2(i,j) be as shown in formula (18):

[0054]

[0055] Substituting formula (18) into formula (17), we get:

[0056]

[0057] For all pixel positions in space that satisfy the relationship of formula (19), an overdetermined set of equations about parameters k1 and k2 is constructed based on all pixel positions in space.

[0058] Furthermore, the specific process of step 4 is as follows:

[0059] Step 4.1. Construct a cost function F about k1 and k2 according to the least squares criterion:

[0060]

[0061] The cost function F calculates the partial derivatives of the two parameters k1 and k2 respectively, and sets the partial derivatives to 0;

[0062]

[0063] Let the intermediate variables B1, B2, B3, B4 and B5 be:

[0064]

[0065] Solving formula (21) yields a linear equation system about k1 and k2. Substituting the five intermediate variables of formula (22) into the obtained linear equation system yields:

[0066]

[0067] Solving the equations of formula (23) using the substitution method yields the expressions of k1 and k2 shown in formula (24):

[0068]

[0069] Substituting k1 and k2 in formula (24) into formula (16), we can obtain the image after suppressing photon noise of the degraded image d1: And the image after suppressing photon noise of the degraded image d2

[0070] The beneficial effects of the present invention are:

[0071] The present invention first uses a Poisson-Gaussian model containing two unknown parameters to model photon noise; then, the two images collected by the binocular vision system are aligned to the same coordinate system through an image registration method based on SIFT. For the two registered images, a linear equation about the two unknown parameters is constructed for each pixel, thereby forming an overdetermined system of equations; and a cost function is constructed according to the least squares criterion. The cost function is solved by the least squares method to obtain the optimal estimate of the two unknown parameters in the Poisson-Gaussian model, thereby obtaining the optimal estimate of the original image. Experiments have shown that the method of the present invention significantly improves the ability to suppress photon noise in the image and fully retains the detail information in the image. BRIEF DESCRIPTION OF THE DRAWINGS

[0072] Figure 1 This is a flow chart of a method for suppressing photon noise in a binocular vision system based on the least squares method of the present invention;

[0073] Figure 2(a) is an image 1 captured by camera 1 of the binocular vision system;

[0074] Figure 2(b) is an image 1 captured by camera 2 of the binocular vision system;

[0075] Figure 3(a) is the restored image corresponding to Figure 2(a);

[0076] Figure 3(b) is the restored image corresponding to Figure 2(b);

[0077] Figure 4(a) is the second image captured by camera 1 of the binocular vision system;

[0078] Figure 4(b) is the second image captured by camera 2 of the binocular vision system;

[0079] Figure 5(a) is the restored image corresponding to Figure 4(a);

[0080] Figure 5(b) is the restored image corresponding to Figure 4(b). DETAILED DESCRIPTION

[0081] Specific implementation method 1: Combination Figure 1 This embodiment describes a method for suppressing photon noise in a binocular vision system based on the least squares method, and the method specifically includes the following steps:

[0082] Step 1: Align the two images collected by the binocular vision system into a two-dimensional Cartesian coordinate system;

[0083] Step 2: Estimate the expression of the original image using linear least squares method;

[0084] Step 3: Construct an overdetermined set of equations based on the alignment result of step 1 and the original image expression of step 2;

[0085] Step 4: construct a cost function F according to the overdetermined equations, and obtain an image after suppressing photon noise based on the cost function F.

[0086] Specific embodiment 2: This embodiment differs from specific embodiment 1 in that the two images collected by the binocular vision system are aligned to two-dimensional Cartesian coordinates in space by using an image registration method based on SIFT.

[0087] Other steps and parameters are the same as those in the first embodiment.

[0088] Specific implementation method three: This implementation method is different from specific implementation methods one or two in that the specific process of step two is as follows:

[0089] Step 21: Obtain the degraded image d contaminated by Poisson-Gaussian noise:

[0090]

[0091] Where f represents the original image not contaminated by noise, u represents white noise with a Gaussian distribution and a mean of 0 and a variance of 1 (independent of f), and k1 and k2 are unknown parameters.

[0092] Let the optimal linear estimate of the original image f be as shown in formula (2):

[0093]

[0094] Among them, k and b are unknown coefficients. is the linear optimal estimate of the original image f;

[0095] Step two, obtain the cost function h about undetermined coefficients k and b according to least square criterion:

[0096]

[0097] Wherein, E[·] represents the expectation of random variable;

[0098] Substitute formula (2) into formula (3) and organize to obtain:

[0099] h(k,b)=E[(f-k×d-b) 2 ] (4)

[0100] When the cost function h is minimum, It is the least square estimation for original image. In order to make the cost function h minimum, the partial derivative about independent variable k and b of formula (4) is taken respectively, and two partial derivatives must be 0 simultaneously.

[0101] Take the partial derivative about undetermined coefficients k and b of formula (4) respectively:

[0102]

[0103] Substitute formula (4) into formula (5), and obtain a binary quadratic linear equation group about k and b as shown in formula (6) through derivation:

[0104]

[0105] Wherein, E[d] represents the mean of the gray scale of all pixels in the degradation image d, E[f] represents the mean of the gray scale of all pixels in the original image; the gray scale value of the pixel in the i-th row and j-th column in the degradation image d is denoted as d ij , then the value of the i-th row and j-th column in d 2 is d E[d 2 ] represents the expectation of the value of all elements in d 2 , and the gray scale value of the pixel in the i-th row and j-th column in the image f is denoted as f ij , then the value of the i-th row and j-th column in f×d is d ij ×f ij , E[f×d] represents the expectation of the value of all elements in f×d;

[0106] Obtain the solution of equation (6) through substitution method:

[0107]

[0108] Wherein, It represents the variance of the grayscale values ​​of all pixels in the degraded image d. From formula (7), we can see that only E[f×d] and E[f] are unknown in the formula.

[0109] Step 2 and 3: To solve E[f], take the expectation on both sides of formula (1) and get:

[0110]

[0111] Because the noise u is independent of the original image f, the random variable u is Independent of each other. And because the mean value of noise u is 0, the expectation of f can be obtained by expanding and sorting formula (8):

[0112] E[f]=E[d] (9)

[0113] To solve E[f×d], substitute formula (1) into E[f×d] to obtain:

[0114]

[0115] Among them, E[f 2 ] is defined in the same way as E[d 2 ]same, The definition of is the same as E[f×d];

[0116] Perform square operations on the left and right ends of formula (1) respectively, and then calculate the average value of the square operation results to obtain:

[0117]

[0118] Since the mean of noise u is 0 and the variance is 1, substitute formula (9) into formula (11) and sort it out to get:

[0119] E[f 2 ]=E[d 2 ]-k1×E[d]-k2 (12)

[0120] Substituting formula (12) into formula (10) yields:

[0121] E[f×d]=E[d 2 ]-k1×E[d]-k2 (13)

[0122] Substituting formula (13) and formula (9) into formula (7), we can obtain the expressions of k and b:

[0123]

[0124] Substituting formula (14) into formula (2) and sorting it out, we can get the linear least squares estimation expression of f:

[0125]

[0126] Other steps and parameters are the same as those in the first or second embodiment.

[0127] Specific embodiment 4: This embodiment differs from any one of specific embodiments 1 to 3 in that the specific process of step 3 is as follows:

[0128] After aligning the two degraded images collected by the binocular vision system to the spatial two-dimensional Cartesian coordinate system, the grayscale values ​​of the spatial pixel position (i, j) in the two original images are obtained according to formula (15):

[0129]

[0130] Among them, d1 and d2 represent two degraded images collected by the binocular vision system. represents the variance of the grayscale values ​​of all pixels in the observed degraded image d1, represents the variance of the grayscale values ​​of all pixels in the observed degraded image d2, E[d1] represents the mean grayscale value of all pixels in the degraded image d1, E[d2] represents the mean grayscale value of all pixels in the degraded image d2, d1(i,j) represents the grayscale value of the pixel (i,j) in the observed degraded image d1, d2(i,j) represents the grayscale value of the pixel (i,j) in the observed degraded image d2, Indicates the original image corresponding to d1 The grayscale value of the pixel (i, j) in Indicates the original image corresponding to d2 The gray value of the pixel (i, j);

[0131] Since the two registered images have the same spatial position relationship, the grayscale values ​​of the pixels corresponding to the spatial position (i, j) are equal, and formula (17) holds:

[0132]

[0133] Let the intermediate variables A1(i,j) and A2(i,j) be as shown in formula (18):

[0134]

[0135] Substituting formula (18) into formula (17), we get:

[0136]

[0137] For all pixel positions in space that satisfy the relationship of formula (19), an overdetermined set of equations about parameters k1 and k2 is constructed based on all pixel positions in space.

[0138] The other steps and parameters are the same as those in the first to third embodiments.

[0139] Specific embodiment 5: This embodiment differs from any one of specific embodiments 1 to 4 in that the specific process of step 4 is as follows:

[0140] Step 4.1. Construct a cost function F about k1 and k2 according to the least squares criterion:

[0141]

[0142] The cost function F calculates the partial derivatives of the two parameters k1 and k2 respectively, and sets the partial derivatives to 0;

[0143]

[0144] Let the intermediate variables B1, B2, B3, B4 and B5 be:

[0145]

[0146] Solving formula (21) yields a linear equation system about k1 and k2. Substituting the five intermediate variables of formula (22) into the obtained linear equation system yields:

[0147]

[0148] Solving the equations of formula (23) using the substitution method yields the expressions of k1 and k2 shown in formula (24):

[0149]

[0150] Substituting k1 and k2 in formula (24) into formula (16), we can obtain the image after suppressing photon noise of the degraded image d1: And the image after suppressing photon noise of the degraded image d2

[0151] The other steps and parameters are the same as those in the first to fourth embodiments.

[0152] Experimental results and analysis

[0153] The experimental hardware platform used a desktop computer with an i7-12700 CPU, 32GB of DDR5 RAM, and a GeForce RTX 4080 graphics card. The operating system was Windows 12, and the simulation platform was Matlab 2016a. The Middlebury Stereo Datasets were used as the binocular vision database. Both the input images and the algorithm output were in jpg format. Figures 2(a) and 2(b) show the first set of images captured by the binocular vision system, and Figures 3(a) and 3(b) show the corresponding restored images after photon noise suppression. Figures 4(a) and 4(b) show the second set of images captured by the binocular vision system, and Figures 5(a) and 5(b) show the corresponding restored images after photon noise suppression. Comparing the degraded and restored images reveals that the degraded images contain random, granular noise, which severely interferes with vision and reduces image quality. After noise suppression, the restored image no longer has the grainy effect of the degraded image, and the visual effect of the image is significantly improved. Therefore, the algorithm proposed in this invention can effectively suppress photon noise and significantly improve the visual effect of the restored image.

[0154] The above examples are merely illustrative of the calculation model and process of the present invention and are not intended to limit the embodiments of the present invention. Persons skilled in the art will readily appreciate that other variations or modifications based on the above description are possible. This list of embodiments is not exhaustive; however, any obvious variations or modifications derived from the technical solution of the present invention remain within the scope of protection of the present invention.

Claims

1. A method for suppressing photon noise in a binocular vision system based on the least squares method, characterized in that: The method specifically comprises the following steps: Step 1: Align the two images collected by the binocular vision system into a two-dimensional Cartesian coordinate system; Step 2: Estimate the expression of the original image using linear least squares method; Step 3: Construct an overdetermined set of equations based on the alignment result of step 1 and the original image expression of step 2; Step 4: construct a cost function F according to the overdetermined equations, and obtain an image after suppressing photon noise based on the cost function F.

2. The method for suppressing photon noise in a binocular vision system based on the least squares method according to claim 1, wherein: The two images collected by the binocular vision system are aligned to the two-dimensional Cartesian coordinates in space, and an image registration method based on SIFT is adopted.

3. The method for suppressing photon noise in a binocular vision system based on the least squares method according to claim 2, wherein: The specific process of step 2 is as follows: Step 21: Obtain the degraded image d contaminated by Poisson-Gaussian noise: Where f represents the original image that is not contaminated by noise, u represents white noise with a Gaussian distribution and a mean of 0 and a variance of 1, and k1 and k2 are unknown parameters. Let the optimal linear estimate of the original image f be as shown in formula (2): Among them, k and b are unknown coefficients, is the linear optimal estimate of the original image f; Step 22: Obtain the cost function h for the unknown coefficients k and b according to the least squares criterion: Where E[·] represents the expectation of a random variable; Substitute formula (2) into formula (3) and sort it out to get: h(k,b)=E[(f-k×d-b) 2 ] (4) Take the partial derivatives of formula (4) with respect to the unknown coefficients k and b: Substituting formula (4) into formula (5): Where E[d] represents the mean grayscale value of all pixels in the degraded image d, and E[f] represents the mean grayscale value of all pixels in the original image. The solution of equation (6) is obtained by substitution method: in, Represents the variance of the grayscale values ​​of all pixels in the degraded image d; Step 2 and 3: Take the expectation on both sides of formula (1) to get: Expanding and sorting formula (8) yields the expectation of f: E[f]=E[d] (9) Substituting formula (1) into E[f×d], we can obtain: Perform square operations on the left and right ends of formula (1) respectively, and then calculate the average value of the square operation results to obtain: Substituting formula (9) into formula (11) and sorting it out, we get: E[f 2 ]=E[d 2 ]-k1×E[d]-k2 (12) Substituting formula (12) into formula (10) yields: E[f×d]=E[d 2 ]-k1×E[d]-k2 (13) Substituting formula (13) and formula (9) into formula (7), we can obtain the expressions of k and b: Substituting formula (14) into formula (2) and sorting it out, we can get the linear least squares estimation expression of f:

4. The method for suppressing photon noise in a binocular vision system based on the least squares method according to claim 3, characterized in that: The specific process of step three is: After aligning the two degraded images collected by the binocular vision system to the spatial two-dimensional Cartesian coordinate system, the grayscale values ​​of the spatial pixel position (i, j) in the two original images are obtained according to formula (15): Among them, d1 and d2 represent two degraded images collected by the binocular vision system. represents the variance of the grayscale values ​​of all pixels in the observed degraded image d1, represents the variance of the grayscale values ​​of all pixels in the observed degraded image d2, E[d1] represents the mean grayscale value of all pixels in the degraded image d1, E[d2] represents the mean grayscale value of all pixels in the degraded image d2, d1(i,j) represents the grayscale value of the pixel (i,j) in the observed degraded image d1, d2(i,j) represents the grayscale value of the pixel (i,j) in the observed degraded image d2, Indicates the original image corresponding to d1 The gray value of the pixel (i, j) in Indicates the original image corresponding to d2 The gray value of the pixel (i, j); Then formula (17) holds: Let the intermediate variables A1(i,j) and A2(i,j) be as shown in formula (18): Substituting formula (18) into formula (17), we get: For all pixel positions in space that satisfy the relationship of formula (19), an overdetermined set of equations about parameters k1 and k2 is constructed based on all pixel positions in space.

5. The method for suppressing photon noise in a binocular vision system based on the least squares method according to claim 4, characterized in that: The specific process of step 4 is as follows: Step 4.

1. Construct a cost function F about k1 and k2 according to the least squares criterion: The cost function F calculates the partial derivatives of the two parameters k1 and k2 respectively, and sets the partial derivatives to 0; Let the intermediate variables B1, B2, B3, B4 and B5 be: Solving formula (21) yields a linear equation system about k1 and k2. Substituting the five intermediate variables of formula (22) into the obtained linear equation system yields: Solving the equations of formula (23) using the substitution method yields the expressions of k1 and k2 shown in formula (24): Substituting k1 and k2 in formula (24) into formula (16), we can obtain the image after suppressing photon noise of the degraded image d1: And the image after suppressing photon noise of the degraded image d2

Images