A long-wave infrared image denoising method

Abstract

The present invention discloses a long-wave infrared image denoising method, which includes: using a singular value decomposition algorithm, decomposing a long-wave infrared image with high noise and low contrast collected by long-wave infrared through SVD, setting the maximum singular value of the decomposed eigenvalue matrix to zero, calculating the average singular value, replacing the remaining singular values ​​except the first singular value with the average singular value, assigning successively decreasing weights to each singular value according to its position, reconstructing the eigenvalue matrix, and then reconstructing the long-wave infrared image matrix, and finally, using a median filtering algorithm to further denoise the reconstructed long-wave infrared image. In terms of image processing, the present invention overcomes the shortcomings of high noise and low contrast of the long-wave infrared image itself; in terms of feature processing, the present invention overcomes the problem of feature mistracking caused by the defects of the long-wave infrared image itself, and provides a feasible solution for the subsequent SLAM method to operate in a weak light environment.

Description

Technical Field

[0001] The invention belongs to the technical field of image processing, and in particular relates to a long-wave infrared image denoising method. Background Art

[0002] The vision-based simultaneous localization and mapping (SLAM) system uses the environmental information obtained by the camera to estimate the current position information of the carrier and build a map model of the explored area. With the development of mechanical hardware and computer vision, the types of cameras are increasing. Among them, visible light cameras are widely used in SLAM systems due to their low cost and light weight. At present, the research on visible light SLAM has achieved good results. In the case of stable light source, the SLAM system based on visible light camera performs well, but when there are non-ideal environmental conditions such as lighting changes, no light source or heavy fog in the environment, the positioning and mapping accuracy of the SLAM system designed based on visible light camera will be seriously affected, so that it cannot complete the simultaneous positioning and mapping tasks.

[0003] Compared with visible light cameras, long-wave infrared images are formed by sensing infrared information in the environment. The imaging results reflect the temperature distribution of the surrounding environment. Even in the dark environment at night, images can be formed and the impact of light changes and bad weather is small. Therefore, SLAM based on long-wave infrared can effectively solve the problem of poor system robustness of visible light cameras in the absence of light, fog and other conditions. However, affected by the principle of long-wave infrared imaging, its imaging results will have problems such as low signal-to-noise ratio, missing texture, and low contrast, which will lead to feature extraction errors and feature mistracking in the SLAM front end, which will directly affect the performance of the SLAM system based on long-wave infrared.

[0004] Therefore, in order to reduce the impact of the above-mentioned shortcomings of long-wave infrared on the performance of the SLAM system, the present invention proposes a long-wave infrared image denoising method, which uses the SVD method to decompose the long-wave infrared collected image, processes the eigenvalue matrix obtained by the decomposition, reduces the impact of noise, enhances image contrast, and improves image feature extraction and tracking accuracy, provides reliable data association for the subsequent SLAM system, and improves positioning accuracy and robustness. Summary of the invention

[0005] The technical problem to be solved by the present invention is to provide a long-wave infrared image denoising method, which can process high-noise, low-contrast long-wave infrared data in a weak light environment and provide a data association basis for subsequent synchronous positioning and mapping functions.

[0006] The object of the present invention is achieved through the following technical solution: A long-wave infrared image denoising method comprises the following steps:

[0007] Step (1): In a weak light environment, a long-wave infrared camera C with calibrated internal parameters is set at t 0 Long-wave infrared images with noise interference are collected at all times 0 , image I 0 The size is M×N, where M and N are the height and width of the image respectively;

[0008] Step (2): Decompose the long-wave infrared image I using the SVD method 0 , and obtain the corresponding eigenvalue matrix Σ. Based on the definition of SVD decomposition, the eigenvalue matrix Σ is processed for the long-wave infrared image I 0 Reconstruct and obtain the reconstructed long-wave infrared image I 1 ;

[0009] Step (3): Use the median filter method to process the reconstructed image I 1 , and get the denoised long-wave infrared image I 2 .

[0010] Furthermore, the process of processing the eigenvalue matrix Σ in step (2) is as follows: setting the maximum singular value of the eigenvalue matrix Σ to zero, calculating the average singular value, replacing the remaining singular values ​​except the first singular value with the average singular value, assigning decreasing weights to each singular value according to its position, and forming a new eigenvalue matrix.

[0011] Furthermore, the process of processing the eigenvalue matrix Σ in step (2) is:

[0012] Decomposition of long wave infrared image using SVD method 0 get:

[0013]

[0014] Among them, the matrix U is of size M×M, consisting of the left singular vector (Left Singular Vector, LSV) u i The matrix V is N×N in size and consists of the right singular vector (Right Singular Vector, RSV) v i Composition; the size of the matrix Σ is M×N, and its diagonal elements represent the singular values ​​s of the image i ,s i In descending order, they are arranged from large to small on the diagonal of the eigenvalue matrix Σ, where T represents the transpose operation; from the perspective of the image matrix, the singular value s i Represents the overall brightness of the image, the singular vector u i 、v iRepresents the texture features in the image. At the same time, the maximum singular value can reflect the feature information with a high proportion in the image, and the remaining singular values ​​can reflect the feature information with a low proportion in the image.

[0015] The decomposed eigenvalue matrix Σ is processed to make its maximum singular value s 1 = 0, and calculate the average singular value s avg

[0016]

[0017] The average singular value is used to replace all singular values ​​except the first singular value in the eigenvalue matrix Σ, and the weights are assigned in descending order according to the position, so that the singular value s′ is obtained. i The new eigenvalue matrix Σ′

[0018]

[0019] Use the new eigenvalue matrix Σ′ to reconstruct the image matrix and obtain the reconstructed long-wave infrared image I 1

[0020] I 1 =[U][Σ′][V] T (4)

[0021] Furthermore, in step (3), the median filter processes the image I 1 The process is:

[0022] Suppose the sliding window size is K×K, and the window is arranged in columns in the image I 1 Slide up, the processed pixel value p in the window i 'for

[0023]

[0024] Among them, the collection For image I 1 The original pixel value, the Median function is to take the median of the set; the window slides across the entire image by column, and the pixel values ​​in the median processing window are obtained to obtain the processed image I 2 .

[0025] The beneficial effects of the present invention are as follows: the present invention utilizes a singular value decomposition algorithm to decompose a high-noise, low-contrast long-wave infrared image collected by long-wave infrared through SVD, sets the maximum singular value of the decomposed eigenvalue matrix to zero, calculates the average singular value, replaces the remaining singular values ​​except the first singular value with the average singular value, assigns successively decreasing weights to each singular value according to its position, reconstructs the eigenvalue matrix, and then reconstructs the long-wave infrared image matrix, and finally, uses a median filter algorithm to further denoise the reconstructed long-wave infrared image. In terms of image processing, the present invention overcomes the shortcomings of high noise and low contrast of the long-wave infrared image itself; in terms of feature processing, the present invention overcomes the problems of feature mistracking caused by the defects of the long-wave infrared image itself, and provides a feasible solution for the subsequent SLAM method to operate in a weak light environment. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] In order to more clearly illustrate the technical solution of the present invention, the following will briefly introduce the drawings required for use in the embodiments. It should be understood that the following drawings only illustrate certain embodiments of the present invention and should not be regarded as limiting the scope of protection of the present invention. In each of the drawings, similar components are numbered similarly.

[0027] Figure 1 It is a flow chart of a long-wave infrared image denoising method;

[0028] Figure 2 This is a specific algorithm flow chart of SVD reconstruction of long-wave infrared images during the use of a long-wave infrared image denoising method;

[0029] Figure 3 This is a specific algorithm flow chart of median filtering in the process of using a long-wave infrared image denoising method;

[0030] Figure 4 This is the optical flow tracking effect of the original long-wave infrared image;

[0031] Figure 5 This is an optical flow tracking effect diagram of the reconstructed long-wave infrared image of the present invention. DETAILED DESCRIPTION

[0032] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments.

[0033] The components of the embodiments of the present invention generally described and shown in the drawings herein may be arranged and designed in a variety of different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the claimed invention, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0034] In the following, various embodiments of the present disclosure will be described more fully. The present disclosure may have various embodiments, and adjustments and changes may be made therein. However, it should be understood that there is no intention to limit the various embodiments of the present disclosure to the specific embodiments disclosed herein, but rather the present disclosure should be understood to cover all adjustments, equivalents and / or alternatives that fall within the spirit and scope of the various embodiments of the present disclosure.

[0035] Hereinafter, the terms "including", "having" and their cognates, which may be used in various embodiments of the present invention, are intended only to indicate specific features, numbers, steps, operations, elements, components or combinations of the foregoing items, and should not be understood as first excluding the existence of one or more other features, numbers, steps, operations, elements, components or combinations of the foregoing items or adding the possibility of one or more features, numbers, steps, operations, elements, components or combinations of the foregoing items.

[0036] Furthermore, the terms “first”, “second”, “third”, etc. are merely used for distinguishing descriptions and are not to be understood as indicating or implying relative importance.

[0037] Unless otherwise defined, all terms (including technical terms and scientific terms) used herein have the same meanings as those generally understood by those skilled in the art to which the various embodiments of the present invention belong. The terms (such as those defined in generally used dictionaries) will be interpreted as having the same meanings as the contextual meanings in the relevant technical field and will not be interpreted as having idealized meanings or overly formal meanings unless clearly defined in the various embodiments of the present invention.

[0038] The present invention provides a long-wave infrared image denoising method. In order to make the purpose, technical solution and advantages of the present invention more clear, the present invention is further described in detail below in conjunction with embodiments. Figure 1 The following is a flow chart of a long-wave infrared image denoising method provided in an embodiment, the method comprising:

[0039] Step 110: In a poor light environment such as fog, a long-wave infrared camera C with a calibrated internal parameter is set at t 0 Long-wave infrared images with noise interference are collected at all times 0 , image I0 Size is 240×640;

[0040] Step 120: Decompose the long-wave infrared image I using the SVD method 0 , and obtain the corresponding eigenvalue matrix Σ. Based on the definition of SVD decomposition, the eigenvalue matrix Σ is processed for the long-wave infrared image I 0 Reconstruct and obtain the reconstructed long-wave infrared image I 1 ;

[0041] Step 130: Use the median filtering method to process the reconstructed image I 1 , and get the denoised long-wave infrared image I 2 .

[0042] Furthermore, in step 120, the maximum singular value of the eigenvalue matrix Σ is set to zero, the average singular value is calculated, the remaining singular values ​​except the first singular value are replaced by the average singular value, and each singular value is assigned a decreasing weight according to its position to form a new eigenvalue matrix, such as Figure 2 As shown, it specifically includes the following sub-steps:

[0043] Step 121, decomposing the long-wave infrared image I using the SVD method 0 get:

[0044]

[0045] Among them, the size of matrix U is 240×240, which is composed of the left singular vector (Left Singular Vector, LSV) u i The matrix V is 320×320 in size and consists of the right singular vector (Right Singular Vector, RSV) v i Composition; the size of the matrix Σ is 240×320, and its diagonal elements represent the singular values ​​s of the image i ,s i In descending order, they are arranged from large to small on the diagonal of the eigenvalue matrix Σ. From the perspective of the image matrix, the singular value s i Represents the overall brightness of the image, the singular vector u i 、v i It represents the texture features in the image. At the same time, the maximum singular value can reflect the feature information with a higher proportion in the image, and the other singular values ​​can reflect the feature information with a lower proportion in the image.

[0046] Step 122, process the decomposed eigenvalue matrix Σ to make its maximum singular value s 1 =0.

[0047] Step 123, calculate the average singular value s avg

[0048]

[0049] Step 124, use the average singular value to replace all singular values ​​in the eigenvalue matrix Σ except the first singular value, and assign decreasing weights according to the position, so as to obtain the singular value s i The new eigenvalue matrix Σ′

[0050]

[0051] Step 125, reconstruct the image matrix using the new eigenvalue matrix Σ′, and the reconstructed long-wave infrared image I 1

[0052] I 1 =[U][Σ′][V] T (4)

[0053] Furthermore, in step 130, the median filter is used to process the image I 1 The process is as Figure 3 As shown, including:

[0054] Step 131, create a sliding window of size 3×3, the window is arranged in columns in image I 1 Swipe up to scroll through the images.

[0055] Step 132, calculate the processed pixel value p' in the window i

[0056] p′ i =Median{p 1 ,...,p 9},i=1,...,9 (5)

[0057] Among them, the set {p 1 ,...,p 9} is the image I in the window 1 The original pixel value, the Median function is to get the median of the set.

[0058] Step 133, the window slides across the entire image by column, and the pixel values ​​in the window are processed by median. After the traversal is completed, the window exits to obtain the processed image I 2 .

[0059] In order to further illustrate the effectiveness of the present invention in denoising, the original image I 0 and the reconstructed image I 2 At the same time, rotate 10° clockwise and use the LK (Lucas Kanade) optical flow method to track the optical flow between the original image and the rotated image. The results are shown in Figure 4. Figure 5The root mean square error RMSE is used to calculate the error of the tracking point, and the results are shown in the following table:

[0060] image Tracking point error / pixel <![CDATA[I 0 ]]> 50.1044 <![CDATA[I 2 ]]> 3.2068

[0061] Original long-wave infrared image I 0 Due to the problems of noise interference and low contrast, the error in feature tracking is large. 2 The error in feature tracking is smaller, with high precision and system robustness, and can provide basic data association for subsequent autonomous driving positioning and mapping functions, verifying the effectiveness of the method of the present invention.

[0062] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A long-wave infrared image denoising method, characterized in that: include: Step (1): In a weak light environment, a long-wave infrared camera C with calibrated internal parameters is set up. At time t0, a long-wave infrared image I0 with noise interference is collected. The size of image I0 is M×N, where M and N are the height and width of the image respectively. Step (2): Decompose the long-wave infrared image I0 using the SVD method to obtain the corresponding eigenvalue matrix Σ; process the eigenvalue matrix Σ based on the definition of SVD decomposition, reconstruct the long-wave infrared image I0, and obtain the reconstructed long-wave infrared image I1; the process of processing the eigenvalue matrix Σ is as follows: set the maximum singular value of the eigenvalue matrix Σ to zero, calculate the average singular value, replace the remaining singular values ​​except the first singular value with the average singular value, assign a decreasing weight to each singular value according to its position, and form a new eigenvalue matrix; Step (3): Use the median filtering method to process the reconstructed image I1 to obtain the denoised long-wave infrared image I2.

2. The method according to claim 1, characterized in that: The process of processing the eigenvalue matrix Σ in step (2) is: Using the SVD method to decompose the long-wave infrared image I0, we get: Among them, the matrix U is of size M×M, consisting of the left singular vector u i The matrix V is N×N in size and consists of the right singular vector v i Composition; the size of the matrix Σ is M×N, and its diagonal elements represent the singular values ​​s of the image i ,s i In descending order, they are arranged from large to small on the diagonal of the eigenvalue matrix Σ, where T represents the transpose operation; from the perspective of the image matrix, the singular value s i Represents the overall brightness of the image, the singular vector u i 、v i Represents the texture features in the image. At the same time, the maximum singular value can reflect the feature information with a high proportion in the image, and the remaining singular values ​​can reflect the feature information with a low proportion in the image. The decomposed eigenvalue matrix Σ is processed so that its maximum singular value s1 = 0, and the average singular value s is calculated. avg Use the average singular value to replace all singular values ​​except the first singular value in the eigenvalue matrix Σ, and assign decreasing weights according to their positions, so as to obtain the singular value s i The new eigenvalue matrix Σ′ Use the new eigenvalue matrix Σ′ to reconstruct the image matrix and obtain the reconstructed long-wave infrared image I1 I1=[U][Σ′][V] T 。 3. The method according to claim 1, characterized in that: The process of median filtering image I1 in step (3) is as follows: Suppose the sliding window size is K×K, the window slides on the image I1 by column, and the processed pixel value p in the window i 'for Among them, the collection is the original pixel value of image I1, and the Median function is to take the median of the set; the window slides across the entire image by column, and the pixel values ​​in the median processing window are obtained to obtain the processed image I2.

Images