Point cloud intensity adaptive filtering method based on multi-sensor fusion

Abstract

The application discloses a point cloud intensity self-adaptive filtering method based on multi-sensor fusion and relates to the technical field of mine unmanned driving point cloud filtering. The method introduces a fog and dust concentration recognition algorithm based on PCA, constructs a fog and dust concentration point cloud intensity filtering parameter self-adaptive adjustment model, and can self-adaptively adjust the point cloud intensity filtering parameter according to the environment, so that high-precision filtering of laser radar point cloud data under different concentrations of fog and dust is realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of mine unmanned point cloud filtering, and particularly relates to a point cloud intensity adaptive filtering method based on multi-sensor fusion. BACKGROUND

[0002] In the field of unmanned driving, laser radar is often used to determine obstacle information on the path, and the accurate determination of obstacle information by laser radar is a key to realizing safe unmanned driving.

[0003] At present, in the underground environment with multiple water mists and multiple dusts, water mist particles and dust particles can scatter light and return to the radar receiver, and the perception performance of the laser radar is affected, resulting in that accurate point cloud data cannot be obtained, thereby causing obstacle misidentification. In the field of point cloud filtering, few methods can process and filter point cloud data in the underground environment with multiple water mists and multiple dusts, which leads to that an unmanned vehicle in the underground cannot accurately identify and determine obstacles.

[0004] Therefore, it is urgent to develop a filtering technology with good filtering performance and strong environmental adaptability in a harsh environment. SUMMARY

[0005] In view of the above problems, the present application provides a point cloud intensity adaptive filtering method based on multi-sensor fusion, which uses camera image data to capture and quantify the current scene mist and dust concentration, adaptively controls the mist and dust concentration parameter to filter out mist and dust point clouds, and realizes effective filtering of dust-free point clouds by the laser radar in a high-water-mist and high-dust environment.

[0006] According to the purpose of the present application, a point cloud intensity adaptive filtering method based on multi-sensor fusion is provided, which comprises the following steps:

[0007] Step 1, collecting and analyzing laser radar sensor point cloud data and binocular camera image data;

[0008] Step 2, calculating the atmospheric light value in the current environment by the PCA-based atmospheric light value calculation method according to the camera image data;

[0009] Step 3, calculating the mist and dust concentration D according to the measured atmospheric light value AT and the standard atmospheric light value A c ;

[0010] Step 4, calibrating different mist and dust concentration point cloud intensity filtering parameters, and constructing a mist and dust concentration point cloud intensity filtering parameter adaptive adjustment model;

[0011] Step 5, based on the adaptive adjustment point cloud intensity filtering model, outputting the optimal intensity filtering parameter I in real time according to the system acquisition information.

[0012] Preferably, the atmospheric light value calculation method in step 2 comprises the following steps:

[0013] Step 2.1, calculate the covariance matrix of the haze image index parameter matrix:

[0014] The brightness, saturation, contrast, variance, information entropy, mean square error, correlation texture, signal-to-noise ratio, smoothness, noise analysis, average gradient, and offset of the haze picture are used as index parameters. Each image index parameter is pulled into a one-dimensional row vector to form an index parameter matrix. The number of rows of the index parameter matrix is the number of index parameters. The covariance matrix of the index parameter matrix is calculated as follows:

[0015]

[0016] Where ∑ is the covariance matrix of the index parameters; X is the standard parameter matrix of the haze image; is the matrix composed of each index parameter minus the mean value of the index parameter;

[0017] The eigenvalues λ of the covariance matrix are calculated. The first m eigenvalues of ∑ are the variances of the first m original variables.

[0018] Step 2.2, calculate the comprehensive index according to the covariance matrix of the index parameter matrix, as follows:

[0019]

[0020] Where F is the comprehensive index parameter, X is the index parameter matrix, and α is the transformation matrix of the comprehensive index parameter replacing the index parameter. Given F m , they are not correlated with each other, and Var(F i ) = α′ i ∑α i , i = 1, 2…m; m is the number of comprehensive index parameters, and p is the number of index parameters; F1 is the maximum variance among F1, F2, …F m .

[0021] Step 2.3, calculate the variance contribution according to formula 3 and select the comprehensive index:

[0022]

[0023] Step 2.4, determine the principal component variable according to formula 4 by calculating the variance cumulative contribution rate G (m) :

[0024]

[0025] When the cumulative contribution rate of the principal component is greater than 85%, the first m comprehensive index is confirmed as the principal component.

[0026] Step 2.5, calculate the atmospheric light value according to formula 5:

[0027]

[0028] wherein ω i is a weight,

[0029] Preferably, in step 3, the calculation method of the dust concentration D c includes the following steps:

[0030] Step 3.1, taking the atmospheric light value of the mine upper yard image as the standard atmospheric light value A, which can be calculated through step 2;

[0031] Step 3.2, judging the dust concentration D T according to the measured atmospheric light value A c and the standard atmospheric light value A;

[0032] According to formula (6), D c is calculated:

[0033] D c = 1-(A T -A) / A (6).

[0034] Preferably, in step 4, the optimal filtering parameter I corresponding to different dust concentrations is collected, and the data collection form is (D c , I); the collected data is fitted through the least square method to obtain a point cloud intensity filtering parameter adaptive adjustment model:

[0035] I=f(D c , k);

[0036] k is the coefficient obtained by the least square method.

[0037] Compared with the prior art, the point cloud intensity adaptive filtering method based on multi-sensor fusion disclosed in the present application has the following advantages:

[0038] The present application introduces a PCA-based dust concentration recognition algorithm to solve the problem of poor recognition effect of laser radar point cloud in the underground multi-water mist and multi-dust environment, and constructs a dust concentration point cloud intensity filtering parameter adaptive adjustment model. The model can adaptively adjust the point cloud intensity filtering parameter according to the environment, and realize high-precision filtering of laser radar point cloud data under different concentrations of dust. BRIEF DESCRIPTION OF DRAWINGS

[0039] In order to more clearly illustrate the technical scheme of the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiment or prior art description. Obviously, the drawings in the following description are only some embodiments of the present application, and those skilled in the art can obtain other drawings according to these drawings without creative labor.

[0040] Figure 1 A flowchart of the filtering method provided in an embodiment of the present invention.

[0041] Figure 2 This is a structural diagram of the filtering system disclosed in this invention.

[0042] Figure 3 This is a comparison diagram of the point cloud filtering effect before and after, provided in an embodiment of the present invention.

[0043] In the diagram: 1-LiDAR sensor; 2-Binocular camera; 3-Industrial control computer; 4-Switch. Detailed Implementation

[0044] The specific embodiments of the present invention will be briefly described below with reference to the accompanying drawings. Obviously, the described embodiments are merely some, not all, embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0045] Figures 1-3 A preferred embodiment of the present invention is shown and analyzed in detail.

[0046] like Figure 2 The point cloud intensity adaptive filtering method based on multi-sensor fusion shown uses a system including a lidar sensor 1 for acquiring three-dimensional point cloud data; a binocular camera 2 for acquiring image information data; a switch 4 for connecting the binocular camera 2, lidar sensor 1, and industrial control computer 3; and an industrial control computer 3 for directly connecting to the switch 4 to acquire and calculate data.

[0047] like Figure 1 As shown, the present invention also discloses a point cloud intensity adaptive filtering method based on multi-sensor fusion, which includes the following steps:

[0048] Step 1: Collect and analyze point cloud data from lidar sensor 1 and image data from binocular camera 2.

[0049] Step 2: Calculate the atmospheric light value under the current environment based on the camera image data using the PCA-based atmospheric light value calculation method.

[0050] Step 2.1: Calculate the covariance matrix of the fog and dust image index parameter matrix:

[0051] Brightness, saturation, contrast, variance, information entropy, mean square error, correlation texture, signal-to-noise ratio, smoothness, noise analysis, average gradient, and offset of the fog image are used as index parameters. Each image index parameter is represented as a one-dimensional row vector to form an index parameter matrix, where the number of rows in the matrix equals the number of index parameters. The covariance matrix of the index parameter matrix is ​​calculated using the following formula:

[0052]

[0053] Where ∑ is the covariance matrix of the index parameters; X is the standard parameter matrix of the fog image; It is a matrix composed of each indicator parameter minus its mean;

[0054] Calculate the eigenvalues ​​λ of the covariance matrix. The first m eigenvalues ​​of the ∑ are the variances of the first m original variables.

[0055] Step 2.2: Calculate the comprehensive index based on the covariance matrix of the index parameter matrix, using the following formula:

[0056]

[0057] Where F is the comprehensive index parameter, X is the index parameter matrix, and α is the transformation matrix that substitutes the comprehensive index parameter for the index parameter; given F m They are unrelated to each other and have Var(F) i )=α′ i ∑α i i = 1, 2, ..., m; m is the number of comprehensive index parameters, p is the number of index parameters; F1 is F1, F2, ..., F m The biggest difference is between China and the US.

[0058] Step 2.3: Calculate the variance contribution rate according to Equation 3 and select a comprehensive index:

[0059]

[0060] Step 2.4: Calculate the cumulative variance contribution rate G according to Equation 4. (m) Determine the principal component variables:

[0061]

[0062] When the cumulative contribution rate of the principal components is greater than 85%, the top m comprehensive indicators can be selected as principal components.

[0063] Step 2.5: Calculate the atmospheric light value according to Equation 5:

[0064]

[0065] Where, ω i As weight,

[0066] Step 3, calculating the fog dust concentration D according to the measured atmospheric light value A T , the standard atmospheric light value A c .

[0067] Step 3.1, taking the atmospheric light value of the mine upper station image as the standard atmospheric light value A, which can be calculated through step 2;

[0068] Step 3.2, judging the fog dust concentration D according to the measured atmospheric light value A T , the standard atmospheric light value A c .

[0069] According to formula (6), D c is calculated:

[0070] D c = 1- (A T -A) / A (6).

[0071] Step 4, calibrating the point cloud intensity filtering parameters of different fog dust concentrations, and constructing a point cloud intensity filtering parameter adaptive adjustment model; collecting the optimal filtering parameters I corresponding to different fog dust concentrations, the data collection form is (D c , I); the collected data is fitted through the least square method to obtain the point cloud intensity filtering parameter adaptive adjustment model:

[0072] I = f (D c , k);

[0073] k is the coefficient obtained by the least square method.

[0074] Step 5, according to the system collected information, the optimal intensity filtering parameter I is output in real time based on the adaptive adjustment point cloud intensity filtering model.

[0075] The above description of the disclosed embodiments enables a person skilled in the art to implement and use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit and scope of the present application. Therefore, the present application will not be limited to these embodiments shown herein, but will be consistent with the widest scope of the principles and novel features disclosed herein.

Claims

1. A multi-sensor fusion based point cloud intensity adaptive filtering method, characterized in that, The method comprises the following steps: Step 1, collecting and analyzing the point cloud data of the laser radar sensor (1) and the image data of the binocular camera (2); Step 2, calculating the atmospheric light value under the current environment according to the camera image data through the PCA-based atmospheric light value calculation method; Step 3, Calculate the fog density D according to the measured atmospheric light value A T ; Standard atmospheric light value A c ; Step 4, calibrating the point cloud intensity filtering parameters of different fog and dust concentrations, and constructing a fog and dust concentration point cloud intensity filtering parameter adaptive adjustment model; Step 5, according to the system acquisition information, based on the adaptive adjustment point cloud intensity filtering model, the optimal intensity filtering parameter I is output in real time.

2. The method of claim 1, wherein, The atmospheric light value calculation method in step 2 comprises the following steps: Step 2.1, calculating the covariance matrix of the fog and dust image index parameter matrix: The luminance, saturation, contrast, variance, information entropy, mean square error, correlation texture, signal-to-noise ratio, smoothness, noise analysis, average gradient, and offset of the fog and dust picture are taken as index parameters, and each image index parameter is pulled into a one-dimensional row vector to form an index parameter matrix. The number of rows of the index parameter matrix is the number of index parameters. The covariance matrix of the index parameter matrix is calculated, and the formula is as follows: Wherein, ∑ is the covariance matrix of index parameters; X is the standard parameter matrix of the fog dust image; is the matrix composed of each index parameter minus the mean value of the index parameter; Calculate the eigenvalue λ of the covariance matrix ∑, and the first m eigenvalues of ∑ are the variances of the first m original variables; Step 2.2, calculating the comprehensive index according to the covariance matrix of the index parameter matrix, the formula is as follows: Wherein, F is the comprehensive index parameter, X is the index parameter matrix, and a is the transformation matrix of the comprehensive index parameter instead of the index parameter; F is known m and are not correlated with each other, and Var(F i ) = a′ i ∑a i , i = 1, 2…m; m is the number of comprehensive index parameters, and p is the number of index parameters; F1 is the maximum variance among F1, F2, …, F m . Step 2.3, calculating the variance contribution according to formula 3, and selecting the comprehensive index: Step 2.

4. Calculate the contribution rate G of variance according to Formula 4 (m) Determine principal component variables: When the cumulative contribution rate of the principal component is greater than 85%, the first m comprehensive indexes are confirmed as the principal components; Step 2.5, calculating the atmospheric light value according to formula 5: where ω i is a weight, 3. The method of claim 1, wherein, In step 3, the concentration D of the mist and dust c is calculated by the following steps: Step 3.1, taking the atmospheric light value of the mine upper yard image as the standard atmospheric light value A, which can be calculated through step 2; Step 3.2, judging the fog and dust concentration D according to the measured atmospheric light value A T , standard atmospheric light value A c ; D is calculated according to formula (6) c : D c = 1 - (AT - A) / A (6).

4. The method of claim 1, wherein, In step 4, the optimal filtering parameters I corresponding to different mist and dust concentrations are collected, and the data collection form is (D c , I); the collected data is fitted by the least square method to obtain a point cloud intensity filtering parameter adaptive adjustment model: I = f(D c , k); k is the coefficient obtained by least square fitting.

Images