Multi-period target recognition method based on laser radar and vision fusion in complex environment

Abstract

This invention discloses a multi-cycle target recognition method that fuses lidar and vision in complex environments. The method includes: preprocessing lidar point cloud data to extract features; calibrating the image and point cloud acquired by a monocular camera, adding RGB information to the lidar point cloud data; calculating target membership degrees and constructing a preliminary probability allocation function model; calculating the correlation matrix between evidence bodies based on the Pearson correlation coefficient, normalizing it, and calculating the credibility of each evidence body; using the credibility to perform a weighted average of the preliminary probability allocation function models of each evidence body to obtain the final weighted average evidence body, and fusing it according to the DS evidence combination rule; using the recognition result of the previous cycle as a new evidence body, reconstructing the probability allocation function, recalculating the correlation matrix between evidence bodies, and iterating multiple times to obtain the final fusion result. This invention features reasonable probability allocation, simple operation, low computational load, and higher target recognition accuracy.

Description

Technical Field

[0001] The present invention relates to the technical field of data fusion, and in particular to a multi-period target recognition method that integrates laser radar and vision in a complex environment. Background Art

[0002] In a complex environment, a single sensor cannot accurately obtain accurate information about the target. Multi-sensor data fusion can utilize the complementary performance between heterogeneous sensors to detect multiple features of the target and improve the accuracy of target recognition. The present invention uses a laser radar and a monocular camera for data fusion to obtain the shape, reflection intensity, number of point clouds and other features of the target. The DS evidence theory algorithm is often used in the field of multi-sensor target recognition due to its ability to process uncertain information, but when the evidence conflicts, it will produce erroneous fusion results. Therefore, the present invention uses the Pearson correlation coefficient to calculate the degree of correlation between the evidence bodies, thereby calculating the credibility of the evidence, and constructing a new evidence body by weighted averaging the original evidence body, and then performing the DS evidence theory rule combination.

[0003] The setting of probability distribution function is a very important part of DS evidence theory algorithm, which directly affects the fusion effect of the target. However, how to effectively solve the modeling and fusion of uncertain information in complex environment is still a problem to be solved. At present, expert experience and prior knowledge are mainly used to assign probabilities, but in the context of complex environment, with the changes of environment and the influence of noise, this method will cause large errors. Some literatures propose to construct a triangular fuzzy number model through the maximum, minimum and mean of data to calculate the probability distribution function, but the maximum and minimum values ​​cannot accurately reflect the discrete degree of target data. The present invention takes into account that most sensor recognition errors obey normal distribution. Therefore, based on fuzzy set theory, the present invention proposes to calculate the target membership according to the probability distribution function of normal distribution, so as to construct a basic probability distribution model of the target.

[0004] In a complex environment, due to sensor failure or noise interference, there will be serious conflicts between the evidence bodies constructed by sensors. At this time, the data of a single cycle cannot give accurate recognition results. Therefore, the present invention introduces multi-cycle data fusion to fuse the data collected in multiple cycles to improve the recognition accuracy. In addition, multi-cycle data fusion can solve the problem of target recognition errors caused by data association errors due to dense targets. Summary of the invention

[0005] The present invention provides a multi-period target recognition method by integrating laser radar and vision in a complex environment. The method uses laser radar and a monocular camera to respectively obtain multiple characteristic value data of the target, and at the same time introduces historical period information to perform fusion calculation on the data to realize recognition and classification of the target.

[0006] The technical solution to achieve the purpose of the present invention is: a multi-period target recognition method integrating laser radar and vision in a complex environment, comprising the following steps:

[0007] Step 1: Preprocess the laser radar point cloud data, remove the background by plane segmentation, and then perform point cloud clustering to extract features;

[0008] Step 2: Calibrate the image and point cloud collected by the monocular camera and add RGB information to the lidar point cloud data;

[0009] Step 3: Calculate the target membership based on the data feature information detected by the sensor, thereby constructing a preliminary probability distribution function model;

[0010] Step 4: Calculate the correlation matrix between the evidence bodies according to the Pearson correlation coefficient between the evidence bodies, perform normalization, and calculate the credibility of each evidence body;

[0011] Step 5: Use credibility to perform weighted average on the preliminary probability distribution function models of n evidence bodies to obtain the final weighted average evidence body, and fuse it according to the DS evidence combination rule;

[0012] Step 6: Use the recognition result of the previous cycle as a new body of evidence, reconstruct the probability distribution function, and then return to step 4 and iterate again multiple times to obtain the final fusion result.

[0013] Compared with the prior art, the present invention has the following significant advantages: (1) It combines the outputs of the laser radar and the monocular camera by means of feature fusion, which makes up for the shortcomings of the laser radar point cloud being sparse and the image information having no depth perception; (2) It constructs the membership by using the probability distribution function of the normal distribution, which is more in line with the actual distribution of the error, and the probability distribution is more reasonable, the operation is simple, the amount of calculation is small, and the zero confidence problem caused by the probability distribution being zero is avoided; (3) The credibility of the evidence body is constructed by the Pearson correlation coefficient, which can well reflect the correlation between the two evidence bodies, and then the DS rule combination is performed after the credibility weighted average is performed, which fully solves the problem that the conflict of evidence bodies cannot be resolved in the DS evidence theory; (4) It introduces the multi-period data fusion method, which makes full use of the historical recognition information and improves the accuracy of target recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 It is the overall flow chart of multi-cycle target recognition of LiDAR vision.

[0015] Figure 2 This is a schematic diagram of the installation of lidar and vision sensors. DETAILED DESCRIPTION

[0016] The multi-period target recognition method of laser radar and vision fusion in complex environment of the present invention uses laser radar and vision sensor to detect the target respectively, obtains multiple characteristic value data of the target, and uses the probability distribution function of normal distribution to calculate the target membership function, so as to construct a preliminary probability distribution model, and at the same time uses the recognition result of the historical period as a new body of evidence, combines the Pearson correlation coefficient between the evidence to perform weighted correction on the evidence body, and finally uses the DS evidence theory algorithm to fuse to obtain the final fusion result. Specifically, it includes the following steps:

[0017] Step 1: Preprocess the laser radar point cloud data, remove the background by plane segmentation, and then perform point cloud clustering to extract features;

[0018] Step 2: Calibrate the image and point cloud collected by the monocular camera and add RGB information to the lidar point cloud data;

[0019] Step 3: Calculate the target membership based on the data feature information detected by the sensor, thereby constructing a preliminary probability distribution function model;

[0020] Step 4: Calculate the correlation matrix between the evidence bodies according to the Pearson correlation coefficient between the evidence bodies, perform normalization, and calculate the credibility of each evidence body;

[0021] Step 5: Use credibility to perform weighted average on the preliminary probability distribution function models of n evidence bodies to obtain the final weighted average evidence body, and fuse it according to the DS evidence combination rule;

[0022] Step 6: Use the recognition result of the previous cycle as a new body of evidence, reconstruct the probability distribution function, and then return to step 4 and iterate again multiple times to obtain the final fusion result.

[0023] Furthermore, in step 3, the target membership is calculated according to the data feature information detected by the sensor, so as to construct a preliminary probability distribution function model, which is specifically:

[0024] According to the actual distribution of errors, the target membership is calculated using the probability distribution function of normal distribution;

[0025] Let T = {T1…T i …T m} is a set of test data measured by m sensors, X={X1…X k …X n} is the set of categories that the target may belong to, then the i-th feature value T of the test target T i Belongs to the kth target category X k Membership for:

[0026]

[0027] in is the kth target category X measured by the i-th sensor k The data mean, Yes X k The standard deviation of the data;

[0028] After calculating the membership matrix, normalize the membership:

[0029]

[0030] Thus we get the preliminary evidence matrix:

[0031]

[0032] satisfy Then a preliminary probability distribution function model is constructed.

[0033] Furthermore, in step 4, the formula is used to calculate any two pieces of evidence m i ,m j Pearson correlation coefficient ρ ij :

[0034]

[0035] Among them, cov represents the covariance between variables, Indicates m i ,m j The standard deviation of the variable, E is the expectation of the variable, ρ ij ∈[-1,1], the value of the correlation between two evidence bodies is less than or equal to 0 is set to 0.01, and the correlation between the two evidence bodies s ij It is expressed as:

[0036]

[0037] Then construct the correlation matrix of the evidence body and normalize it to calculate the evidence m i Credibility cred(m i ):

[0038]

[0039] Normalization:

[0040]

[0041] in

[0042] Furthermore, in step 5, the initial probability distribution function model of n evidence bodies is weighted averaged using the credibility to obtain the final weighted average evidence body The formula is as follows:

[0043]

[0044] Furthermore, in step 5, the fusion is performed according to the DS evidence combination rule, specifically:

[0045] The obtained weighted average evidence body is fused using the classic DS evidence theory algorithm:

[0046]

[0047] Among them, K is a normalization constant, which reflects the similarity between evidences:

[0048]

[0049] To facilitate understanding by technical personnel, the technical solutions in the present invention will be clearly and completely described below in conjunction with the accompanying drawings and specific embodiments. The contents mentioned in the implementation methods are not limitations of the present invention.

[0050] The present invention is used to solve the problem of low target recognition accuracy in complex environments. It makes full use of historical period information to improve the confidence of target recognition, can fully improve the accuracy of target recognition in complex environments, and can solve the problem of target recognition errors caused by dense targets.

[0051] The present invention is further described in detail below with reference to the accompanying drawings and specific embodiments.

[0052] Example

[0053] like Figure 1-2 As shown, the multi-period target recognition method of laser radar and vision fusion in a complex environment of the present invention includes the following:

[0054] Step 1: Receive the data collected by the lidar, use the random consistent sampling method to perform plane segmentation to remove background information, then use the Euclidean algorithm to cluster the point cloud and extract features from each point cloud cluster.

[0055] Step 2: Obtain the image data of the monocular camera, then calibrate it with the point cloud data, convert the image data into the point cloud, and obtain the color features.

[0056] Step 3: Based on the training data, the mean and standard deviation of each feature of different target categories are obtained, and the membership matrix of the measured target is calculated using the normal distribution:

[0057] Let T = {T1…T i…T m} is a set of test data, that is, the corresponding measurement value. Then the i-th feature of T belongs to the target X k The membership degree of is:

[0058]

[0059] in is the measured X k The data mean of the target i-th feature, Yes X k The standard deviation of the target data, after calculating the membership matrix, normalize the membership:

[0060]

[0061] Get the preliminary evidence matrix:

[0062]

[0063] satisfy

[0064] Step 4: Calculate the support of the body of evidence based on the Pearson correlation coefficient between the evidence:

[0065]

[0066] Among them, cov represents the covariance between variables, Indicates m i ,m j The standard deviation of the variable, E is the expectation of the variable, ρ ij ∈[-1,1].

[0067] Since the Pearson correlation coefficient varies from -1 to 1, in order to avoid the negative impact of negative credibility caused by negative correlation between evidence, the value of the correlation between two bodies of evidence is set to 0.01, so the correlation between the two bodies of evidence can be expressed as:

[0068]

[0069] Then the correlation matrix between the evidence is constructed as follows:

[0070]

[0071] The credibility of the evidence body is calculated after normalization according to the following formula:

[0072]

[0073] Step 5: Use the modified credibility of the evidence body to perform weighted averaging on the preliminary probability distribution model:

[0074]

[0075] Step 6: Perform DS rule fusion on the weighted average evidence:

[0076]

[0077] Among them, K is a normalization constant, which reflects the similarity between evidences:

[0078]

[0079] Step 7: Add the fusion result of the current cycle as a new evidence body to the probability distribution function of the next cycle, thereby constructing a probability distribution model containing multiple evidence bodies, and then substitute it into step 4 and iterate again to calculate the final fusion result.

[0080] The above description is only a preferred embodiment of the present invention and does not limit the present invention. It should be pointed out that for those skilled in the art, any improvements and modifications made within the scope of the principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A multi-period target recognition method based on laser radar and vision fusion in complex environments, characterized in that: The following steps are involved: Step 1: Preprocess the lidar point cloud data, remove the background by plane segmentation, and then perform point cloud clustering to extract features; Step 2: Calibrate the image and point cloud collected by the monocular camera and add RGB information to the lidar point cloud data; Step 3: Calculate the target membership based on the data feature information detected by the sensor, thereby constructing a preliminary probability distribution function model; Step 4: Calculate the correlation matrix between the evidence bodies according to the Pearson correlation coefficient between the evidence bodies, perform normalization, and calculate the credibility of each evidence body; Step 5: Use credibility to perform weighted average on the preliminary probability distribution function models of n evidence bodies to obtain the final weighted average evidence body, and fuse it according to the DS evidence combination rule; Step 6: Use the recognition result of the previous cycle as a new body of evidence, reconstruct the probability distribution function, and then return to step 4 and iterate again to obtain the final fusion result; In step 3, the target membership is calculated based on the data feature information detected by the sensor, so as to construct a preliminary probability distribution function model, which is specifically: According to the actual distribution of errors, the target membership is calculated using the probability distribution function of normal distribution; Let T = {T1…T i …T m } is a set of test data measured by m sensors, X={X1…X k …X n } is the set of categories that the target may belong to, then the i-th feature value T of the test target T i Belongs to the kth target category X k Membership for: in is the kth target category X measured by the i-th sensor k The mean of the data, Yes X k The standard deviation of the data; After calculating the membership matrix, normalize the membership: Thus we get the preliminary evidence matrix: satisfy Then a preliminary probability distribution function model is constructed.

2. The multi-period target recognition method of laser radar and vision fusion in complex environment according to claim 1 is characterized in that: In step 4, use the formula to calculate any two pieces of evidence m i ,m j Pearson correlation coefficient ρ ij : Among them, cov represents the covariance between variables, Indicates m i ,m j The standard deviation of the variable, E is the expectation of the variable, r ij ∈[-1,1], the value of the correlation between two evidence bodies is less than or equal to 0 is set to 0.01, and the correlation between the two evidence bodies s ij It is expressed as: Then construct the correlation matrix of the evidence body and normalize it to calculate the evidence m i Credibility cred(m i ): Normalization: in 3. The multi-period target recognition method of laser radar and vision fusion in complex environment according to claim 2 is characterized in that: In step 5, the credibility is used to perform weighted average on the preliminary probability distribution function model of n evidence bodies to obtain the final weighted average evidence body The formula is as follows:

4. The multi-period target recognition method of laser radar and vision fusion in complex environment according to claim 1 is characterized in that: In step 5, the fusion is performed according to the DS evidence combination rule, specifically: The obtained weighted average evidence body is fused using the classic DS evidence theory algorithm: Among them, K is a normalization constant, which reflects the similarity between evidences:

Images