Point cloud object recognition method based on intensity image compensation

Abstract

The present application relates to computer graphics and three-dimensional point cloud target recognition field, specifically to a kind of point cloud target recognition method based on intensity image compensation.The present application includes the following steps: (1) constructing the local reference coordinate system of point cloud;(2) calculating the deviation angle between the normal of each point in the local neighborhood of point cloud and local reference coordinate system;(3) extracting the contour information of the corresponding intensity image of point cloud, and the distance sequence between the contour barycenter and each point of contour line is carried out discrete fourier transform, and the result obtained is used as the contour DFT feature of target;(4) constructing model library, and identifying target by the way of feature matching.The present application enhances the feature information of target point cloud by representing the intensity feature of target, and identifies unknown object by the way of feature matching, which has the advantages of high description and high timeliness.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of computer graphics and three-dimensional point cloud target recognition, and particularly relates to a point cloud target recognition method based on intensity image compensation. BACKGROUND

[0002] Light Detection and Ranging (LiDAR) based on Avalanche Photon Diode (APD) has higher sensitivity and faster response time compared with ordinary laser radar due to its extremely high photon sensitivity and picosecond-level time resolution. It can detect weak reflection signals in low light environments, thereby obtaining higher quality range images and intensity images, and has been widely used in three-dimensional imaging. Point cloud target recognition based on LiDAR is a basic research direction of optoelectronics and computer vision technologies. With the development of LiDAR technology, point cloud target recognition is increasingly applied in 3D reconstruction, mapping, and autonomous driving.

[0003] At present, the target recognition of LiDAR point cloud is generally through two ways of directly recognizing the point cloud of the target or information fusion (infrared laser fusion or visible light laser fusion, etc.). The method of directly recognizing based on point cloud has sometimes lower recognition accuracy than the information fusion recognition method due to the single type of information. The recognition method based on information fusion needs to use multiple detectors to detect and image the target, and the heterogenous image registration needs to be performed before recognition, which increases the calculation time of the target recognition algorithm. In the document "Vehicle Recognition and Measurement Research Based on Sensor Information Fusion", a target recognition method based on fusion of laser radar point cloud and visible light camera is disclosed, which includes the following steps: first, the data of the two sensors are matched in space and time, the coordinate conversion of the data of the two sensors is completed, and then the target detection results of the two sensors are fused in the unified image plane based on the multi-hypothesis idea. The problem is that the data obtained by the heterogenous camera needs to be registered before recognition, which increases the running time of the overall algorithm.

[0004] Therefore, it is crucial to study an intensity feature + point cloud feature target recognition algorithm that takes into account efficiency and speed. SUMMARY

[0005] The present application provides a point cloud target recognition method based on intensity image compensation to overcome the problem of long running time in the prior art.

[0006] In order to achieve the purpose of the present application, the present application provides a point cloud target recognition method based on intensity image compensation, which includes the following steps:

[0007] Step 1: Input the point cloud and intensity image obtained by the APD lidar. For an input point cloud containing N points, P = {p1, p2, ..., p...} N}, construct its local reference frame;

[0008] Step 2: Calculate the deviation angle between the normal to each point in the local neighborhood of the point cloud and the local reference coordinate system;

[0009] Step 3: Calculate the contour DFT features of the intensity image corresponding to the point cloud;

[0010] Step 4: Build a model library and perform feature matching.

[0011] Furthermore, in step two above, the specific steps for calculating the point cloud normal deviation angle feature are as follows:

[0012] First, calculate the normal vector of the point cloud for a given point P. The barycenter coordinates of its local neighborhood for:

[0013]

[0014] in For P i The number of point clouds in a local neighborhood.

[0015] Then, establish P ik The covariance matrix Cov(P) ik )

[0016]

[0017] The semi-positive definite matrix Cov(P) is calculated. ik ):

[0018]

[0019] By solving Cov(P) ik From the eigenvalues ​​and eigenvectors of point P, we obtain point P. ik normal vector

[0020] Finally, calculate The deviation angles between the x, y, and z axes and the local reference coordinate system:

[0021]

[0022]

[0023]

[0024]

[0025]

[0026]

[0027] where are cosine values of the deviation angles between the x, y and z axes of the local reference coordinate system, are deviation angles between the x, y and z axes of the local reference coordinate system.

[0028] Further, the specific steps of calculating the contour DFT feature of the point cloud corresponding to the intensity image in step three are

[0029] First, the Canny operator is used to extract the contour of the target in the scene. The contour line of the target is composed of n pixels, and the complex representation of each point (x i ,y i ) is:

[0030] l i =x i +jy i ,i=1,2,...,n (10)

[0031] The barycentric coordinates b(x b ,y b ) of the contour line are:

[0032]

[0033] Select any point on the contour line as the starting point, and calculate the distance d(i) between each point on the contour line and the barycenter in turn:

[0034] d(i)=[(x i -x b ) 2 +(y i -y b ) 2 ] 1 / 2 (12)

[0035] Combine the n distances calculated into a distance set D={d1,d2,…,d n}. We do the discrete Fourier transform on D to get the Fourier feature of the target contour, as shown in equation (16).

[0036]

[0037] Further, the specific steps of constructing the local reference system in step one are

[0038] Select a point Pi Let P i be a point in the point cloud, and its coordinates be P i Let r be the support radius of the local neighborhood of P Let P i be the k-th point in the local neighborhood of P i The weight of the point cloud is increased for the points close to P

[0039]

[0040] where f i =||P ik -P i ||2. The eigenvalues of C are The corresponding eigenvectors are x + , The corresponding eigenvectors are z + . -x + and -z + represent the opposite direction of x + and z + , denoted as x - , z - . The x-axis and z-axis directions of the local reference coordinate system are calculated by formula (2):

[0041]

[0042] If the number of elements in is more than , the x-axis of the local reference coordinate system is selected as the direction of x + , otherwise as the direction of x - , and the selection of the z-axis is the same, and the direction of the y-axis is determined by x x z. The local reference coordinate system of P i is shown in formula (3).

[0043]

[0044] where and are the x-axis, y-axis and z-axis of the local reference coordinate system, respectively.

[0045] Compared with the prior art, the advantages of the present application are:

[0046] 1. The present application takes into account the efficient, high-speed strength feature + point cloud feature target recognition algorithm, which calculates the point cloud feature and intensity feature of the target, making the target feature representation more complete, and thus having higher description.

[0047] 2、In step two, the principal component analysis method is used to calculate the normal of the point cloud, and then the deviation angle features of the normal of each point in the local neighborhood of the point cloud and the x, y and z three axes of the local reference coordinate system are represented. Through this step, the distribution mode of the point cloud in space can be accurately represented, which has high description.

[0048] 3、In step three, the edge extraction + discrete Fourier transform method is used to represent the distribution features of the target intensity image profile. Since the APD laser radar can simultaneously acquire the point cloud and intensity images of the target, and the two images are one-to-one corresponding, there is no need for registration operation, so the feature description can be performed without heterogeneous image registration, which improves the speed of target recognition.

[0049] 4、Since step one constructs a local reference coordinate system, the target is divided into a plurality of local small regions, and parallel operation can be performed on each local region, thereby effectively improving the calculation efficiency of the algorithm. In summary, the entire recognition process has high description and high timeliness.

[0050] 5、The method of the present application is suitable for images that can collect intensity images, and is particularly suitable for processing APD laser radar based on intensity image target recognition. BRIEF DESCRIPTION OF DRAWINGS

[0051] Figure 1 The flowchart of the present application.

[0052] Figure 2 The photos of the APD array laser radar actual data are shown in (a) for the photo of the cup, (b) for the photo of the banana, (c) for the photo of the tank model, and (d) for the photo of the shoe.

[0053] Figure 3 The point cloud data of the detected object is shown in (a) for the point cloud data of the cup, (b) for the point cloud data of the banana, (c) for the point cloud data of the tank model, and (d) for the point cloud data of the shoe.

[0054] Figure 4 The intensity image of the detected object is shown in (a) for the intensity image of the cup, (b) for the intensity image of the banana, (c) for the intensity image of the tank model, and (d) for the intensity image of the shoe.

[0055] Figure 5 The profile information of the detected object is shown in (a) for the profile information of the cup, (b) for the profile information of the banana, (c) for the profile information of the tank model, and (d) for the profile information of the shoe.

[0056] Figure 6 The distance set of the detected object is shown in (a) for the distance set of the cup, (b) for the distance set of the banana, (c) for the distance set of the tank model, and (d) for the distance set of the shoe.

[0057] Figure 7 Flow chart for model library construction and feature matching. DETAILED DESCRIPTION

[0058] The application will be described in detail below with reference to the accompanying drawings and examples:

[0059] The application provides a point cloud target recognition method based on intensity image compensation, which enhances the feature information of the target point cloud by characterizing the intensity features of the target, and recognizes unknown objects by feature matching. Figure 1 The method is realized by the following steps:

[0060] Take a cup, a banana, a tank model and a shoe as examples to introduce the specific embodiment of the application. The corresponding photos of the objects are shown in Figure 2 .

[0061] Step 1: input the point cloud and intensity image obtained by APD laser radar detection to construct a local reference frame of the target. The point cloud images of the cup, banana, tank model and shoe are shown in Figure 3 .

[0062] Let P be a point in P P i The support radius of the local neighborhood is r. The coordinates of the kth point in the local neighborhood of P i Increase the weight of the point cloud close to P i The divergence matrix of the point cloud is:

[0063]

[0064] Where f i =||P ik -P i ||2. Perform eigenvalue decomposition on C, and the eigenvalues are The corresponding eigenvectors are x + , The corresponding eigenvectors are z + . -x + and -z + represent the opposite directions of x + and z + , denoted as x - , z - . The x-axis and z-axis directions of the local reference coordinate system are calculated by formula (2):

[0065]

[0066] If the number of elements in is more than Then the x-axis of the local reference coordinate system is chosen as x + Otherwise, x - Otherwise, x i The local reference coordinate system of P

[0067]

[0068] where and are the x-axis, y-axis and z-axis of the local reference coordinate system, respectively.

[0069] Step 2, calculate the deviation angle between the normal of each point in the local neighborhood of the point cloud and the local reference coordinate system:

[0070] First, the principal component analysis method on the covariance matrix is used to calculate the normal vector of the point cloud. For a point P in P , the coordinates of the center of gravity of its local neighborhood are

[0071]

[0072] where is the number of point clouds in the local neighborhood of P i . Then, the covariance matrix Cov(P ik ) of P ik is established

[0073]

[0074] The semi-positive definite matrix Cov(P ik ) is calculated:

[0075]

[0076] By solving the eigenvalues and eigenvectors of Cov(P ik ), the normal vector of point P ik is obtained Finally, the deviation angles between and the x, y and z axes of the local reference coordinate system are calculated:

[0077]

[0078]

[0079]

[0080]

[0081]

[0082]

[0083] where are cosine of the deviation angle between the x, y and z axis of the local reference coordinate system, are deviation angle between the x, y and z axis of the local reference coordinate system.

[0084] Step three, extract the contour information of the intensity image corresponding to the point cloud, and perform discrete Fourier transform on the distance sequence between the contour barycenter and each point on the contour line, and take the obtained result as the contour DFT feature of the target.

[0085] Intensity images of cup, banana, tank model and shoe are shown in Figure 4 ; contour information of cup, banana, tank model and shoe are shown in Figure 5 ; distance sets of cup, banana, tank model and shoe are shown in Figure 6 .

[0086] First, the Canny operator is used to extract the contour of the target in the scene. The contour line of the target is composed of n pixels, and each point (x i ,y i ) can be regarded as a vector, and its complex representation is:

[0087] l i =x i +jy i ,i=1,2,…,n (29)

[0088] The barycenter coordinates b(x b ,y b ) of the contour line are:

[0089]

[0090] Select any point on the contour line as the starting point, and calculate the distance d(i) between each point on the contour line and the barycenter in turn:

[0091] d(i)=[(x i -x b ) 2 +(y i -y b ) 2 ] 1 / 2 (31)

[0092] Combine the n distances calculated to form a distance set D={d1,d2,…,d n} We do discrete Fourier transform to D, and get the Fourier feature of the target profile, as shown in equation (16).

[0093]

[0094] Step four: constructing model library and feature matching

[0095] The present application uses APD array laser radar to collect point cloud data and intensity image data of various objects, and repeatedly performs steps one to three on the collected data to perform feature description; then the obtained results are stored to obtain a model library. When identifying a target to be identified (unknown object), steps one to three are performed, and the calculated results are matched with the model library. If a certain category in the model library is successfully matched, the target to be identified is regarded as this category; if the matching is wrong, the process is repeated until successful matching is achieved. The flow chart of constructing the model library and feature matching is shown in Fig. 2. Figure 7

[0096] For the cup, banana, tank model and shoe data collected by the APD array laser radar, the model library-feature matching mode is used to identify them, and the results are shown in Table 2.

[0097] Table 2 identification results

[0098] Time / s Precision Recall Cup recognition result 0.051 0.95 0.88 Banana recognition result 0.039 0.92 0.85 Tank model recognition result 0.090 0.88 0.82 Shoe recognition result 0.083 0.94 0.91

[0099] It can be seen that the whole identification process has high description and high timeliness by using the method of the present application.

[0100] The above embodiments are only the preferred technical solutions of the present application, and are not regarded as the limitation of the present application. The protection scope of the present application should be the technical solutions recited in the claims, including the equivalent replacement solutions of the technical features recited in the claims, i.e. the equivalent replacement improvements within this range are also within the protection scope of the present application.​

Claims

1. A point cloud object recognition method based on intensity image compensation, characterized in that: It is realized by the following steps: Step one, input the point cloud and intensity image obtained by APD laser radar detection, for the input point cloud P = {p1, p2, ···, pN} containing N points, construct its local reference system; N} Step two, calculating the deviation angle between the normal of each point in the local neighborhood of the point cloud and the local reference coordinate system; Step three, calculating the contour DFT feature of the corresponding intensity image of the point cloud; Step four, constructing a model library and feature matching; In the step two, the specific steps of calculating the normal deviation angle feature of the point cloud are First, the normal vector of the point cloud is calculated, for a point in P The barycentric coordinates of its local neighborhood are: wherein P is P i number of points in the local neighborhood Then, the covariance matrix Cov(P ik ) of P ik ) is established A semidefinite matrix Cov(P ik ) is computed By solving the eigenvalues and eigenvectors of Cov(P ik ), the normal vector of point P ik is obtained Finally, the deviations of the x, y and z axes of the local reference frame from the x, y and z axes of the global reference frame are calculated as follows: deviation angles between the x, y and z axes of the local reference frame and the x, y and z axes of the wherein respectively are cosine values of the deviation angles between the x, y and z three axes of the local reference coordinate system, respectively are deviation angles between the x, y and z three axes of the local reference coordinate system; In the step three, the specific steps of calculating the contour DFT feature of the corresponding intensity image of the point cloud are First, the Canny operator is used to extract the contour of the target in the scene; the contour line of the target is composed of n pixels, and the complex representation of each point (x i ,y i ) is as follows: l i = x i + jy i , i = 1, 2, ···, n (13) The barycentric coordinates b(x b ,y b ) of the contour line are: Selecting an arbitrary point in the contour line as the starting point, and sequentially calculating the distance d(i) between each point in the contour line and the center of gravity: d(i) = [(x i - x b ) 2 + (y i - y b ) 2 ] 1 / 2 (15) The calculated n distances are combined into a distance set D = {d1, d2, ..., dn}. n We perform a discrete Fourier transform on D to obtain the Fourier features of the target contour, as shown in equation (16).

2. The point cloud object recognition method based on intensity image compensation according to claim 1, characterized in that: In the step one, the specific steps of constructing the local reference system are Select a point P in P i , let the coordinate of P i be P i , the support radius of the local neighborhood is r; , the coordinate of the kth point in the local neighborhood of P i ; the weight of the point cloud close to P i is increased, and the scatter matrix of the point cloud is: where f i = ||P ik -P i ||2, C is subjected to eigenvalue decomposition, and the eigenvalues are The corresponding eigenvectors are x + , The corresponding eigenvectors are z + , -x + and -z + represent the opposite directions of x + and z + , denoted as x - , z - , the x-axis and z-axis directions of the local reference coordinate system are calculated by formula (2): if The number of elements in it is greater than Then the x-axis of the local reference coordinate system is chosen as x + Direction, otherwise x - The direction of the z-axis is chosen in the same way, and the direction of the y-axis is determined by x × z. i The local reference coordinate system is shown in equation (3); wherein and x, y, and z are the x-axis, y-axis, and z-axis of the local reference coordinate system, respectively.

Images