A method for object feature part detection based on three-dimensional laser radar point cloud

Abstract

The application discloses a kind of based on three-dimensional laser radar point cloud and carries out the method for object feature part detection, step 1, obtains the three-dimensional laser point cloud data of object;Step 2, according to the distribution of laser point in three-dimensional laser point cloud in horizontal and vertical angle, three-dimensional laser point cloud is mapped to binary image;Step 3, the key feature point of feature part in object is identified from binary image;Step 4, the key feature point is mapped back to three-dimensional laser radar point cloud, and the three-dimensional laser radar point cloud coordinates of the feature part are obtained.The application can solve the problem that three-dimensional space point cloud is not complete due to the limited field of view of laser radar, and the position and attitude of the object in space are obtained by key point detection, to improve the accuracy and stability of the detection target.

Description

Technical Field

[0001] This invention relates to the field of lidar detection, and in particular to a method for detecting feature parts of an object based on a three-dimensional lidar point cloud. Background Technology

[0002] With the development of robotics and laser sensors, 3D LiDAR is being used more and more widely in the field of target detection. Especially in problems requiring target position or orientation, the point cloud information of 3D LiDAR often plays an important role.

[0003] In applications requiring precise location detection of targets in space or accurate extraction of a specific location from an object, traditional LiDAR point cloud matching algorithms are limited by factors such as the LiDAR field of view, the difficulty in acquiring the initial global LiDAR point cloud, and the disorder of the 3D point cloud. Algorithms based on camera and LiDAR fusion are severely affected by environmental interference and cannot guarantee detection accuracy and stability. Deep learning algorithms based on neural networks require large datasets and their inference speed is limited by the amount of 3D point cloud data. Summary of the Invention

[0004] In view of this, the present invention provides a method for detecting object feature parts based on three-dimensional lidar point clouds, which can solve the problem of incomplete three-dimensional spatial point clouds caused by the limited field of view of lidar, and obtain the position and orientation of objects in space by using key point detection, thereby improving the accuracy and stability of target detection.

[0005] To solve the above-mentioned technical problems, the present invention is implemented as follows.

[0006] A method for detecting object feature parts based on 3D lidar point clouds includes:

[0007] Step 1: Obtain the 3D laser point cloud data of the object;

[0008] Step 2: Based on the distribution of laser points in the 3D laser point cloud at horizontal and vertical angles, map the 3D laser point cloud to a binarized image;

[0009] Step 3: Identify key feature points of characteristic parts of the object from the binarized image;

[0010] Step 4: Reverse map the key feature points back to the 3D LiDAR point cloud to obtain the 3D LiDAR point cloud coordinates of the feature area.

[0011] Preferably, in step 2, mapping the point cloud to the binarized image is as follows:

[0012] Step 201: Calculate the laser point (x) i ,y i ,zi index pairs (m) i ,n i );

[0013] The lidar coordinate system is a traditional right-handed system. The lidar installation direction is defined as the x-axis directly in front, the y-axis to the left in front, and the z-axis directly above. The xy plane of the lidar coordinate system is parallel to the ground, and the positive z-axis is perpendicular to the ground and points upwards. The vertical angles of the lidar laser points are defined according to the angles defined internally by the lidar, and the horizontal angles are symmetrical about the x-axis. In the image coordinate system, the top-left pixel of the image is the origin, the x-axis points to the right, and the y-axis points downwards.

[0014] Let the laser point (x) i ,y i ,z i Angle θ in the vertical direction 1i It is the angle between the laser line corresponding to the laser point and the horizontal plane (the same as the internal definition of the lidar), the laser point (x i ,y i ,z i Angle θ in the horizontal direction 2i It is the angle between the laser line and the front of the 3D lidar along the x-axis; the angle θ of all laser points. 1i The set of vertical angles Ω1 of the three-dimensional lidar consists of the angles θ of all laser points. 2i The set of horizontal angles Ω2 of the three-dimensional lidar is formed; then,

[0015] Index m i For the set of angles Ω1 in the vertical direction of the 3D lidar, and θ 1i Index of the angle with the smallest error;

[0016] Index n i For the three-dimensional lidar azimuth direction angle set Ω2, the angle is related to θ 2i Index of the angle with the smallest error;

[0017] Step 202: Transfer the index pair (m) i ,n i Using the index as coordinates, the coordinate points pointed to by the index pair are marked in the binarized image, thereby realizing the mapping from the 3D laser point cloud to the binarized image.

[0018] Preferably, in step 201, the index m i The calculation formula is:

[0019]

[0020] m i =argmin{θ 1i -ω 1i},ω 1i ∈Ω1 (2)

[0021] The index n i The calculation formula is:

[0022]

[0023] n i =argmin{θ 2i -ω 2i},ω 2i ∈Ω2 (4)

[0024] Step 202 is as follows: Construct a binary image Image with a width of index m. i The maximum value m max The length is index n i The maximum value n max The index pairs (m) in the index set Index are sequentially... i ,n i The image is mapped to a binary image Image, and the mapping rule is shown in formula (5).

[0025]

[0026] Preferably, the key feature points for identifying characteristic parts of an object from a binarized image are:

[0027] Based on the characteristic of preserving the external shape of an object when mapping a 3D laser point cloud to a binarized image, key feature points are identified from the binarized image according to the shape characteristics of the feature parts of the object to be detected.

[0028] Preferably, when the feature part of the object has an outer sharp point and an inner hollow structure, the top sharp point of the outer contour is selected as key point A, and the intersection of the horizontal line of the vertex of the inner hollow contour and the outer contour is selected as key points B and C.

[0029] The key feature points for identifying characteristic parts of an object are:

[0030] Step 301: Extract the midline of the object from the binarized image;

[0031] Step 302: Designate the first marked point on the centerline from the top as key point A;

[0032] Step 303: Starting from key point A, find the position on the centerline where the marked point becomes an unmarked point, and use it as auxiliary point D. Obtain a straight line bc that passes through auxiliary point D and is perpendicular to the centerline. The first marked point on the straight line bc from left to right is key point B, and the last marked point is key point C.

[0033] Preferably, step 4 is: calculating the index pairs (m) of the three-dimensional laser point cloud obtained in step 2. i ,n i Find the index pairs that are close to key points A, B, and C respectively, and extract the laser point coordinates corresponding to the found index pairs to form a laser point coordinate set Ω corresponding to the three key points. A Ω B Ω C Calculate the mean of the coordinate set of each laser point to obtain the three-dimensional lidar point cloud coordinates of the three key points, that is, the three-dimensional lidar point cloud coordinates of the feature part.

[0034] Preferably, the index pairs (m) of the three-dimensional laser point cloud calculated from step 2 i ,n i The index pairs that are close to key points A, B, and C respectively are found as follows:

[0035] Traverse the index pairs (m) of the 3D laser point cloud calculated in step 2 i ,n i Find index pairs that satisfy the following conditions, which are the index pairs that are close to key points A, B, and C respectively;

[0036]

[0037]

[0038]

[0039] Where abs represents calculating the absolute value, T is a given threshold, (imgx A ,imgy A (imgx) B ,imgy B (imgx) C ,imgy C ) represents the coordinates of the key points in the binary image.

[0040] Beneficial effects:

[0041] (1) This invention maps three-dimensional point clouds into binarized images for key point lookup. Binarized images are ordered data structures, not disordered data like three-dimensional point clouds. When looking up the required feature points for ordered data, relevant information of surrounding points can be fully utilized, such as known vertical relationships, linear relationships, fitting relationships, etc., so as to obtain the local position of the object and then calculate the pose of the object in space, which can improve the accuracy and stability of detection.

[0042] (2) This invention provides a mapping algorithm that does not use the traditional bird's-eye view mapping method, which directly compresses the Z-direction of the point cloud. This method loses the shape of the object in space and the one-to-one correspondence between the point cloud and the pixel, thus making position detection impossible. In contrast, this invention maps the 3D laser point cloud to a binary image based on the distribution of laser points in the horizontal and vertical angles. This is equivalent to unfolding the entire detection surface of the lidar, which can preserve the external shape of the object and use the shape features of the object itself for detection.

[0043] (3) This invention can identify and detect any feature with structural characteristics. For feature with an external tip and an internal hollow structure, this invention only requires three key points A, B, and C to detect the feature. The identification of key points only requires addition and subtraction calculations and line fitting, which is simple, convenient and effective.

[0044] (4) In a preferred embodiment, when detecting key points A, B, and C, a threshold T is set to obtain a set of key points, and then the mean is calculated to obtain a more accurate detection result. The threshold T can be adjusted according to the required accuracy. Attached Figure Description

[0045] Figure 1 This is a flowchart illustrating the three-dimensional lidar point cloud object pose detection method of the present invention.

[0046] Figure 2 This is a schematic diagram of the radar beam distribution in this invention.

[0047] Figure 3 This is a diagram showing the points A, B, and C. Detailed Implementation

[0048] This method detects key points in a specific part of an object, solving the problem of detection and localization of a specific feature.

[0049] This invention provides a method for detecting feature parts of an object based on 3D LiDAR point clouds. It can detect key points within a specific part of an object, solving the problem of detecting and locating a specific feature part. (See also...) Figure 1 The core idea of ​​this method is:

[0050] Step 1: Obtain the 3D laser point cloud data of the object.

[0051] Step 2: Based on the distribution of laser points in the 3D laser point cloud at horizontal and vertical angles, map the 3D laser point cloud to a binarized image.

[0052] Step 3: Identify key feature points of the object's characteristic parts from the binarized image.

[0053] Step 4: Reverse map the key feature points back to the 3D LiDAR point cloud to obtain the 3D LiDAR point cloud coordinates of the feature area.

[0054] As can be seen, the invention maps 3D point clouds into binarized images for key point lookup. Binarized images are ordered data structures, unlike the disordered data of 3D point clouds. When looking up the required feature points for ordered data, relevant information of surrounding points can be fully utilized, such as known vertical relationships, linear relationships, fitting relationships, etc., thereby obtaining the local position of the object, and then calculating the pose of the object in space, improving the accuracy and stability of detection.

[0055] Furthermore, the three-dimensional lidar point cloud coordinates obtained by reverse mapping according to the present invention can be used to calculate the pose of objects in space.

[0056] The following embodiment illustrates the invention in detail. This embodiment demonstrates a mapping algorithm that avoids the traditional bird's-eye view mapping method, which directly compresses the Z-axis of the point cloud. This method loses the shape of objects in space and the one-to-one correspondence between the point cloud and pixels, thus making position detection impossible. Instead, this invention maps the 3D laser point cloud to a binarized image based on the distribution of laser points in the horizontal and vertical angles. This is equivalent to unfolding the image across the entire detection surface of the lidar, preserving the external shape of the object and utilizing its inherent shape features for detection.

[0057] Step 1: Obtain the 3D laser point cloud data of the object.

[0058] Step 2: Based on the distribution of laser points in the 3D laser point cloud at horizontal and vertical angles, map the 3D laser point cloud to a binarized image.

[0059] This process specifically includes steps 201-202.

[0060] Step 201: Calculate the laser point (x) i ,y i ,z i index pairs (m) i ,n i ).

[0061] The lidar coordinate system is a traditional right-handed system. The lidar installation direction is defined as the x-axis directly in front, the y-axis to the left in front, and the z-axis directly above. The xy plane of the lidar coordinate system is parallel to the ground, and the positive z-axis is perpendicular to the ground and points upwards. The vertical angles of the lidar laser points are defined according to the angles defined internally by the lidar, and the horizontal angles are symmetrical about the x-axis. In the image coordinate system, the top-left pixel of the image is the origin, the x-axis points to the right, and the y-axis points downwards.

[0062] Let the laser point (x) i ,y i ,z i Angle θ in the vertical direction 1i It is the angle between the laser line corresponding to the laser point and the horizontal plane, where the laser point (x) is the angle between the laser line and the horizontal plane. i ,y i ,z i Angle θ in the horizontal direction 2i It is the angle between the laser line and the front of the 3D lidar along the x-axis; the angle θ of all laser points. 1i The set of vertical angles Ω1 of the three-dimensional lidar consists of the angles θ of all laser points. 2i The set of horizontal angles Ω2 constitutes the three-dimensional lidar;

[0063] Then index m i For the set of angles Ω1 in the vertical direction of the 3D lidar, and θ 1i The index of the angle with the smallest error; index n i For the three-dimensional lidar azimuth direction angle set Ω2, the angle is related to θ 2i The index of the angle with the smallest error.

[0064] In a preferred embodiment, the error is calculated using the difference, then the index m i The calculation formula is:

[0065]

[0066] m i =argmin{θ 1i -ω 1i},ω 1i ∈Ω1 (2)

[0067] Similarly, index n i The calculation formula is:

[0068]

[0069] n i =argmin{θ 2i -ω 2i},ω 2i ∈Ω2 (4)

[0070] Where arcsin is the arcsine function, arctan is the arctangent function, and argmin{·} is the index in the set of the angle that minimizes ·.

[0071] Step 202: Transfer the index pair (m) i ,n iUsing the index as coordinates, the coordinate points pointed to by the index pair are marked in the binarized image, thereby realizing the mapping from the 3D laser point cloud to the binarized image.

[0072] The marking mentioned in this step refers to setting the coordinates to 0, 255, or other values, as long as the purpose of differentiation is achieved. In a preferred embodiment, the pixel position of the binarized image mapped by the laser point is marked as 255. Then step 202 is: construct a binarized image Image with a width of index m. i The maximum value m max The length is index n i The maximum value n max The index pairs (m) in the index set Index are sequentially... i ,n i The image is mapped to a binary image Image, and the mapping rule is shown in formula (5).

[0073]

[0074] Step 3: Identify key feature points of the object's characteristic parts from the binarized image.

[0075] In the above steps, because a mapping scheme based on two-dimensional index is adopted, the three-dimensional laser point cloud retains the characteristics of the external shape of the object when mapped to the binarized image. Knowing the shape characteristics of the feature parts of the object to be detected, key feature points can be identified from the binarized image by using the geometric relationship of the image.

[0076] See Figure 3 The example given describes an object with a pointed outer edge and an internal hollow structure, with the internal hollow structure having a corresponding internal hollow vertex. In practice, this structure might resemble a fixed-wing drone. For this structure, the pointed top edge of the outer contour is selected as key point A, and the intersection of the horizontal line of the internal hollow contour vertex and the outer contour is selected as key points B and C.

[0077] The method for identifying key feature points of feature regions from objects with the above structural features includes the following steps 301-303:

[0078] Step 301: Extract the midline of the object from the binarized image.

[0079] against Figure 2 The structure first involves traversing the image, recording the points in the same row where pixel values ​​change abruptly (from 0 to 255 or from 255 to 0), and recording the position of these points in the image coordinate system (imgx). left ,imgy left ) and (imgx left ,imgy left), and calculate the coordinates of the midline point (midx). row ,midy row As shown in calculation formula (6),

[0080]

[0081] All midline points form the set ROW.

[0082] Next, the midline fitting calculation is performed. The least squares method can be used to fit the midline point set to obtain the midline line equation y = k in the pixel coordinate system. mid *x+b mid , where k mid Let b be the slope of the line. mid The line intercept is calculated as shown in formula (7), where num represents the number of coordinates of the midline point.

[0083]

[0084] Step 302: Designate the first marker point on the centerline from the top as key point A.

[0085] In this step, the centerline is traversed from top to bottom to find the first point with a value of 255, which is recorded as key point A, with coordinates (imgx). A ,imgy A ).

[0086] Step 303: Starting from keypoint A, find the pixel on the fitted midline whose pixel value changes from 255 to 0 and use it as auxiliary point D. Draw a straight line bc perpendicular to the midline through auxiliary point D. Traverse the line bc from left to right. The first point marked is keypoint B and the last point marked is keypoint C.

[0087] In this step, the binarized image is traversed downwards from point A. The first pixel on the fitted midline whose value changes from 255 to 0 is marked as an auxiliary point D. A straight line bc, perpendicular to the midline and passing through point D, is obtained. The first marked point on line bc is keypoint B, and the last marked point is keypoint C. Keypoints B and C are symmetrical about the fitted midline, and the equation of the straight line passing through points B and C is y = k. bc *x+b bc As shown in the calculation formula (8), k bc To fit the slope of the midline, b bc To fit the midline intercept, (imgx D ,imgy D ( ) represents the coordinates of auxiliary point D in the image coordinate system.

[0088]

[0089] Perform a column traversal on the binarized image, and determine if there is a point with a value of 255 on the line bc. Count the first point as the keypoint B(imgx). B ,imgy B The last point is the key point C(imgx). C ,imgy C ).

[0090] Step 4: Reverse map the key feature points back to the 3D LiDAR point cloud to obtain the 3D LiDAR point cloud coordinates of the feature area.

[0091] In this step, the index pairs (m) of the three-dimensional laser point cloud calculated in step 2 are... i ,n i Find the index pairs that are close to key points A, B, and C respectively, and extract the laser point coordinates corresponding to the found index pairs to form a laser point coordinate set Ω corresponding to the three key points. A Ω B Ω C In a preferred embodiment, "close to" is calculated using absolute values. Then, the index pairs (m) of the three-dimensional laser point cloud obtained in step 2 are traversed. i ,n i Find index pairs that satisfy the following conditions, which are the index pairs that are close to key points A, B, and C respectively;

[0092]

[0093] Where abs represents calculating the absolute value, T is a given threshold, (imgx A ,imgy A (imgx) B ,imgy B (imgx) C ,imgy C ) represents the coordinates (X, Y) of the keypoint in the binary image. A ,X B ,X C ).

[0094] Next, the mean of the coordinate set of each laser point is calculated to obtain the three-dimensional lidar point cloud coordinates of the three key points, that is, the three-dimensional lidar point cloud coordinates of the feature part. As shown in the calculation formula (10), where num a ,num b ,num c The coordinates of the laser point Ω are respectively A Ω B Ω C The number of coordinates.

[0095]

[0096] This concludes the feature area detection.

[0097] Furthermore, the spatial coordinates (X, X) of three specific points of the rigid object in space are obtained. A ,X B ,X C After that, the accurate spatial coordinates of any point on the rigid object can be calculated through spatial geometric relationships. At the same time, the Euler angles of the object in space under its coordinate system can be obtained by using the spatial rotation theorem. In summary, the pose of the spatial object can be obtained.

[0098] Compared to other existing LiDAR point cloud feature extraction algorithms, this invention utilizes images obtained by mapping LiDAR point clouds, extracts features from the images, and then maps them back to 3D point clouds. It then detects and locates specific positions within the point cloud, achieving stable and accurate keypoint detection for LiDAR point cloud feature extraction, thereby obtaining the pose of objects in space. This solves the problem of previous LiDAR point cloud algorithms being unable to accurately detect the position of specific parts, improving stability and accuracy, and is worthy of widespread application.

[0099] The specific embodiments described above only illustrate the design principles of the present invention. The shapes and names of the components in this description may differ and are not limited. Therefore, those skilled in the art can modify or make equivalent substitutions to the technical solutions described in the foregoing embodiments; and these modifications and substitutions do not depart from the inventive spirit and technical solutions of the present invention, and should all fall within the protection scope of the present invention.

Claims

1. A method for object feature part detection based on a three-dimensional laser radar point cloud, characterized in that, The method comprises the following steps: Step 1, obtaining three-dimensional laser point cloud data of an object; Step 2, mapping the three-dimensional laser point cloud to a binary image according to the distribution of laser points in the three-dimensional laser point cloud in horizontal and vertical angles; Step 3, identifying key feature points of a feature part of the object from the binary image; Step 4, reversely mapping the key feature points to the three-dimensional laser radar point cloud to obtain three-dimensional laser radar point cloud coordinates of the feature part; In step 2, the mapping of the point cloud to the binary image is as follows: Step 201, calculate the index pair of the laser point (x, y) ) of the laser point (x, y) , ) Let the laser point be ( Angle in the vertical direction of the lidar coordinate system It is the angle between the laser line corresponding to the laser point and the horizontal plane (the same as the internal definition of the lidar), the laser point ( Angle in the horizontal direction It is the angle between the laser line and the front of the 3D lidar along the x-axis; the angle of all laser points. Composition of vertical angle set of 3D LiDAR The angles of all laser points Composition of the horizontal angle set of a 3D lidar ;but, Index for a three-dimensional laser radar vertical direction angle set with index of the angle with the smallest error Index for a set of azimuth angles of a three-dimensional laser radar with index of the angle with the smallest error Step 202, put the index pairs ( , Using the index as coordinates, the coordinate points pointed to by the index pair are marked in the binarized image, thereby realizing the mapping from the 3D laser point cloud to the binarized image; In step 201, the index The calculation formula is: (1) (2) The index The calculation formula is: (3) (4) Step 202 is as follows: Construct a binary image Image, with a width equal to the index. Maximum value The length is the index. Maximum value The index pairs in the index set Index are sequentially... , The image is mapped to a binary image Image, and the mapping rule is shown in formula (5). (5)。 2. The method of claim 1, wherein the key feature points of the feature part of the object are identified from the binary image according to the shape characteristics of the feature part of the object to be detected based on the characteristics of the object shape kept when the three-dimensional laser point cloud is mapped to the binary image.

3. The method of claim 2, wherein when the feature part of the object has an external sharp end and an internal hollow structure, the top sharp end of the external contour is selected as a key point A, and the intersection of the horizontal line of the top point of the internal hollow contour and the external contour is selected as key points B and C. The key feature points of the feature part of the object are identified as follows: Step 301: extracting a center line of the object in the binary image; Step 302: taking the first marked point of the center line from the top end as a key point A; Step 303: finding a position where the center line changes from a marked point to an unmarked point from the key point A downward as an auxiliary point D, obtaining a straight line bc perpendicular to the center line through the auxiliary point D, and taking the first marked point from left to right on the straight line bc as a key point B and the last marked point as a key point C. ​ 4. The method as described in any one of claims 2-3, wherein step 4 is: calculating the index pairs of the three-dimensional laser point cloud obtained in step 2 ( , Find the index pairs that are close to key points A, B, and C respectively, and extract the laser point coordinates corresponding to the found index pairs to form a set of laser point coordinates corresponding to the three key points. Calculate the mean of the coordinate set of each laser point to obtain the three-dimensional lidar point cloud coordinates of the three key points, that is, the three-dimensional lidar point cloud coordinates of the feature part.

5. The method as described in claim 4, wherein the index pairs of the three-dimensional laser point cloud calculated from step 2 ( , The index pairs that are close to key points A, B, and C respectively are found as follows: The index pairs of the three-dimensional laser point cloud calculated in step 2 are traversed to find index pairs satisfying the following conditions, that is, the index pairs close to the key points A, B and C respectively. , ​ wherein, abs denotes the calculation of the absolute value, T is a given threshold value, , , are the coordinates of the key points in the binarized image.

Images