Curb recognition method, device, intelligent device and storage medium

Abstract

This application applies to the field of robotics and provides a method, apparatus, intelligent device, and storage medium for identifying curbs. The method comprises: obtaining a keyframe laser point cloud within a preset area of ​​a robot's environment; sequentially arranging the keyframe laser point cloud to obtain arrangement information of laser points in the keyframe laser point cloud; performing neighborhood distance detection on the laser points based on the arrangement information, and constructing a set of candidate curb points based on the results of the neighborhood distance detection; clustering the candidate curb points in the set of candidate curb points, and identifying curb boundaries based on the clustering results. This method can automatically detect curbs, allowing the robot to accurately identify curbs and effectively constrain the robot's driving area, thereby ensuring safe movement of the robot.

Description

Technical Field

[0001] The present application relates to the field of robotics, and in particular to a curb recognition method, apparatus, intelligent device, and storage medium. Background Art

[0002] With the rapid development of science and technology, the intelligent robot industry has also developed rapidly. There are more and more robots of various types, and it is also common to use robots to assist or replace some manual work. For example, substations use robots to conduct power inspections.

[0003] Substation scenarios involve a variety of roads and terrain, limiting the areas within which wheeled robots can navigate. When navigating substation roads, wheeled robots rely on accurate positioning to ensure safe movement. Occasional positioning failures can cause the robot to derail and exit the designated safe zone, resulting in serious equipment damage.

[0004] In view of this, in order to ensure that the inspection robot can safely perform inspection tasks in the substation, how to enable the robot to accurately identify the roadside and effectively constrain the machine's driving area to ensure the robot's safe movement is an issue that needs to be considered at present. Summary of the Invention

[0005] The embodiments of the present application provide a curb recognition method, apparatus, intelligent device, and storage medium, which enable a robot to accurately identify curbs and effectively constrain the machine's driving area to ensure the robot's safe movement.

[0006] In a first aspect, an embodiment of the present application provides a curb recognition method, comprising:

[0007] Obtain keyframe laser point cloud in the preset area of ​​the robot's environment;

[0008] Arranging the key frame laser point cloud in order to obtain arrangement information of laser points in the key frame laser point cloud;

[0009] Performing neighborhood distance detection on the laser points based on the arrangement information, and constructing a set of candidate roadside points according to the results of the neighborhood distance detection;

[0010] Clustering is performed on the candidate roadside points in the candidate roadside point set, and roadside boundaries are identified based on the clustering results.

[0011] In a possible implementation of the first aspect, the arrangement information includes a laser beam number and a column number, and the step of orderly arranging the key frame laser point cloud to obtain the arrangement information of the laser points in the key frame laser point cloud includes:

[0012] Arrange the key frame laser point cloud in order, and obtain the vertical angle and horizontal angle of the laser point in the key frame laser point cloud;

[0013] Determine the laser beam number of the laser point according to the vertical angle;

[0014] The column number of the column where the laser point is located is determined according to the horizontal angle.

[0015] In a possible implementation of the first aspect, the step of determining the laser beam number of the laser point according to the vertical angle includes:

[0016] Determine the beam number of laser point P according to the following formula:

[0017] row P =(α+γ) / 2,

[0018] Among them, row P represents the laser beam number of the laser point P, α represents the vertical angle value of the laser point P, and γ is a preset angle value;

[0019] The step of determining the column number of the column where the laser point is located according to the horizontal angle includes:

[0020] The column number of the column where the laser point P is located is determined according to the following formula:

[0021] col P =β / b,

[0022] Among them, col P represents the column number of the column where the laser point P is located, β represents the horizontal angle value of the laser point P, and b is a preset threshold.

[0023] In a possible implementation of the first aspect, the step of performing neighborhood distance detection on the laser points based on the arrangement information, and constructing a set of candidate roadside points according to the result of the neighborhood distance detection, includes:

[0024] Marking ground object points based on the arrangement information, and removing laser points marked as ground object points from the key frame laser point cloud, wherein the ground object points are laser points irradiated on the ground object surface;

[0025] A neighborhood distance detection is performed on the target laser point to construct a set of candidate roadside points, wherein the target laser point is the laser point remaining in the key frame laser point cloud after the ground object point is removed.

[0026] In a possible implementation of the first aspect, the arrangement information includes a laser beam number and a column number, and the step of marking ground object points based on the arrangement information and removing the laser points marked as ground object points from the keyframe laser point cloud includes:

[0027] Marking the laser points in the key frame laser point cloud whose laser beam numbers are greater than the preset beam numbers as ground feature points; and / or,

[0028] Calculate the horizontal distance between a laser point having a laser beam number less than or equal to a preset beam number and an adjacent laser point in the key frame laser point cloud, where the adjacent laser point refers to a laser point having a laser beam number adjacent to the laser point but having the same column number as the laser point;

[0029] If the horizontal distance is less than a preset horizontal distance threshold, marking the laser point as the ground feature point;

[0030] The laser points marked as the ground object points are removed from the key frame laser point cloud.

[0031] In a possible implementation of the first aspect, the arrangement information includes a laser beam number and a column number, and the step of performing neighborhood distance detection on the target laser point and constructing a set of candidate roadside points includes:

[0032] Acquire a neighborhood point set of the target laser point, wherein the neighborhood point set includes the target laser point and a preset number of neighborhood points thereof;

[0033] Performing a bird's-eye projection on the key frame laser point cloud, and obtaining a two-dimensional plane distance between the target laser point in the neighborhood point set and the field point based on the bird's-eye projection;

[0034] Sort the two-dimensional plane distances between the target laser point and the field point in the neighborhood point set by size, and calculate the difference between the maximum two-dimensional plane distance and the minimum two-dimensional plane distance in the neighborhood point set;

[0035] If the difference is greater than a preset difference threshold, the target laser point is determined as a candidate roadside point, and a candidate roadside point set is constructed.

[0036] In a possible implementation of the first aspect, the step of clustering the candidate roadside points in the candidate roadside point set and identifying the roadside boundary based on the clustering result includes:

[0037] performing clustering processing on the candidate roadside points in the candidate roadside point set;

[0038] Determining the shape of the curb according to the result of the clustering process;

[0039] A fitting method corresponding to the curb shape is selected for fitting, and the curb boundary is identified based on the fitting result.

[0040] In a possible implementation of the first aspect, before the step of clustering the candidate roadside points in the candidate roadside point set, the method includes:

[0041] Obtaining the three-dimensional coordinates of a candidate roadside point and the three-dimensional coordinates of q neighboring points on the laser beam where the candidate roadside point is located;

[0042] Calculate the covariance matrix and perform eigenvalue decomposition based on the three-dimensional coordinates of the candidate roadside point and the q neighborhood points;

[0043] Determine the eigenvector corresponding to the maximum eigenvalue among the decomposed eigenvalues ​​as the main direction vector of the candidate roadside point;

[0044] Based on the main direction vector, comparing the main direction of the candidate roadside point with the target roadside direction;

[0045] The candidate roadside points whose comparison results are greater than a preset direction difference threshold are removed from the candidate roadside point set, wherein the target roadside direction is the roadside direction corresponding to the adjacent laser beams of the candidate roadside points.

[0046] In a second aspect, an embodiment of the present application provides a curb recognition device, comprising:

[0047] A point cloud acquisition unit, used to acquire key frame laser point clouds within a preset area of ​​the robot's environment;

[0048] An information acquisition unit, configured to arrange the key frame laser point cloud in an orderly manner to obtain sorting information of laser points in the key frame laser point cloud;

[0049] a set construction unit, configured to perform neighborhood distance detection on the laser points based on the sorting information, and construct a set of candidate roadside points according to the result of the neighborhood distance detection;

[0050] The roadside recognition unit is used to cluster the candidate roadside points in the candidate roadside point set and identify the roadside boundary according to the clustering result.

[0051] In a possible implementation of the second aspect, the arrangement information includes a laser beam number and a column number, and the information acquisition unit includes:

[0052] An angle acquisition module, used to arrange the key frame laser point cloud in order and obtain the vertical angle and horizontal angle of the laser point in the key frame laser point cloud;

[0053] a beam number determination module, configured to determine the laser beam number of the laser point according to the vertical angle;

[0054] The column number determination module is used to determine the column number of the column where the laser point is located according to the horizontal angle.

[0055] In a possible implementation of the second aspect, the wiring harness number determination module is specifically configured to:

[0056] Determine the beam number of laser point P according to the following formula:

[0057] row P =(α+γ) / 2,

[0058] Among them, row P represents the laser beam number of the laser point P, α represents the vertical angle value of the laser point P, and γ is a preset angle value;

[0059] The column number determination module is specifically used for:

[0060] The column number of the column where the laser point P is located is determined according to the following formula:

[0061] col P =β / b,

[0062] Among them, col P represents the column number of the column where the laser point P is located, β represents the horizontal angle value of the laser point P, and b is a preset threshold.

[0063] In a possible implementation manner of the second aspect, the arrangement information includes a laser beam number and a column number, and the set construction unit includes:

[0064] A ground object point marking and removal module, configured to mark ground object points based on the arrangement information, and remove laser points marked as ground object points from the key frame laser point cloud, wherein the ground object points are laser points irradiated on the ground object surface;

[0065] The candidate set construction module is used to perform neighborhood distance detection on the target laser point and construct a candidate roadside point set. The target laser point is the laser point remaining in the key frame laser point cloud after the ground object point is eliminated.

[0066] In a possible implementation of the second aspect, the ground feature point marking and removal module includes:

[0067] A first marking submodule is configured to mark the laser points in the key frame laser point cloud whose laser beam numbers are greater than a preset beam number as ground feature points; and / or,

[0068] a horizontal distance calculation submodule, configured to calculate the horizontal distance between a laser point having a laser beam number less than or equal to a preset beam number and an adjacent laser point in the key frame laser point cloud, wherein the adjacent laser point refers to a laser point having a laser beam number adjacent to the laser point but having the same column number as the laser point;

[0069] A second marking submodule is configured to mark the laser point as the ground feature point if the horizontal distance is less than a preset horizontal distance threshold;

[0070] The elimination submodule is used to eliminate the laser points marked as the ground object points from the key frame laser point cloud.

[0071] In a possible implementation manner of the second aspect, the arrangement information includes a laser beam number and a column number, and the candidate set construction module includes:

[0072] A neighborhood point set acquisition submodule is used to acquire a neighborhood point set of the target laser point, wherein the neighborhood point set includes the target laser point and a preset number of neighborhood points thereof;

[0073] A projection submodule, configured to perform a bird's-eye projection on the key frame laser point cloud, and obtain a two-dimensional plane distance between the target laser point in the neighborhood point set and the field point based on the bird's-eye projection;

[0074] A difference calculation submodule is used to sort the two-dimensional plane distances between the target laser point and the field point in the neighborhood point set by size, and calculate the difference between the maximum two-dimensional plane distance and the minimum two-dimensional plane distance in the neighborhood point set;

[0075] The candidate set construction submodule is used to determine the target laser point as a candidate roadside point if the difference is greater than a preset difference threshold, and to construct a candidate roadside point set.

[0076] In a possible implementation of the second aspect, the curb recognition unit includes:

[0077] performing clustering processing on the candidate roadside points in the candidate roadside point set;

[0078] Determining the shape of the curb according to the result of the clustering process;

[0079] A fitting method corresponding to the curb shape is selected for fitting, and the curb boundary is identified based on the fitting result.

[0080] In a possible implementation of the second aspect, the curb recognition device further includes:

[0081] A three-dimensional coordinate acquisition unit, configured to acquire the three-dimensional coordinates of a candidate roadside point and the three-dimensional coordinates of q neighboring points on the laser beam where the candidate roadside point is located;

[0082] an eigenvalue decomposition unit, configured to calculate a covariance matrix and perform eigenvalue decomposition based on the three-dimensional coordinates of the candidate roadside point and the q neighborhood points;

[0083] a main direction vector determining unit, configured to determine an eigenvector corresponding to a maximum eigenvalue among the decomposed eigenvalues ​​as the main direction vector of the candidate roadside point;

[0084] A candidate roadside point filtering unit is used to compare the direction difference between the main direction of the candidate roadside point and the target roadside direction based on the main direction vector; and remove the candidate roadside points whose comparison results are greater than a preset direction difference threshold from the candidate roadside point set, wherein the target roadside direction is the roadside direction corresponding to the adjacent laser beam of the candidate roadside point.

[0085] In a third aspect, an embodiment of the present application provides an intelligent device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the curb recognition method as described in the first aspect above is implemented.

[0086] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the curb recognition method described in the first aspect above.

[0087] In a fifth aspect, an embodiment of the present application provides a computer program product, which, when running on a smart device, enables the smart device to execute the curb recognition method as described in the first aspect above.

[0088] In an embodiment of the present application, a keyframe laser point cloud within a preset area of ​​the robot's environment is obtained, the keyframe laser point cloud is arranged in an orderly manner, and arrangement information of the laser points in the keyframe laser point cloud is obtained. Then, based on the arrangement information, neighborhood distance detection is performed on the laser points. Based on the results of the neighborhood distance detection, a set of candidate roadside points is constructed. The candidate roadside points in the set of candidate roadside points are then clustered, and roadside boundaries are automatically identified based on the clustering results. This application solution enables the robot to automatically identify roadside edges, thereby effectively constraining the machine's driving area to ensure the robot's safe movement. BRIEF DESCRIPTION OF THE DRAWINGS

[0089] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the embodiments or descriptions of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0090] Figure 1 This is a flowchart of the implementation of the curb recognition method provided in the embodiment of the present application;

[0091] Figure 2 This is a flowchart for the specific implementation of step S102 in the curb recognition method provided in an embodiment of the present application;

[0092] Figure 3 This is a flowchart for the specific implementation of step S103 in the curb recognition method provided in an embodiment of the present application;

[0093] Figure 4 This is a specific implementation flowchart of marking and removing ground feature points in the curb recognition method provided in an embodiment of the present application;

[0094] Figure 5 This is a specific implementation flowchart of constructing a candidate roadside point set in the roadside recognition method provided in an embodiment of the present application;

[0095] Figure 6 This is a specific implementation flowchart of step S104 in the curb recognition method provided in an embodiment of the present application;

[0096] Figure 7 This is a specific implementation flowchart of filtering candidate roadside points in a set of candidate roadside points in the roadside identification method provided in an embodiment of the present application;

[0097] Figure 8 This is a structural block diagram of a curb recognition device provided in an embodiment of the present application;

[0098] Figure 9 Schematic diagram of a smart device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0099] In the following description, specific details such as specific system structures and techniques are provided for purposes of illustration rather than limitation to facilitate a thorough understanding of the embodiments of the present application. However, it will be apparent to those skilled in the art that the present application may be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid obscuring the description of the present application with unnecessary detail.

[0100] It should be understood that when used in the present specification and the appended claims, the term "comprising" indicates the presence of described features, integers, steps, operations, elements and / or components, but does not preclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or collections thereof.

[0101] It will also be understood that the term "and / or" used in this specification and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.

[0102] As used in this specification and the appended claims, the term "if" can be interpreted as "when" or "upon" or "in response to determining" or "in response to detecting," depending on the context. Similarly, the phrase "if it is determined" or "if [described condition or event] is detected" can be interpreted as meaning "upon determination" or "in response to determining" or "upon detection of [described condition or event]" or "in response to detecting [described condition or event]," depending on the context.

[0103] In addition, in the description of the present application specification and the appended claims, the terms "first", "second", "third", etc. are only used to distinguish the descriptions and cannot be understood as indicating or implying relative importance.

[0104] References to "one embodiment" or "some embodiments" in this specification mean that a particular feature, structure, or characteristic described in conjunction with that embodiment is included in one or more embodiments of the present application. Thus, phrases such as "in one embodiment," "in some embodiments," "in other embodiments," and "in other embodiments" appearing in various places in this specification do not necessarily refer to the same embodiment, but rather mean "one or more but not all embodiments," unless otherwise specifically emphasized. The terms "including," "comprising," "having," and variations thereof all mean "including but not limited to," unless otherwise specifically emphasized.

[0105] The curb recognition method provided in the embodiments of the present application can be applied to smart devices equipped with multi-line laser radars, such as robots equipped with multi-line laser radars. The multi-line laser radars are installed in parallel on the smart devices, and a three-dimensional coordinate system is established with the origin of the multi-line laser radar as the origin of the coordinate system. The multi-line laser radar can simultaneously emit multiple laser beams at different tilt angles and scan 360 degrees around. Each laser line corresponds to a vertical angle, and different laser lines correspond to different vertical angles. The signals reflected by the lasers are received and analyzed to obtain point cloud data of the environment.

[0106] Figure 1 The implementation process of the curb recognition method provided by the embodiment of the present application is shown, and the method process includes steps S101 to S104. The specific implementation principles of each step are as follows:

[0107] Step S101: Acquire a key frame laser point cloud within a preset area of ​​the robot's environment.

[0108] In this embodiment, the key frame laser point cloud within the preset area of ​​the robot's environment is obtained in real time.

[0109] Generally, there are a lot of point cloud data scanned by multi-line laser radar. In this embodiment, key frame laser point cloud is selected from the point cloud data scanned by multi-line laser radar. The method of obtaining key frame laser point cloud can refer to the existing technology and will not be repeated here.

[0110] In some embodiments, the preset area of ​​the robot's environment can be within the robot's environment, with the robot as the starting point and within a set distance in the robot's forward direction; in other embodiments, the preset area of ​​the robot's environment can be within a circular area with the robot as the center and a specified distance as the radius in the robot's environment.

[0111] Step S102: Arrange the key frame laser point cloud in order to obtain arrangement information of laser points in the key frame laser point cloud.

[0112] Since different laser beams have different geometric shapes on the curb and the road surface, in order to effectively utilize the difference to extract the curb point, in an embodiment of the present application, the key frame laser point cloud is arranged in order to obtain the arrangement information of the laser points in the key frame laser point cloud.

[0113] As a possible implementation of the present application, the arrangement information includes the laser beam number and the column number, such as Figure 2 As shown, the step of arranging the key frame laser point cloud in order to obtain arrangement information of laser points in the key frame laser point cloud includes:

[0114] A1: Arrange the key frame laser point cloud in order, and obtain the vertical angle and horizontal angle of the laser point in the key frame laser point cloud.

[0115] In an embodiment of the present application, a laser scanning radar is mounted in parallel on the robot, and the origin of the laser radar is used as the origin of the coordinate system, the X-axis points to the direction of the front of the robot, the Y-axis points to the left direction of the robot, and the Z-axis points to the direction of the upper side of the robot to establish a three-dimensional coordinate system. Usually, the angle between the laser line and the plane formed by the X-axis and the Y-axis in the three-dimensional coordinate system is used as the vertical angle. The angle above the plane is positive, and the angle below the plane is negative, so the vertical angle corresponding to the laser line hitting the ground is negative. Usually, the angle between the laser line and the plane formed by the X-axis and the Z-axis in the three-dimensional coordinate system is used as the horizontal angle. The zero-degree direction of the horizontal angle is the positive direction of the X-axis, and the horizontal angle gradually increases in the clockwise direction.

[0116] A2: Determine the laser beam number of the laser point according to the vertical angle.

[0117] In one possible implementation, the beam number of the laser point P is determined according to the following formula (1):

[0118] row P=(α+γ) / 2 (1)

[0119] Among them, row P represents the laser beam number of the laser point P, α represents the vertical angle value of the laser point P, and γ is a preset angle value. In this embodiment, γ can be specifically 15.

[0120] A3: Determine the column number of the column where the laser point is located according to the horizontal angle.

[0121] In a possible implementation, the column number of the column where the laser point P is located is determined according to the following formula (2):

[0122] col P =β / b (2)

[0123] Among them, col P represents the column number of the column where the laser point P is located, β represents the horizontal angle value of the laser point P, and b is a preset threshold. In this embodiment, b can be specifically 0.2.

[0124] For example, taking the Velodyne VLP-16 laser radar as an example, the calculation method of the beam number row and column number of the laser point p is as follows: row P =(α+15) / 2,col P =β / 0.2; where α and β are the vertical angle and horizontal angle of the laser point respectively.

[0125] Step S103: performing neighborhood distance detection on the laser points based on the arrangement information, and constructing a set of candidate roadside points according to the result of the neighborhood distance detection.

[0126] In this embodiment, range detection refers to detecting the distance between a laser point and its neighboring points and the multi-line lidar. The laser points in the candidate roadside point set are all labeled as candidate roadside points. Candidate roadside points within the same candidate roadside point set are laser points on the same laser beam. Different laser beams correspond to different candidate roadside point sets.

[0127] As a possible implementation of this application, Figure 3 As shown, the above-mentioned step of performing neighborhood distance detection on the laser points based on the arrangement information and constructing a set of candidate roadside points according to the result of the neighborhood distance detection includes:

[0128] B1: Based on the arrangement information, mark the ground object points, and remove the laser points marked as ground object points from the key frame laser point cloud, wherein the ground object points are laser points irradiated on the ground object surface.

[0129] The arrangement information includes the laser beam number and the column number. As a possible implementation of the present application, Figure 4 The specific implementation process of marking ground object points based on the arrangement information and removing the laser points marked as ground object points from the key frame laser point cloud in the embodiment of the present application is shown as follows:

[0130] B11: Mark the laser points in the key frame laser point cloud whose laser beam number is greater than the preset beam number as ground object points. For example, for a 16-line laser radar, if the laser beam number of a laser point is greater than 9, then mark the laser point as a ground object point.

[0131] and / or,

[0132] B12: Calculate the horizontal distance between a laser point whose laser beam number is less than or equal to a preset beam number and its adjacent laser point in the key frame laser point cloud, where the adjacent laser point refers to a laser point whose laser beam number is adjacent to the laser point but has the same column number as the laser point.

[0133] The above horizontal distance is a three-dimensional horizontal distance, which can be calculated based on the three-dimensional coordinates of the laser point and the three-dimensional coordinates of its adjacent laser points. It should be noted that in the embodiment of the present application, only the case of one adjacent laser point is considered. Specifically, the above adjacent laser point refers to a laser point whose laser beam number is 1 greater than the laser beam number of the laser point and whose column number is the same as that of the laser point.

[0134] B13: If the horizontal distance is less than a preset horizontal distance threshold, the laser point is marked as the ground feature point.

[0135] For a laser point whose laser beam number is less than or equal to the preset beam number, whether to mark the laser point as a ground feature point is determined by comparing the three-dimensional horizontal distance between the laser point and its adjacent laser points with the preset horizontal distance threshold.

[0136] For example, with a 16-line laser radar, for the laser point P with beam number i, column number j, and i≤9 i,j , calculate point P i,j With P i+1,j When the 3D horizontal distance is less than the preset horizontal distance threshold, the laser point P i,j Marked as a feature point.

[0137] B14: Eliminate the laser point marked as the ground object point from the key frame laser point cloud.

[0138] In order to reduce the interference of useless laser points during the curb recognition process, in the embodiment of the present application, the ground object points in the key frame laser point cloud are marked and eliminated according to the sorting information, thereby improving the efficiency of curb recognition.

[0139] B2: Perform neighborhood distance detection on the target laser point to construct a set of candidate roadside points. The target laser point is the laser point remaining in the key frame laser point cloud after removing the ground feature point.

[0140] The arrangement information includes the laser beam number and the column number. As a possible implementation of the present application, Figure 5 The following is a specific implementation process of performing neighborhood distance detection on a target laser point and constructing a set of candidate roadside points in an embodiment of the present application:

[0141] B21: Acquire a neighborhood point set of the target laser point, where the neighborhood point set includes the target laser point and a preset number of neighborhood points.

[0142] Specifically, with the target laser point as the starting point, k neighborhood points are extracted to the left and right of the target laser point to form a neighborhood point set, and the neighborhood point set includes 2k+1 laser points.

[0143] B22: Performing a bird's-eye projection on the key frame laser point cloud, and obtaining a two-dimensional plane distance between the target laser point in the neighborhood point set and the domain point based on the bird's-eye projection.

[0144] Due to the changes in the shape of the laser beam on the surfaces of different shaped objects, the two-dimensional horizontal distance from a bird's-eye view has a corresponding change pattern. The point cloud of the laser beam on flat ground has a uniform laser point distribution, mainly showing a relatively uniform arc shape, while the point cloud of the laser beam on the curb is mainly a continuous line. To facilitate calculation, in this embodiment, the three-dimensional point cloud collected by the lidar is projected onto the XOY plane from a bird's-eye view. Based on the bird's-eye view projection, the two-dimensional plane distance between the target laser point and the area point in the neighborhood point set is obtained.

[0145] B23: sorting the two-dimensional plane distances between the target laser point and the field point in the neighborhood point set by size, and calculating the difference between the maximum two-dimensional plane distance and the minimum two-dimensional plane distance in the neighborhood point set.

[0146] B24: If the difference is greater than a preset difference threshold, the target laser point is determined as a candidate roadside point, and a candidate roadside point set is constructed.

[0147] In this embodiment, a neighborhood point set is constructed for each target laser point. According to the above steps B22-B23, the target laser points whose difference is less than or equal to the preset difference threshold are discarded, while the target laser points whose difference is greater than the preset difference threshold are retained and determined as candidate roadside points, thereby constructing a candidate roadside point set.

[0148] For example, the target laser point P i,j(i≤9) is the starting point, and k domain points are extracted to its left and right sides respectively to form a neighborhood point set S of 2k+1 points. i,j {P i,j-k ,…,P i,j …, P i,j+k}; Set the neighborhood point set S i,j The two-dimensional plane distance R of each laser point 2D(i，j) Sort them and calculate the difference between the maximum two-dimensional plane distance and the minimum two-dimensional plane distance. If the difference is greater than the preset difference threshold, the target laser point P is i,j Marked as a candidate roadside point. In this embodiment, the two-dimensional plane distance R 2D The calculation method is as follows (3):

[0149]

[0150] Step S104: clustering the candidate roadside points in the candidate roadside point set, and identifying roadside boundaries based on the clustering results.

[0151] In this embodiment, clustering is performed on the candidate roadside points in the candidate roadside point set of each laser beam, and the roadside boundary is identified based on the clustering processing results. The clustering processing algorithm can refer to the existing technology.

[0152] As a possible implementation of this application, Figure 6 As shown, the above step of clustering the candidate roadside points in the candidate roadside point set and identifying the roadside boundary according to the clustering result includes:

[0153] C1: performing clustering processing on the candidate roadside points in the candidate roadside point set.

[0154] In this embodiment, clustering processing can use Euclidean clustering to cluster the candidate curb points on each laser beam. The main purpose is to further eliminate noise points outside the curb area. At the same time, the same-direction curb points extracted from each laser beam are also merged to obtain clustered point clouds of the curbs on both sides.

[0155] C2: Determine the shape of the curb based on the result of the clustering process.

[0156] C3: Select a fitting method corresponding to the curb shape for fitting, and identify the curb boundary based on the fitting result.

[0157] In this embodiment, the curb shape is first determined based on the clustering results. Due to the sparsity of multi-line LiDAR point clouds, the extracted curb boundary may be incomplete and fragmented. Therefore, it is necessary to use the extracted curb points to complete the incomplete parts through fitting. Different curb shapes use different fitting methods to improve the accuracy of the fitting results.

[0158] For example, for a curb with a purely straight shape, two-dimensional straight line fitting can be performed directly; for a curb with a curved portion, multi-straight line end approximation is performed using a segmented fitting method.

[0159] Finally, the roadside boundary is identified based on the fitting results, so that the robot's movement range can be constrained according to the identified roadside boundary to prevent it from leaving the safe driving area.

[0160] As a possible implementation of this application, Figure 7 As shown, in the roadside recognition method provided in the embodiment of the present application, before the step of clustering the candidate roadside points in the candidate roadside point set, the method includes:

[0161] D1: Obtain the three-dimensional coordinates of the candidate roadside point and the three-dimensional coordinates of q neighboring points on the laser beam where the candidate roadside point is located.

[0162] D2: Calculate the covariance matrix and perform eigenvalue decomposition based on the three-dimensional coordinates of the candidate roadside point and the q neighboring points. The specific process of covariance matrix calculation and eigenvalue decomposition can be referred to the existing technology and will not be repeated here.

[0163] D3: Determine the eigenvector corresponding to the maximum eigenvalue among the decomposed eigenvalues ​​as the main direction vector of the candidate roadside point.

[0164] D4: Based on the main direction vector, compare the main direction of the candidate roadside point with the target roadside direction;

[0165] D5: Eliminate the candidate roadside points whose comparison results are greater than a preset direction difference threshold from the candidate roadside point set, wherein the target roadside direction is the roadside direction corresponding to the adjacent laser beams of the candidate roadside points.

[0166] Candidate roadside points marked on adjacent laser beams all correspond to the same roadside direction and are essentially consistent with the robot's current direction of movement. Therefore, in this embodiment, the principal direction vector of the candidate roadside points on the laser beam is first calculated. The covariance matrix of the candidate roadside point and its K adjacent neighboring points on the left and right of the laser beam is calculated, and eigenvalue decomposition is performed. The eigenvector corresponding to the largest eigenvalue is taken as the principal direction vector of the candidate roadside point. This is then compared with the roadside direction of the adjacent laser beam. If the principal direction of the candidate roadside point differs significantly from the roadside direction on the adjacent laser beam, it is removed from the candidate roadside point set; otherwise, it is retained, further improving the efficiency and accuracy of roadside recognition.

[0167] As can be seen from the above, in the embodiment of the present application, by obtaining the key frame laser point cloud in the preset area of ​​the robot's environment, the key frame laser point cloud is arranged in order to obtain the arrangement information of the laser points in the key frame laser point cloud, and then the neighborhood distance detection is performed on the laser points based on the arrangement information, and according to the result of the neighborhood distance detection, a set of candidate roadside points is constructed, and then the candidate roadside points in the candidate roadside point set are clustered, and the roadside boundary is automatically identified according to the result of the clustering processing, which enables the robot to automatically identify the roadside, thereby effectively constraining the driving area of ​​the machine to ensure the safe movement of the robot.

[0168] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

[0169] Corresponding to the curb recognition method described in the above embodiment, Figure 8 A structural block diagram of a curb recognition device provided in an embodiment of the present application is shown. For ease of explanation, only the parts related to the embodiment of the present application are shown.

[0170] Reference Figure 8 The roadside recognition device includes: a point cloud acquisition unit 81, an information acquisition unit 82, a set construction unit 83, and a roadside recognition unit 84, wherein:

[0171] The point cloud acquisition unit 81 is used to acquire the key frame laser point cloud in the preset area of ​​the robot's environment;

[0172] An information acquisition unit 82 is used to arrange the key frame laser point cloud in order to obtain sorting information of the laser points in the key frame laser point cloud;

[0173] A set construction unit 83 is configured to perform neighborhood distance detection on the laser points based on the sorting information, and to construct a candidate roadside point set according to the result of the neighborhood distance detection;

[0174] The roadside identification unit 84 is used to cluster the candidate roadside points in the candidate roadside point set and identify the roadside boundary according to the clustering result.

[0175] As a possible implementation of the present application, the arrangement information includes the laser beam number and the column number, and the information acquisition unit 82 includes:

[0176] An angle acquisition module, used to arrange the key frame laser point cloud in order and obtain the vertical angle and horizontal angle of the laser point in the key frame laser point cloud;

[0177] a beam number determination module, configured to determine the laser beam number of the laser point according to the vertical angle;

[0178] The column number determination module is used to determine the column number of the column where the laser point is located according to the horizontal angle.

[0179] As a possible implementation of the present application, the wiring harness number determination module is specifically configured to:

[0180] Determine the beam number of laser point P according to the following formula:

[0181] row P =(α+γ) / 2,

[0182] Among them, row P represents the laser beam number of the laser point P, α represents the vertical angle value of the laser point P, and γ is a preset angle value;

[0183] The column number determination module is specifically used for:

[0184] The column number of the column where the laser point P is located is determined according to the following formula:

[0185] col P =β / b,

[0186] Among them, col P represents the column number of the column where the laser point P is located, β represents the horizontal angle value of the laser point P, and b is a preset threshold.

[0187] As a possible implementation of the present application, the arrangement information includes the laser beam number and the column number, and the set construction unit 83 includes:

[0188] A ground object point marking and removal module, configured to mark ground object points based on the arrangement information, and remove laser points marked as ground object points from the key frame laser point cloud, wherein the ground object points are laser points irradiated on the ground object surface;

[0189] The candidate set construction module is used to perform neighborhood distance detection on the target laser point and construct a candidate roadside point set. The target laser point is the laser point remaining in the key frame laser point cloud after the ground object point is eliminated.

[0190] As a possible implementation of the present application, the above-mentioned ground feature point marking and removal module includes:

[0191] A first marking submodule is configured to mark the laser points in the key frame laser point cloud whose laser beam numbers are greater than a preset beam number as ground feature points; and / or,

[0192] a horizontal distance calculation submodule, configured to calculate the horizontal distance between a laser point having a laser beam number less than or equal to a preset beam number and an adjacent laser point in the key frame laser point cloud, wherein the adjacent laser point refers to a laser point having a laser beam number adjacent to the laser point but having the same column number as the laser point;

[0193] A second marking submodule is configured to mark the laser point as the ground feature point if the horizontal distance is less than a preset horizontal distance threshold;

[0194] The elimination submodule is used to eliminate the laser points marked as the ground object points from the key frame laser point cloud.

[0195] As a possible implementation of the present application, the arrangement information includes the laser beam number and the column number, and the candidate set construction module includes:

[0196] A neighborhood point set acquisition submodule is used to acquire a neighborhood point set of the target laser point, wherein the neighborhood point set includes the target laser point and a preset number of neighborhood points thereof;

[0197] A projection submodule, configured to perform a bird's-eye projection on the key frame laser point cloud, and obtain a two-dimensional plane distance between the target laser point in the neighborhood point set and the field point based on the bird's-eye projection;

[0198] A difference calculation submodule is used to sort the two-dimensional plane distances between the target laser point and the field point in the neighborhood point set by size, and calculate the difference between the maximum two-dimensional plane distance and the minimum two-dimensional plane distance in the neighborhood point set;

[0199] The candidate set construction submodule is used to determine the target laser point as a candidate roadside point if the difference is greater than a preset difference threshold, and to construct a candidate roadside point set.

[0200] As a possible implementation of the present application, the curb recognition unit 84 includes:

[0201] performing clustering processing on the candidate roadside points in the candidate roadside point set;

[0202] Determining the shape of the curb according to the result of the clustering process;

[0203] A fitting method corresponding to the curb shape is selected for fitting, and the curb boundary is identified based on the fitting result.

[0204] As a possible implementation of the present application, the curb recognition device further includes:

[0205] A three-dimensional coordinate acquisition unit, configured to acquire the three-dimensional coordinates of a candidate roadside point and the three-dimensional coordinates of q neighboring points on the laser beam where the candidate roadside point is located;

[0206] an eigenvalue decomposition unit, configured to calculate a covariance matrix and perform eigenvalue decomposition based on the three-dimensional coordinates of the candidate roadside point and the q neighborhood points;

[0207] a main direction vector determining unit, configured to determine an eigenvector corresponding to a maximum eigenvalue among the decomposed eigenvalues ​​as the main direction vector of the candidate roadside point;

[0208] A candidate roadside point filtering unit is used to compare the direction difference between the main direction of the candidate roadside point and the target roadside direction based on the main direction vector; and remove the candidate roadside points whose comparison results are greater than a preset direction difference threshold from the candidate roadside point set, wherein the target roadside direction is the roadside direction corresponding to the adjacent laser beam of the candidate roadside point.

[0209] As can be seen from the above, in the embodiment of the present application, by obtaining the key frame laser point cloud in the preset area of ​​the robot's environment, the key frame laser point cloud is arranged in order to obtain the arrangement information of the laser points in the key frame laser point cloud, and then the neighborhood distance detection is performed on the laser points based on the arrangement information, and according to the result of the neighborhood distance detection, a set of candidate roadside points is constructed, and then the candidate roadside points in the candidate roadside point set are clustered, and the roadside boundary is automatically identified according to the result of the clustering processing, which enables the robot to automatically identify the roadside, thereby effectively constraining the driving area of ​​the machine to ensure the safe movement of the robot.

[0210] It should be noted that the information interaction, execution process, etc. between the above-mentioned devices / units are based on the same concept as the method embodiment of this application. Their specific functions and technical effects can be found in the method embodiment section and will not be repeated here.

[0211] The embodiment of the present application further provides a computer-readable storage medium storing a computer program, wherein the computer program is executed by a processor to implement the following Figures 1 to 7 The steps of any curb recognition method are shown.

[0212] The embodiment of the present application also provides an intelligent device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, Figures 1 to 7 The steps of any curb recognition method are shown.

[0213] The embodiment of the present application also provides a computer program product, which, when executed on a smart device, enables the smart device to execute the following Figures 1 to 7 The steps of any curb recognition method are shown.

[0214] Figure 9 Schematic diagram of a smart device provided by an embodiment of the present application. Figure 9 As shown, the smart device 9 of this embodiment includes: a processor 90, a memory 91, and a computer program 92 stored in the memory 91 and executable on the processor 90. When the processor 90 executes the computer program 92, the steps in the above-mentioned embodiments of the curb recognition method are implemented, for example Figure 1 Alternatively, when the processor 90 executes the computer program 92, the functions of the modules / units in the above-mentioned device embodiments are realized, for example, Figure 8 The functions of the units 81 to 83 are shown.

[0215] For example, the computer program 92 may be divided into one or more modules / units, which are stored in the memory 91 and executed by the processor 90 to implement the present application. The one or more modules / units may be a series of computer-readable instruction segments capable of performing specific functions, and the instruction segments are used to describe the execution process of the computer program 92 in the smart device 9.

[0216] The smart device 9 may be a robot. The smart device 9 may include, but is not limited to, a processor 90 and a memory 91. Those skilled in the art will understand that Figure 9 It is only an example of the smart device 9 and does not constitute a limitation of the smart device 9. It may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the smart device 9 may also include input and output devices, network access devices, buses, etc.

[0217] The processor 90 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0218] The memory 91 can be an internal storage unit of the smart device 9, such as a hard disk or memory of the smart device 9. The memory 91 can also be an external storage device of the smart device 9, such as a plug-in hard disk equipped on the smart device 9, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. Furthermore, the memory 91 can also include both the internal storage unit of the smart device 9 and an external storage device. The memory 91 is used to store the computer program and other programs and data required by the smart device. The memory 91 can also be used to temporarily store data that has been output or is about to be output.

[0219] It should be noted that the information interaction, execution process, etc. between the above-mentioned devices / units are based on the same concept as the method embodiment of this application. Their specific functions and technical effects can be found in the method embodiment section and will not be repeated here.

[0220] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0221] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, which can be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and when the computer program is executed by the processor, it can implement the steps of the above-mentioned various method embodiments. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may at least include: any entity or device that can carry the computer program code to the device / terminal device, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electric carrier signal, a telecommunication signal and a software distribution medium. For example, a USB flash drive, a mobile hard disk, a magnetic disk or an optical disk. In some jurisdictions, according to legislation and patent practice, computer-readable media cannot be electric carrier signals and telecommunication signals.

[0222] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant description of other embodiments.

[0223] The above-described embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present application, and should all be included in the scope of protection of the present application.

Claims

1. A curb recognition method, characterized in that: include: Obtain keyframe laser point cloud in the preset area of ​​the robot's environment; Arrange the key frame laser point cloud in order to obtain arrangement information of laser points in the key frame laser point cloud, wherein the arrangement information includes laser beam number and column number; Based on the arrangement information, a neighborhood distance detection is performed on the laser point, and according to the result of the neighborhood distance detection, a set of candidate roadside points is constructed, including: marking the laser point with a laser beam number greater than a preset beam number in the key frame laser point cloud as a ground object point, where the ground object point is a laser point irradiated on the ground object surface; and / or calculating the horizontal distance between the laser point with a laser beam number less than or equal to the preset beam number and its adjacent laser point in the key frame laser point cloud, where the adjacent laser point refers to a laser point with a laser beam number adjacent to the laser point but the same column number as the laser point; if the horizontal distance is less than a preset horizontal distance threshold, marking the laser point as the ground object point; removing the laser point marked as the ground object point from the key frame laser point cloud; performing neighborhood distance detection on the target laser point to construct a set of candidate roadside points, where the target laser point is the laser point remaining in the key frame laser point cloud after removing the ground object point; Clustering is performed on the candidate roadside points in the candidate roadside point set, and roadside boundaries are identified based on the clustering results.

2. The method according to claim 1, characterized in that The step of arranging the key frame laser point cloud in order to obtain arrangement information of laser points in the key frame laser point cloud includes: Arrange the key frame laser point cloud in order, and obtain the vertical angle and horizontal angle of the laser point in the key frame laser point cloud; Determine the laser beam number of the laser point according to the vertical angle; The column number of the column where the laser point is located is determined according to the horizontal angle.

3. The method according to claim 1, characterized in that The step of performing neighborhood distance detection on the target laser point and constructing a set of candidate roadside points includes: Acquire a neighborhood point set of the target laser point, wherein the neighborhood point set includes the target laser point and a preset number of neighborhood points thereof; Performing a bird's-eye projection on the key frame laser point cloud, and obtaining a two-dimensional plane distance between the target laser point in the neighborhood point set and the field point based on the bird's-eye projection; Sort the two-dimensional plane distances between the target laser point and the field point in the neighborhood point set by size, and calculate the difference between the maximum two-dimensional plane distance and the minimum two-dimensional plane distance in the neighborhood point set; If the difference is greater than a preset difference threshold, the target laser point is determined as a candidate roadside point, and a candidate roadside point set is constructed.

4. The method according to claim 1, wherein The step of clustering the candidate roadside points in the candidate roadside point set and identifying the roadside boundary according to the clustering result comprises: performing clustering processing on the candidate roadside points in the candidate roadside point set; Determining the shape of the curb according to the result of the clustering process; A fitting method corresponding to the curb shape is selected for fitting, and the curb boundary is identified based on the fitting result.

5. The method according to any one of claims 1 to 4, characterized in that Before the step of clustering the candidate roadside points in the candidate roadside point set, the method includes: Obtaining the three-dimensional coordinates of a candidate roadside point and the three-dimensional coordinates of q neighboring points on the laser beam where the candidate roadside point is located; Calculate the covariance matrix and perform eigenvalue decomposition based on the three-dimensional coordinates of the candidate roadside point and the q neighborhood points; Determine the eigenvector corresponding to the maximum eigenvalue among the decomposed eigenvalues ​​as the main direction vector of the candidate roadside point; Based on the main direction vector, comparing the main direction of the candidate roadside point with the target roadside direction; The candidate roadside points whose comparison results are greater than a preset direction difference threshold are removed from the candidate roadside point set, wherein the target roadside direction is the roadside direction corresponding to the adjacent laser beams of the candidate roadside points.

6. A curb recognition device, characterized in that: include: A point cloud acquisition unit, used to acquire key frame laser point clouds within a preset area of ​​the robot's environment; An information acquisition unit, configured to arrange the key frame laser point cloud in order to obtain sorting information of laser points in the key frame laser point cloud, wherein the sorting information includes a laser beam number and a column number; a set construction unit, configured to perform neighborhood distance detection on the laser points based on the sorting information, and construct a set of candidate roadside points according to the result of the neighborhood distance detection; The set construction unit includes: a first marking submodule, configured to mark a laser point in the keyframe laser point cloud having a laser beam number greater than a preset beam number as a ground object point, wherein the ground object point is a laser point irradiated on a ground object surface; and / or a horizontal distance calculation submodule, configured to calculate a horizontal distance between a laser point in the keyframe laser point cloud having a laser beam number less than or equal to the preset beam number and an adjacent laser point, wherein the adjacent laser point is a laser point having a laser beam number adjacent to the laser point but having the same column number as the laser point; A second marking submodule is configured to mark the laser point as the ground feature point if the horizontal distance is less than a preset horizontal distance threshold; A removal submodule, configured to remove the laser points marked as the ground object points from the key frame laser point cloud; A candidate set construction module is used to perform neighborhood distance detection on a target laser point and construct a candidate roadside point set, wherein the target laser point is the laser point remaining in the key frame laser point cloud after removing the ground object point; The roadside recognition unit is used to cluster the candidate roadside points in the candidate roadside point set and identify the roadside boundary according to the clustering result.

7. An intelligent device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the curb recognition method according to any one of claims 1 to 5 is implemented.

8. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the curb recognition method according to any one of claims 1 to 5 is implemented.

Images