A two-dimensional laser point cloud pedestrian detection method for a mobile robot end and application
By combining 3D and 2D laser point cloud data, a 2D laser point cloud classifier is trained and localization compensation is performed, which solves the problems of inaccurate pedestrian detection and localization in mobile robots, and achieves efficient and low-cost pedestrian detection and localization.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- ROSIWIT TECHNOLOGY CO LTD
- Filing Date
- 2022-12-29
- Publication Date
- 2026-08-04
AI Technical Summary
Existing technologies struggle to effectively detect pedestrians under conditions of limited robot sensor data and computing power. In particular, 2D LiDAR data has few features and is easily affected by human leg models, leading to inaccurate detection and positioning.
This paper adopts a method combining 3D and 2D laser point clouds. Through classifier training and localization processing, a large amount of labeled data is obtained using 3D laser point clouds to train a 2D laser point cloud classifier. Combined with clustering analysis and binary classification network, the amount of data processing is reduced and the detection accuracy is improved. The impact of pedestrians on localization is reduced through localization compensation method.
It achieves low-cost and efficient pedestrian detection on mobile robots, improves detection accuracy and positioning robustness, reduces computing power requirements, and is suitable for practical applications in scenarios such as supermarkets and hotels.
Smart Images
Figure CN116206283B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of mobile robots, and in particular to a method and application for pedestrian detection using two-dimensional laser point clouds on a mobile robot. Background Technology
[0002] With the advancement of intelligent manufacturing technology and the gradual maturation of intelligent transformation of robots, intelligent products, represented by intelligent mobile robots, are gradually enriching the application fields of products. Among them, mobile robots, due to their small and cute appearance and rich and diverse functions, are gradually playing an increasingly important role in areas with high personnel flow, such as supermarkets and hotels. Given the limited sensor data and computing power of robots, the core of ensuring operational safety for robots to effectively detect pedestrians lies in lightweight and efficient pedestrian detection algorithms.
[0003] Pedestrian detection is a typical binary classification problem in the field of deep learning. It is often used in applications such as autonomous driving in large spaces and motion-sensing products in small spaces. It often uses convolutional neural network algorithms such as GNN for classification operations. Its data requirements are dense point clouds, and its computing power requirements are accelerated matrix operation units such as GPUs.
[0004] For mobile robots used in scenarios such as hotels and supermarkets, the commonly used sensor is 2D LiDAR. The computing power is multi-core CPU but no GPU, so it is not possible to use pedestrian detection algorithms in the field of intelligent driving. 2D LiDAR collects 2D laser point clouds in a space of more than ten meters. Since most mobile robots are low, the collection height is mostly at the position of the human lower leg. In the laser point cloud data, after clustering, certain distance and curvature features can be found in the normal human lower leg. Therefore, human features can be calculated by using the LiDAR features of the human lower leg.
[0005] Chinese patent document with publication number CN111444767B discloses a pedestrian detection and tracking method based on lidar. It directly identifies human legs from lidar point cloud data through a multi-scale adaptive random forest classifier, establishes a uniform human motion model, and thus can continuously track the human body and effectively identify it.
[0006] There are two main types of existing pedestrian detection technologies: the first is the method in the field of intelligent driving, which has a large amount of data and computation, and the detection results have a high true positive rate and a high true negative rate; the second is 2D LiDAR data, which is processed by random forest classification method, combined with uniform motion assumption and Kalman filtering to improve the accuracy of pedestrian detection.
[0007] The first approach is mainly applied to the field of intelligent vehicles. Intelligent driving vehicles operate at high speeds, so the requirements for safety are very high. It uses expensive 3D LiDAR to acquire dense point cloud data. The data volume is large and the data features are rich. The calculation is performed by an on-board chip with a built-in dedicated GPU for accelerating matrix operations. Therefore, it can build multi-layer neural networks for repeated calculations, resulting in accurate calculation results.
[0008] However, due to differences in application scenarios, application requirements, cost control, and the amount of labeled data, pedestrian detection solutions in the field of intelligent driving are not suitable for intelligent mobile robots.
[0009] The second approach is to directly process the laser point data using the random forest method. This approach has a simple logic and uses random forest voting to determine whether a person is a pedestrian. It requires little computing power. However, the laser data processing method is simple and does not take into account the complex situation of lasers hitting the lower legs of people, which requires massive amounts of data to support it. Specifically, the data of LiDAR varies greatly depending on whether the person is wearing suit pants, bell-bottoms, skirts, bare legs, or a signpost. Therefore, directly solving the problem using a normal human lower leg model will inevitably lead to false positives and false negatives in data label classification.
[0010] Furthermore, for mobile robots, robot localization largely depends on the matching degree of 2D laser data with environmental features. When the LiDAR is dynamically occluded by pedestrians, it will significantly affect the confidence of the point cloud matching at the current location, thus causing localization drift. Therefore, to solve this problem, we propose a 2D laser point cloud pedestrian detection method and its application for mobile robots. Summary of the Invention
[0011] This invention proposes a two-dimensional laser point cloud pedestrian detection method and application for mobile robots, which solves the problems of high algorithm processing cost, few features of 2D LiDAR, measurement results affected by human leg models, and the influence of pedestrian laser points on mobile robot positioning in the field of intelligent driving.
[0012] To achieve the above objectives, the present invention adopts the following technical solution:
[0013] A method and application for pedestrian detection using two-dimensional laser point cloud on a mobile robot includes classifier training and localization processing. The classifier training is used to label training data and test data, and also to calculate loss values. The classifier training is used to transmit labeled data to clustered point cloud data, and to process and calculate the labeled data using the clustered point cloud data. The localization processing is used to detect the number of pedestrians, process the localization point cloud, and compensate for localization loss.
[0014] The classifier training includes 3D laser point cloud and 2D laser point cloud. The 3D laser point cloud is mounted on a mobile robot testing machine. The 3D laser point cloud is used to record data multiple times in environments such as shopping malls and hotels, so that the 3D laser point cloud can acquire a large amount of labeled training data and test data.
[0015] The 3D laser point cloud has a pedestrian recognition network, which is used to detect each line of laser point cloud above the ground in layers, and the layered data is vectorized and calculated based on human characteristics.
[0016]
[0017] Where H represents the final output of the classifier, and the sign function represents the output of each layer. This represents the weight of the laser layer detection for layer t, where there are a total of T layers;
[0018] The network was trained on datasets such as KITTI. The 2D laser classifier is a neural network using a cross-entropy loss function, which is the difference between the predictions made by the forward error propagation network and the 3D laser data.
[0019]
[0020] Where L represents the value of cross-entropy. This represents the original input, and y is the predicted value of the data, labeled by the 3D LiDAR neural network.
[0021] Preferably, the clustered point cloud data includes clustering processing and a binary classification network. The clustering processing is used to process the point cloud information from 2D laser point cloud scanning, and the binary classification network is used to calculate the clustered point cloud information.
[0022] The main steps of classifier-based point cloud processing methods are as follows:
[0023] Step 1: Scan the point cloud information of different types of pants using 2D laser point cloud scanning;
[0024] Step 2: Process the point cloud information using clustering methods such as K-means or DBSCAN to obtain the average intensity value, centroid pose, and cluster radius data of the trousers type;
[0025] Step 3: Then, the clustered and filtered data is input into the 2D laser point cloud data of the mobile robot. Only data that meet the requirements of mean, center of gravity pose and cluster radius are input into the binary classification network and classified by the binary classification neural network algorithm.
[0026]
[0027] In the formula, This represents the average intensity value of the point cloud, where n represents the total number of cluster points. This represents the laser point cloud intensity value at point i.
[0028]
[0029] In the formula, This represents the centroid pose of the point cloud, where n represents the total number of cluster points. This represents the laser point cloud pose of point i.
[0030]
[0031] In the formula, The value represents the cluster radius of the point cloud, and n represents the total number of cluster points. Let denote the second normal form of the laser point cloud pose from point i to the centroid of the point cloud, and denote the distance between the two points.
[0032] Preferably, the positioning process includes pedestrian detection and positioning compensation, wherein the number of pedestrians detected in the pedestrian detection is the number of pedestrians calculated by a neural network classifier;
[0033] The design is based on the following positioning logic: if there are fewer than 5 pedestrians, the point cloud corresponding to the pedestrians is filtered out; if there are more than 5 but less than 10 pedestrians, the robot slows down and filters out the point cloud corresponding to the pedestrians; if there are more than 10 pedestrians, the robot moves at the lowest speed or stops and re-spreads AMCL positioning particles.
[0034] The location compensation process involves processing the location point cloud based on the detected pedestrian pose, calculating the sequence number of the 2D laser point, and setting the corresponding laser point to empty. The compensation for location loss is mainly at the robot location level. Common and operable location compensation methods include enhancing odometry confidence and increasing the AMCL particle dispersal radius.
[0035] The beneficial effects of this invention are as follows:
[0036] 1. This invention proposes a method and application for pedestrian detection using two-dimensional laser point clouds on a mobile robot. By simultaneously introducing 3D and 2D laser data and automatically labeling the 3D detection data to a 2D binary classifier, this solution better balances the cost and computing power requirements of the mobile robot algorithm compared to the high data volume and high computing power requirements in the field of intelligent driving.
[0037] 2. This invention reduces the processing load of neural network algorithms by clustering common trouser features, thereby reducing the computational burden on mobile robots. Compared with the random forest detection algorithm, it is more in line with actual application scenarios and improves pedestrian detection accuracy. The binary classification algorithm only processes possible data, further reducing the amount of data processing.
[0038] 3. This invention reduces the impact of pedestrians on the positioning of mobile robots by using laser point filtering and positioning compensation methods, which improves the positioning robustness of the robot compared with ordinary random forest detection methods.
[0039] In summary, this invention better balances the cost and computing power requirements of mobile robot-side algorithms, and improves radar characteristics by combining 2D and 3D data, making the measurement results unaffected by human leg models, while also reducing the impact of pedestrians on the positioning of mobile robots. Attached Figure Description
[0040] Figure 1 This is a schematic diagram of the framework structure of the present invention.
[0041] Figure 2 This is a schematic diagram of the laser scanning structure of the present invention. Detailed Implementation
[0042] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.
[0043] Reference Figure 1 and Figure 2 As shown, a method and application for pedestrian detection using two-dimensional laser point cloud on a mobile robot includes classifier training. The classifier training is used to label training data and test data, and it is also used to calculate loss values. The classifier training is used to transmit the labeled data to clustered point cloud data, and to process and calculate the labeled data through the clustered point cloud data. The localization processing is used to detect the number of pedestrians, process the localization point cloud, and compensate for localization loss.
[0044] Classifier training includes both 3D and 2D laser point clouds. The 3D laser point cloud is mounted on a mobile robot testing machine and used to record data multiple times in environments such as shopping malls and hotels. This allows the 3D laser point cloud to acquire the necessary large amounts of labeled training and testing data. The main reason for needing 3D laser data annotation is as follows:
[0045] 1. There are few 2D LiDAR datasets, and the LiDAR models in the datasets are very different, making them difficult to use directly for training;
[0046] 2. Manual annotation of 2D LiDAR is very costly and consumes a lot of time and energy.
[0047] The 3D laser point cloud has a pedestrian recognition network, which is used to detect each layer of the laser point cloud above the ground, and the layered data is vectorized and calculated based on human characteristics.
[0048]
[0049] Where H represents the final output of the classifier, and the sign function represents the output of each layer. This represents the weight of the laser layer detection for layer t, where there are a total of T layers;
[0050] This network can be trained on datasets such as KITTI. The 2D laser classifier is a neural network using a cross-entropy loss function, which is the difference between the predictions made by the forward error propagation network and the 3D laser data.
[0051]
[0052] Where L represents the value of cross-entropy. The input represents the original data, and y represents the predicted value of the data, labeled by a 3D LiDAR neural network. By equipping both 2D and 3D LiDAR on the experimental model, the pedestrian detection results from the 3D LiDAR data are used for pedestrian labeling, while the 2D LiDAR data serves as the original input data. With labeling as the objective, the 3D LiDAR can collect data on the head, neck, left and right shoulders of the human body through classification, resulting in rich pedestrian detection features. High-precision labeling results can be obtained using a binary classifier from a commonly used binary classification network. This classifier serves as the ground truth of the loss function and is used to train the cross-entropy classification model of the 2D LiDAR. This automatic labeling algorithm significantly saves on algorithm training and labeling costs.
[0053] Clustered point cloud data includes clustering processing and binary classification networks. Clustering processing is used to process point cloud information from 2D laser point cloud scanning, and binary classification networks are used to calculate the clustered point cloud information.
[0054] The main steps of classifier-based point cloud processing methods are as follows:
[0055] Step 1: Scan the point cloud information of different types of pants using 2D laser point cloud scanning;
[0056] Step 2: Process the point cloud information using clustering methods such as K-means or DBSCAN to obtain the average intensity value, centroid pose, and cluster radius data of the trousers type;
[0057] Step 3: Then, the clustered and filtered data is input into the 2D laser point cloud data of the mobile robot. Only data that meet the requirements of mean, center of gravity pose and cluster radius are input into the binary classification network and classified by the binary classification neural network algorithm.
[0058]
[0059] In the formula, This represents the average intensity value of the point cloud, where n represents the total number of cluster points. This represents the laser point cloud intensity value at point i.
[0060]
[0061] In the formula, This represents the centroid pose of the point cloud, where n represents the total number of cluster points. This represents the laser point cloud pose of point i.
[0062]
[0063] In the formula, The value represents the cluster radius of the point cloud, and n represents the total number of cluster points. Let represent the second normal form of the laser point cloud pose from point i to the centroid of the point cloud. Let denot the distance between the two points. First, cluster the laser point cloud data, and then process the clustered point cloud center and point cloud radius to ensure that all trouser features can be sent to the backend for propagation. Then, data that meets the features are fed into the cross-entropy classifier for data regression classification. This method only processes point cloud data that meets the above preliminary features, which greatly reduces the amount of data processing. Moreover, the classification method is a simple cross-entropy binary classification, which has low computational requirements, thus meeting the real-time pedestrian solving needs of practical intelligent mobile robots.
[0064] The localization process includes pedestrian detection and localization compensation. The number of pedestrians detected in pedestrian detection is mainly the number of pedestrians calculated by a neural network classifier.
[0065] The design is based on the following positioning logic: if there are fewer than 5 pedestrians, the point cloud corresponding to the pedestrians is filtered out; if there are more than 5 but less than 10 pedestrians, the robot slows down and filters out the point cloud corresponding to the pedestrians; if there are more than 10 pedestrians, the robot moves at the lowest speed or stops and re-spreads AMCL positioning particles.
[0066] In the localization compensation process, the localization point cloud is processed based on the detected pedestrian pose to calculate the sequence number of 2D laser points and set the corresponding laser points to empty. The compensation for localization loss is mainly at the robot localization level. Common and operable localization compensation methods include enhancing odometry confidence and increasing the AMCL particle spreading radius. This paper proposes to filter out the laser point cloud data after pedestrian identification at the localization level and improve the matching search frequency and step size of laser matching algorithms such as AMCL, thereby compensating for the loss of data filtering.
[0067] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.
Claims
1. A method for pedestrian detection from a two-dimensional laser point cloud of a mobile robot, comprising a classifier training and a localization process, characterized in that, The classifier training is used to label training and test data, and also to calculate loss values. The classifier training transmits the labeled data to clustered point cloud data, which includes clustering processing and a binary classification network. The clustering processing is used to process point cloud information obtained from 2D laser point cloud scanning, and the binary classification network is used to perform classification calculations on the clustered point cloud information. The localization processing is used to detect the number of pedestrians, process the localization point cloud, and compensate for localization loss. The classifier training includes 3D laser point cloud and 2D laser point cloud. The 3D laser point cloud is mounted on a mobile robot testing machine. The 3D laser point cloud is used to record data multiple times in shopping mall and hotel environments, so that the 3D laser point cloud can acquire a large amount of labeled training data and test data. The 3D laser point cloud has a pedestrian recognition network, which is used to detect each line of laser point cloud above the ground in layers, and the layered data is vectorized and calculated based on human characteristics. Wherein, H represents the final output of the classifier, the sign function represents the output of each layer, Indicates the weight of the laser layer detection of the t layer, a total of T layers The network was trained on the KITTI dataset. The 2D laser classifier is a neural network using a cross-entropy loss function, which is the difference between the predictions made by the forward error propagation network and the 3D laser data. wherein L represents a value of cross-entropy, represents the original input, y is the predicted value of the data, which is labeled by the 3D laser radar neural network. 2.The method of claim 1, wherein, The classifier method for processing point clouds mainly includes the following steps: Step 1: Scan the point cloud information of different types of pants using 2D laser point cloud scanning; Step 2: Process the point cloud information using K-means or DBSCAN clustering methods to obtain the average intensity value, centroid pose, and cluster radius data of the trousers type of cluster; Step 3: Then, the clustered and filtered data is input into the 2D laser point cloud data of the mobile robot. Only data that meet the requirements of mean, center of gravity pose and cluster radius are input into the binary classification network and classified by the binary classification neural network algorithm. wherein, represents the point cloud average intensity value, n represents the total number of clustered points, represents the laser point cloud intensity value of the i-th point; In the formula, This represents the centroid pose of the point cloud, where n represents the total number of cluster points. This represents the laser point cloud pose of the i-th point; wherein, denotes the point cloud clustering radius, n denotes the total number of clustered points, denotes the Euclidean distance between two points.
3. The method of claim 1, wherein the method further comprises: The localization process includes pedestrian detection and localization compensation, wherein the number of pedestrians detected in the pedestrian detection is the number of pedestrians calculated by a neural network classifier.
4. The method of claim 3, wherein the method further comprises: The design is based on the following location processing logic: if there are fewer than 5 pedestrians, the point cloud corresponding to each pedestrian will be filtered out. If there are more than 5 but less than 10 pedestrians, the robot slows down and filters out the point cloud corresponding to the pedestrians; if there are more than 10 pedestrians, the robot moves at the lowest speed or stops and re-spreads AMCL localization particles.
5. The method of claim 3, wherein the method further comprises: The location compensation process involves processing the location point cloud based on the detected pedestrian pose, calculating the sequence number of the 2D laser point, and setting the corresponding laser point to empty. The compensation for location loss is mainly at the robot location level. The operable location compensation methods are to enhance the odometry confidence and increase the AMCL particle dispersal radius.