Method, apparatus, and media for real-time detection of people and vehicle flow based on lidar

Abstract

The application discloses a method, device and medium for detecting real-time people and vehicle flow based on a laser radar, comprising: acquiring a three-dimensional point cloud of a current frame, clustering the three-dimensional point cloud, and obtaining multiple preliminary detection results. In people flow detection, multiple head point sets are obtained through searching of upper and lower body ranges, and one head point set is taken as one segmentation result. In vehicle flow detection, further division is performed on relatively rough preliminary detection results in a merging manner to obtain multiple segmentation results. Then, based on intersection between the segmentation results of the current frame and the segmentation results of the last frame, one current frame segmentation result currently processed is matched with all last frame segmentation results, and all current frame segmentation results are repeatedly operated in this way, so that people flow and vehicle flow counts of the current frame are obtained.

Description

Technical Field

[0001] The present invention relates to the field of laser radar technology, and in particular to a method, device and medium for real-time detection of pedestrian and vehicle traffic based on laser radar. Background Art

[0002] According to statistics from the Ministry of Public Security, the national motor vehicle population reached 395 million in 2021, including 302 million cars. In 2021, 36.74 million new motor vehicles were registered nationwide, an increase of 3.46 million vehicles (a 10.38% increase) compared to 2020, and an increase of 4.6 million vehicles (a 14.31% increase) compared to 2019. While the annual increase in the number of cars meets people's travel needs, it also increases pressure on roadways, leading to traffic congestion and accidents. Especially in medium and large cities, traffic congestion during rush hour and holidays has become a daily problem, severely impacting people's daily work and lives. Therefore, obtaining accurate and real-time traffic flow data is extremely important.

[0003] At the same time, large crowd gatherings, due to their density, unpredictability, and unevenness, often lead to public safety incidents such as stampedes and fires. To effectively prevent such incidents, it is necessary to accurately capture the real-time flow of people in the venue for management and decision-making. Summary of the Invention

[0004] Based on this, it is necessary to provide a method, equipment and medium for real-time detection of pedestrian and vehicle flow based on lidar to solve the problem of difficulty in accurately obtaining pedestrian and vehicle flow.

[0005] A method for real-time detection of human traffic based on laser radar, the method comprising:

[0006] Obtaining a two-dimensional point cloud of the current frame, adding a time dimension to the two-dimensional point cloud, and obtaining a three-dimensional point cloud; wherein the two-dimensional point cloud includes a width dimension and a height dimension;

[0007] Clustering the three-dimensional point cloud to obtain a plurality of preliminary detection results; wherein the preliminary detection result is a set of point clouds that are preliminarily determined to belong to the same class;

[0008] Searching within the upper body range of the target detection result to obtain multiple head point sets searched, and searching within the lower body range of the target detection result, and according to the search result, classifying the point cloud within the lower body range of the target detection result into the multiple head point sets to update the multiple head point sets searched; wherein the target detection result is any one of the multiple preliminary detection results, and the head point set indicates a point cloud set corresponding to a single human body with the head as a judgment reference;

[0009] Taking a head point set of the target detection result as a current frame segmentation result, taking all the searched current frame segmentation results as the current frame segmentation result, and obtaining the previous frame segmentation result including multiple previous frame segmentation results;

[0010] Based on the intersection between the segmentation result of the current frame and the segmentation result of the previous frame, a currently processed current frame segmentation result is matched with all the segmentation results of the previous frame. If the number of successful matches of the currently processed current frame segmentation result is greater than the preset successful match number threshold, the traffic count is increased by 1;

[0011] The next current frame segmentation result is used as the current frame segmentation result currently being processed, and the step of performing the intersection between the segmentation result of the current frame and the segmentation result of the previous frame is returned to match the current frame segmentation result currently being processed with all the previous frame segmentation results and subsequent steps until all current frame segmentation results are traversed.

[0012] A method for real-time detection of vehicle flow based on laser radar, the method comprising:

[0013] Obtaining a two-dimensional point cloud of the current frame, adding a time dimension to the two-dimensional point cloud, and obtaining a three-dimensional point cloud; wherein the two-dimensional point cloud includes a width dimension and a height dimension;

[0014] Clustering the three-dimensional point cloud to obtain a plurality of preliminary detection results; wherein the preliminary detection result is a set of point clouds that are preliminarily determined to belong to the same class;

[0015] Calculate the distance in the width dimension and the distance in the time dimension of all preliminary detection results, merge the preliminary detection results according to the distance in the width dimension and the distance in the time dimension, and determine the retained detection results; use one retained detection result as a current frame segmentation result, use all searched current frame segmentation results as the segmentation results of the current frame, and obtain the segmentation result of the previous frame containing multiple previous frame segmentation results;

[0016] Based on the intersection between the segmentation result of the current frame and the segmentation result of the previous frame, a currently processed current frame segmentation result is matched with all the segmentation results of the previous frame. If the number of successful matches of the currently processed current frame segmentation result is greater than the preset successful match number threshold, the traffic flow count is increased by 1;

[0017] The next current frame segmentation result is used as the current frame segmentation result currently being processed, and the step of performing the intersection between the segmentation result of the current frame and the segmentation result of the previous frame is returned to match the current frame segmentation result currently being processed with all the previous frame segmentation results and subsequent steps until all current frame segmentation results are traversed.

[0018] A computer-readable storage medium stores a computer program. When the computer program is executed by a processor, the processor executes the steps of the above-mentioned method for real-time detection of pedestrian flow based on laser radar and the method for real-time detection of vehicle flow based on laser radar.

[0019] A device for real-time detection of pedestrian and vehicle flow based on laser radar includes a memory and a processor. The memory stores a computer program. When the computer program is executed by the processor, the processor executes the steps of the above-mentioned method for real-time detection of pedestrian flow based on laser radar and the method for real-time detection of vehicle flow based on laser radar.

[0020] The present invention provides a method, device, and medium for real-time detection of pedestrian and vehicle flow based on laser radar. When detecting pedestrian flow, the initial detection results obtained by clustering are searched within the upper body to obtain multiple head point sets. The lower body is then searched, and the point clouds within the lower body are incorporated into the multiple head point sets. Each head point set corresponds to a current frame segmentation result, i.e., a possible single person. This further segmentation of the relatively coarse initial detection results is performed, resulting in multiple point cloud sets corresponding to single persons. Then, based on the intersection between the current frame segmentation results and the previous frame segmentation results, the currently processed current frame segmentation results are matched with all previous frame segmentation results. This process is repeated for all current frame segmentation results, resulting in a statistically calculated pedestrian flow count for the current frame. When detecting vehicle flow, the relatively coarse initial detection results are further segmented by merging them based on the calculated distance in the width dimension and the distance in the time dimension, resulting in multiple current frame segmentation results. Each current frame segmentation result corresponds to a possible vehicle. Then, based on the intersection between the segmentation results of the current frame and the segmentation results of the previous frame, the vehicle flow count of the current frame is obtained by matching the current frame segmentation result with all the previous frame segmentation results and repeating this operation for all the current frame segmentation results. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0022] in:

[0023] Figure 11 is a flow chart of a method for real-time detection of pedestrian flow based on laser radar in one embodiment;

[0024] Figure 2 Schematic diagram of a single-line laser radar scanning target;

[0025] Figure 3 Schematic diagram of the obtained two-dimensional point cloud;

[0026] Figure 4 A schematic diagram of adding a time dimension to a two-dimensional foreground point cloud in one embodiment;

[0027] Figure 5 FIG1 is a schematic diagram of a process of searching within the upper body in one embodiment;

[0028] Figure 6 Schematic diagram of a process for searching within the lower body in one embodiment;

[0029] Figure 7 Schematic diagram of a flow chart of cyclic matching in one embodiment;

[0030] Figure 8 1 is a flow chart of a method for real-time detection of vehicle flow based on laser radar in one embodiment;

[0031] Figure 9 This is a structural block diagram of a device for real-time detection of pedestrian and vehicle flow based on laser radar in one embodiment. DETAILED DESCRIPTION

[0032] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0033] The executor of the present invention is a device for real-time detection of pedestrian and vehicle traffic based on laser radar, specifically a single-line laser radar. Since a single-line laser radar is used, the present invention has the advantage of a wide detection range compared to the single-point laser radar solution. Secondly, since the scanning frequency of the single-line laser radar used is above 30HZ, it can also detect vehicles with a speed of more than 70km / h, so the present invention has the advantage of being suitable for high-speed vehicle detection. At the same time, compared to the existing detection scheme that requires the deployment of two or more single-line laser radars, or the deployment of a single multi-line laser radar, the present invention only requires the deployment of one single-line laser radar, so it also has the advantage of lower cost.

[0034] The following is a detailed description of the real-time detection method of human flow based on LiDAR suitable for human flow detection:

[0035] like Figure 1 As shown, Figure 1 The flowchart of a method for real-time detection of pedestrian flow based on a laser radar in one embodiment is shown below. The steps provided in this embodiment include:

[0036] Step 102: Obtain a two-dimensional point cloud of the current frame, add a time dimension to the two-dimensional point cloud, and obtain a three-dimensional point cloud.

[0037] Specifically, such as Figure 2 As shown in , a single-line laser radar emits a patrol laser beam at a set scanning frequency to scan the target within the field of view and obtain a two-dimensional point cloud of the current frame. Figure 3 As shown in the figure, the 2D point cloud of the current frame is shown. Since a single-line LiDAR is used, it can be seen that the 2D point cloud only includes the width dimension (x dimension) and the height dimension (h dimension).

[0038] In a specific embodiment, the two-dimensional point cloud is further pre-processed, including: performing foreground separation and ground registration on the acquired two-dimensional point cloud to obtain a registered two-dimensional foreground point cloud.

[0039] This is because, firstly, the acquired 2D point cloud contains not only the target object (people), but also data about buildings and road surfaces that we are not interested in. Therefore, we extract the target object data and remove the uninteresting data to reduce interference. Here, the target object data is called the foreground, and the non-target object data is called the background.

[0040] Secondly, because the initially acquired 2D point cloud data is constructed with the single-line LiDAR location as the origin, rotating and translating the point cloud data (ground registration) allows the coordinate system to be aligned with one axis parallel to and one axis perpendicular to the ground line. This facilitates subsequent feature extraction of the target object.

[0041] Then, if Figure 4 As shown, in a specific embodiment, a time dimension (t dimension) is added to the two-dimensional foreground point cloud, that is, multiple frames of two-dimensional foreground point clouds scanned at the same position are merged to obtain a three-dimensional foreground point cloud.

[0042] Step 104 : clustering the three-dimensional point cloud to obtain a plurality of preliminary detection results.

[0043] In one specific embodiment, a clustering algorithm is used to cluster only the 3D foreground point clouds, generating multiple preliminary detection results. This reduces background interference in crowd flow detection. The preliminary detection results are collections of point clouds that are preliminarily determined to belong to the same class. These preliminary detection results provide only a rough representation of crowd flow and require further refinement in subsequent steps.

[0044] For example, the DBSCAN clustering algorithm may be used for clustering, and the process specifically includes:

[0045] (1) All three-dimensional foreground point clouds are regarded as objects to be processed p and are marked as unprocessed.

[0046] (2) Calculate the Euclidean distance r between the target object pi (the target object pi is any one of the multiple objects to be processed p) and all other objects to be processed p, expressed as

[0047]

[0048] In the formula and Represents two objects, namely the target object pi and any object p to be processed except the target object pi; and Represents the coordinates of two objects in the i-th dimension; m is the set coordinate dimension.

[0049] Then, among these Euclidean distances r, determine the first number of Euclidean distances r that are smaller than the preset radius Eps. , if the first quantity If the first quantity is less than the minimum preset number Minpts, proceed to step (3); if the first quantity is greater than or equal to the minimum preset number Minpts, proceed to step (4).

[0050] (3) Determine whether the target object pi is a noise point or a boundary point and mark it as processed.

[0051] (4) With the target object pi as the core point, a target cluster Ci with a preset radius Eps is established, the target object pi and the blank objects in the target cluster are classified into the target cluster Ci, and all the objects to be processed in the target cluster are marked as processed.

[0052] The blank object is an object that is not determined to be a noise point or a boundary point and is not classified into any cluster.

[0053] (5) Check whether all pending objects p are marked as processed. If not, return to step (2) until all pending objects are marked as processed.

[0054] After the above cluster analysis, multiple clusters can be formed, that is, multiple preliminary detection results can be obtained.

[0055] Step 106: Search within the upper body range of the target detection result to obtain multiple head point sets searched, and search within the lower body range of the target detection result. According to the search results, the point cloud within the lower body range of the target detection result is classified into multiple head point sets to update the multiple head point sets searched.

[0056] Among them, the target detection result is any one of multiple preliminary detection results, and the head point set indicates a point cloud set corresponding to a single human body based on the head.

[0057] In crowded situations, people are often very close together. Initial detection results show their point cloud data contiguous, making it difficult for common clustering algorithms to accurately separate them, thus affecting counting accuracy. However, in crowded situations, heads often have significant distances between them. Therefore, this step first filters out the head point cloud data from the target detection results and then determines the number of people by segmenting the point cloud. This allows for a more precise point cloud collection based on a single person to be segmented from the crude target detection results.

[0058] In a specific embodiment, Figure 5 As shown, the process of searching within the upper body range includes:

[0059] Step 106a: Obtain a first point cloud above a first preset height, a preset upper body search height range, and a first search height step in the target detection result.

[0060] Among them, the first point cloud is named EH_PTS, the minimum value of the upper body search height range is the upper body minimum height H_MIN_1, the maximum value of the upper body search height range is the upper body maximum height H_MAX_1, and the first search height step is H_STEP_1.

[0061] Step 106b: Initialize the head point set to be empty.

[0062] Step 106c, taking the maximum height of the upper body as the starting point, based on the first search height step and the first current search step number, search for a point cloud in the first point cloud whose height value is within the upper body search height range and higher than the first current search height, and use the currently searched point cloud as the second point cloud.

[0063] The first current search height C_H_TH_1 is equal to the difference between the upper body maximum height H_MAX_1 and the first search distance d1. The first search distance d1 is equal to the product of the first search height step length H_STEP_1 and the first current search step number n1. The initial value of the first current search step number is 1. That is:

[0064] C_H_TH_1=H_MAX_1-n1*H_STEP_1

[0065] That is, in this step, each time a point cloud in EH_PTS is searched for a height value within the range [H_MIN_1, H_MAX_1] and higher than the first current search height C_H_TH_1, the currently searched point cloud is used as the second point cloud S_PTS_1.

[0066] Step 106d: During the search process, determine whether the currently searched second point cloud includes at least one head point set. If the currently searched second point cloud does not include a head point set, execute step 106e; if the currently searched second point cloud does include at least one head point set, execute step 106f.

[0067] Step 106e: All currently searched second point clouds are used as third point clouds.

[0068] Step 106f: clustering the second point cloud that does not belong to the head point set based on at least one head point set, and dividing the second point cloud that does not belong to the head point set into a third point cloud, a clustered point cloud, and a boundary point cloud based on the clustering result.

[0069] It can be understood that steps 106d to 106f are a process of dividing the currently searched second point cloud into a third point cloud, a cluster point cloud, and a boundary point cloud.

[0070] At the beginning of the search, the second point cloud does not include the head point set (the method for determining the head point set is described in detail in step 106g). At this time, step 106e is executed to treat all currently searched second point clouds as third point clouds and exclude them from the subsequent search range.

[0071] As the search progresses, a head point set begins to appear in the second point cloud, and step 106f is executed. Step 106f can also employ the DBSCAN clustering algorithm mentioned above, with the difference being that HEAD_PTS is marked as processed, while the second point cloud S_PTS_1 is treated as the object to be processed, and both are marked as unprocessed. Furthermore, the dimensions used in the calculation are the x and t dimensions.

[0072] Finally, the point cloud that is not clustered into any head point set is regarded as the third point cloud. The point cloud that is clustered into any head point set is regarded as the clustered point cloud and excluded from the subsequent search range. The point cloud that is clustered into at least two head point sets is regarded as the boundary point cloud and excluded from the subsequent search range.

[0073] Step 106g: If a third point cloud is currently searched, all third point clouds are clustered to obtain a suspected head point set, and the suspected head point set whose dimension number in the time dimension is greater than the preset dimension threshold and whose maximum width in the width dimension is greater than the preset width threshold is taken as the head point set.

[0074] It can be understood that this step continuously updates the head point set during the upper body search process.

[0075] Specifically, if a third point cloud S_PTS_1_M that has not been included is currently searched, the DBSCAN algorithm is used to re-cluster the point cloud in S_PTS_1_M to obtain a suspected head point set. The clustering algorithm here is not repeated. If the obtained suspected head point set meets the set requirements, it is used as the head point set. For example: if the number of dimensions of the suspected head point in the t dimension is greater than two, and the maximum width in the x dimension is greater than 0.4 meters, it is considered to be the head point set HEAD_PTS, and these point clouds are excluded from the subsequent search range.

[0076] Step 106h: Set the first current search step number = the first current search step number + 1, and return to step 106c and subsequent steps until the first current search height is less than the minimum height of the upper body.

[0077] This completes the search for the upper body, and allows the search for the lower body to proceed.

[0078] In a specific embodiment, Figure 6 As shown, the process of searching within the lower body includes:

[0079] Step 106A: Obtain a preset lower body search height range and a second search height step.

[0080] The lower body search height range has a minimum lower body height H_MIN_2, a maximum lower body height H_MAX_2, and a second search height step length H_STEP_2.

[0081] Step 106B, taking the maximum height of the lower body as the starting point, based on the second search height step and the second current search step number, searches for a point cloud in the first point cloud whose height value is within the lower body search height range and higher than the second current search height, and uses the currently searched point cloud as the fourth point cloud.

[0082] The second current search height C_H_TH_2 is equal to the difference between the lower body maximum height H_MAX_2 and the second search distance d2. The second search distance d2 is equal to the product of the second search height step length H_STEP_2 and the second current search step number n2. The initial value of the second current search step number is 1. That is:

[0083] C_H_TH_2=H_MAX_2-n2*H_STEP_2

[0084] That is, in this step, each time the point cloud in EH_PTS is searched for a point C_H_TH_2 whose height value is within the range [H_MIN_2, H_MAX_2] and higher than the second current search height, the currently searched point cloud is used as the fourth point cloud S_PTS_2.

[0085] Step 106C: clustering the fourth point cloud based on the at least one head point set, so as to classify the fourth point cloud that meets the clustering condition into the at least one head point set.

[0086] The biggest difference between the search in the upper body range and the search in the lower body range is that there is no need to redefine the head point set. Instead, the fourth point cloud currently searched can be divided into the determined head point set.

[0087] Among them, this step can adopt the DBSCAN clustering algorithm mentioned above, and the clustering condition here is that a certain point cloud is only clustered into any head point set.

[0088] Step 106D, set the second current search step number = the second current search step number + 1, and return to execute step 106B and subsequent steps until the second current search height is less than the minimum height of the lower body.

[0089] This completes the search of the lower body and obtains a complete set of head points.

[0090] In step 108 , a head point set of the target detection result is used as a current frame segmentation result, all the searched current frame segmentation results are used as the current frame segmentation result, and the previous frame segmentation result including multiple previous frame segmentation results is obtained.

[0091] That is, one head point set corresponds to one current frame segmentation result, and all searched current frame segmentation results are used as the current frame segmentation result. The segmentation result of the previous frame is obtained in the same way as steps 102-106, which will not be repeated here.

[0092] Step 110, based on the intersection between the segmentation result of the current frame and the segmentation result of the previous frame, a currently processed current frame segmentation result is matched with all the previous frame segmentation results. If the number of successful matches of the currently processed current frame segmentation result is greater than the preset successful match number threshold, the traffic count is increased by 1.

[0093] Step 112: The next current frame segmentation result is used as the current frame segmentation result to be processed, and the process returns to step 110 and subsequent steps until all current frame segmentation results are traversed.

[0094] It can be understood that steps 110 to 112 are a cyclic matching process based on the intersection between the segmentation results of two frames, and the flow of people in the current field of view is finally determined through this process.

[0095] In a specific embodiment, Figure 7 As shown, the process of loop matching includes:

[0096] Step 110a: Calculate the coverage area of ​​all current frame segmentation results in the coordinate system of the width dimension and the time dimension.

[0097] Step 110b, intersect the coverage area of ​​the target current frame segmentation result with the coverage area of ​​the target previous frame segmentation result, calculate the intersection area between the target current frame segmentation result and the target previous frame segmentation result, obtain and count the intersection areas between all current frame segmentation results and previous frame segmentation results, and obtain a statistically obtained overlap table.

[0098] Among them, the target current frame segmentation result is any one of all current frame segmentation results, the target previous frame segmentation result is any one of all previous frame segmentation results, and a row or column of the overlap table indicates the intersection area of ​​a current frame segmentation result and all previous frame segmentation results.

[0099] For example, assuming that the number of all current frame segmentation results is N and the number of all previous frame segmentation results is M, the overlap table has N rows and M columns (of course, it can also have M rows and N columns). The specific content is shown in Table 1 below:

[0100] Table 1:

[0101]

[0102] Step 110c: read the initial row or initial column in the overlap table, and determine the number of previous frame segmentation results in the initial row or initial column that meet the preset matching requirement.

[0103] Among them, the initial row and initial column are rows and columns with initial values ​​of 1, and the preset matching requirement is that the intersection area between the previous frame segmentation result and the initial current frame segmentation result corresponding to the initial row or initial column is greater than half of the coverage area of ​​the initial current frame segmentation result.

[0104] Taking the current frame result 1 and the previous frame result 1 as an example, if the intersection area 1_1 is greater than half of the coverage area of ​​the current frame result 1, it is considered to meet the preset matching requirement and the number is recorded as 1. Continue to compare the current frame result 1 and the previous frame result 2. If the intersection area 1_2 is greater than half of the coverage area of ​​the current frame result 1, it is considered to meet the preset matching requirement and the number is recorded as 2. Otherwise, the number is recorded as 1. And so on, the number statistics are completed.

[0105] Step 110d: Determine whether the number of previous frame segmentation results that meet the preset matching requirement is 1. If the number of previous frame segmentation results that meet the preset matching requirement is 1, execute step 110e; if the number of previous frame segmentation results that meet the preset matching requirement is not 1, execute step 110j and subsequent steps.

[0106] Step 110e: determine that the match is successful, and calculate the area difference between the intersection area of ​​the successfully matched previous frame segmentation result and the initial current frame segmentation result and the coverage area of ​​the initial current frame segmentation result.

[0107] That is, when the number of the previous frame segmentation results that meets the preset matching requirement is 1, it is determined that the two segmentation results are uniquely corresponding between the two frames, and the matching is deemed successful.

[0108] The following example uses the successful matching of the current frame result 1 and the previous frame result 1 as an example to illustrate. Therefore, in this step, it is necessary to further calculate the area difference between the intersection area 1_1 and the coverage area of ​​the current frame result 1. .

[0109] Step 110f: Determine whether the area difference is less than or equal to a preset area difference threshold. If the area difference is less than or equal to the preset area difference threshold, proceed to step 110g; if the area difference is greater than the preset area difference threshold, proceed to step 110j and subsequent steps.

[0110] In this embodiment, the area difference threshold is set to 0.01. Of course, other values ​​can also be set according to actual conditions, and no specific limitation is made here.

[0111] Therefore, when the area difference between the intersection area 1_1 and the coverage area of ​​the current frame result 1 , is less than or equal to 0.01, execute step 110g. Otherwise, execute step 110j and subsequent steps.

[0112] Step 110g: increment the number of successful matches of the previous frame segmentation result that was successfully matched by one, and assign the number of successful matches of the initial current frame segmentation result to that number.

[0113] It is understandable that, since the time dimension is added to the point cloud data, this step can achieve the inheritance of the number of successful matches, that is, the present invention only performs crowd statistics on point clouds that have been successfully matched continuously.

[0114] Assume that the number of successful matches of the previous frame result 1 before the match is successful is MATCH_NUM. Then in this step, let MATCH_NUM=MATCH_NUM+1, and assign it the number of successful matches of the initial current frame segmentation result.

[0115] Step 110h determines whether the number of successful matches for the initial current frame segmentation result is greater than a preset successful match threshold. If so, step 110i is executed, incrementing the traffic count by 1. If the number of successful matches for the initial current frame segmentation result is less than or equal to the preset successful match threshold, step 110j and subsequent steps are executed.

[0116] In this embodiment, the threshold value of the number of successful matches is set to 5. Of course, other values ​​can also be set according to actual conditions, and no specific limitation is made here.

[0117] In this step, if the number of successful matches of the initial current frame segmentation result is greater than the preset successful match threshold, the initial current frame segmentation result is deemed to have been successfully matched continuously, and the pedestrian flow count is incremented by 1. If the number of successful matches of the initial current frame segmentation result is less than or equal to the preset successful match threshold, the initial current frame segmentation result is deemed to have been unsuccessfully matched continuously, and step 110j and subsequent steps are executed.

[0118] In step 110j, set initial row = initial row + 1, or set initial column = initial column + 1, and return to step 110c and subsequent steps until all rows or all columns in the overlap table are read.

[0119] Corresponding to Table 1, the current frame result 2 is read continuously, and 110c and subsequent steps are repeatedly executed to determine whether the current frame result 2 can increase the human traffic count by 1. Similarly, the intersection between the segmentation results of the two frames is matched cyclically until all rows or columns in the overlap table are read, and the human traffic count within the current field of view can be determined.

[0120] Of course, it is understandable that after scanning and obtaining the latest two-dimensional point cloud, the two-dimensional point cloud of the current frame in this embodiment can be used as the two-dimensional point cloud of the previous frame, and the latest two-dimensional point cloud can be used as the two-dimensional point cloud of the current frame, and steps 102-112 can be repeated, so that the purpose of real-time detection of human flow can be achieved.

[0121] Considering that the direction of movement of people is also an important factor affecting decision-making in actual situations, the present invention further performs the following steps after counting the number of people:

[0122] 1. Perform principal component analysis on the current frame segmentation result being processed to obtain the principal component direction vector corresponding to the current frame segmentation result being processed.

[0123] In a specific embodiment, the step of obtaining the principal component direction vector includes:

[0124] (1) Obtain the height dimension data of all point clouds in the height dimension and the time dimension data in the time dimension in the current frame segmentation result currently being processed.

[0125] For example, assuming that there are only 3 points in the current frame segmentation result being processed, the coordinates of P1 are (h1, t1), the coordinates of P2 are (h2, t2), and the coordinates of P3 are (h3, t3); then the height dimension data TREND_DATA_H = (h1, h2, h3); the time dimension data TREND_DATA_T = (t1, t2, t3).

[0126] (2) Calculate the height mean data based on all height dimension data, and decentralize all height dimension data based on the height mean data to obtain decentralized height dimension data.

[0127] That is, calculate the height mean data, H_MEAN=avg(TREND_DATA_H).

[0128] And subtract H_MEAN from the data (h1,h2,h3) in TREND_DATA_H to obtain the decentralized height dimension data TREND_DATA_H_C=(nh1,nh2,nh3).

[0129] (3) Calculate the time mean data based on all time dimension data, and decentralize all time dimension data based on the time mean data to obtain decentralized time dimension data.

[0130] That is, calculate the time mean data, T_MEAN=avg(TREND_DATA_T).

[0131] And subtract T_MEAN from the data in TREND_DATA_T (t1, t2, t3) to obtain the decentralized time dimension data TREND_DATA_T_C=(nt1, nt2, nt3).

[0132] (4) Construct a covariance matrix based on the decentralized height dimension data and time dimension data, solve the eigenvector of the covariance matrix, and determine the principal component direction vector corresponding to the current frame segmentation result of the current processing based on the solved eigenvector.

[0133] Among them, the constructed covariance matrix is:

[0134]

[0135]

[0136]

[0137]

[0138]

[0139] The process of solving the eigenvector is:

[0140]

[0141]

[0142]

[0143] If Delta < 0, the eigenvector does not exist. If Delta > = 0, continue to calculate the eigenvector and get:

[0144]

[0145]

[0146] Next, determine the component direction vectors:

[0147]

[0148]

[0149] if , then As the main component direction vector output, otherwise Output as principal component direction vectors.

[0150] 2. Calculate the angle between the principal component direction vector and the set in / out direction vector, and determine the in / out direction of the current frame segmentation result according to the calculated angle.

[0151] Specifically, the set entry and exit direction can be the direction of a sidewalk, or the entry and exit direction of a shopping mall, etc., and is not specifically limited here. If the angle between the principal component direction vector and the entry direction is relatively smaller, it is determined to be an entry, and the entry count is incremented by 1. If the angle between the principal component direction vector and the exit direction is relatively smaller, it is determined to be an exit, and the exit count is incremented by 1.

[0152] After the above processing, the movement direction of people in the field of view can be well counted.

[0153] It can be seen that the above-mentioned method of real-time detection of pedestrian flow based on lidar can accurately obtain the real-time pedestrian flow in a venue for use in management and decision-making in the venue.

[0154] The following is a detailed description of a real-time traffic flow detection method based on LiDAR suitable for traffic flow detection:

[0155] like Figure 8 As shown, Figure 8 The flowchart of a method for real-time detection of vehicle flow based on a laser radar in one embodiment is shown below. The steps provided in this embodiment include:

[0156] Step 802: Obtain a two-dimensional point cloud of the current frame, add a time dimension to the two-dimensional point cloud, and obtain a three-dimensional point cloud.

[0157] Step 804: cluster the three-dimensional point cloud to obtain multiple preliminary detection results.

[0158] In a specific implementation scenario, steps 802-804 are basically the same as steps 102-104 in the method for real-time detection of pedestrian flow based on lidar provided by the present invention, and will not be repeated here.

[0159] Step 806 , calculating the distances in the width dimension and the distances in the time dimension of all preliminary detection results, merging the preliminary detection results according to the distances in the width dimension and the distances in the time dimension, and determining the retained detection results.

[0160] Compared with the pedestrian flow detection method which requires segmentation of the preliminary detection results, in vehicle flow detection, since point cloud dispersion is prone to occur, the preliminary detection results are merged to determine the retained detection results.

[0161] In a specific embodiment, the merging process includes:

[0162] If there are two or more preliminary test results that meet the first merging requirements, the preliminary test results that meet the first merging requirements will be merged and retained, and the preliminary test results that do not meet the first merging requirements will be retained.

[0163] Among them, the first merging requirement is that the distance between the two preliminary detection results in the width dimension is less than or equal to the preset first width threshold (for example, 0.6 meters, of course, it can also be set to other values ​​according to actual conditions), and the distance in the time dimension is less than or equal to the preset number of frames (for example, 5 frames, of course, it can also be set to other values ​​according to actual conditions).

[0164] In another specific embodiment, the merging process includes:

[0165] All preliminary detection results that are located in the same lane and whose maximum distance in the width dimension is less than a preset second width threshold (for example, 0.3 meters, which can of course be set to other values ​​based on actual conditions) are merged. If the merged detection results meet the second merging requirements, they are retained. If the merged detection results do not meet the second merging requirements, they are not merged and the preliminary detection results before the merger are retained.

[0166] The second merging requirement is that the maximum distance in the width dimension is less than a preset third width threshold, which indicates the maximum width of the vehicle and is set to 3 meters, for example. Of course, it can also be set to other values ​​according to actual conditions.

[0167] It is understandable that the two combined specific embodiments mentioned above can be implemented either one or both according to actual conditions.

[0168] In step 808 , a retained detection result is used as a current frame segmentation result, all the searched current frame segmentation results are used as the current frame segmentation results, and the previous frame segmentation result including multiple previous frame segmentation results is obtained.

[0169] That is, one retained detection result corresponds to one current frame segmentation result, and all retained current frame segmentation results are used as the segmentation result of the current frame. The segmentation result of the previous frame is obtained in the same manner as steps 802-806, which will not be repeated here.

[0170] Step 810, based on the intersection between the segmentation result of the current frame and the segmentation result of the previous frame, match the current frame segmentation result currently being processed with all the segmentation results of the previous frame. If the number of successful matches of the current frame segmentation result currently being processed is greater than the preset successful match number threshold, the traffic flow count is increased by 1.

[0171] Step 812: Use the next current frame segmentation result as the current frame segmentation result being processed, and return to execute step 810 and subsequent steps until all current frame segmentation results are traversed.

[0172] In a specific implementation scenario, steps 808-812 are basically the same as steps 108-112 in the method for real-time detection of pedestrian flow based on lidar provided by the present invention. The main difference is that the vehicle flow count is finally increased by 1. The rest of the content will not be repeated.

[0173] Of course, it is understandable that after scanning and obtaining the latest two-dimensional point cloud, the two-dimensional point cloud of the current frame in this embodiment can be used as the two-dimensional point cloud of the previous frame, and the latest two-dimensional point cloud can be used as the two-dimensional point cloud of the current frame, and steps 802-812 can be repeated, so as to achieve the purpose of real-time detection of traffic flow.

[0174] Furthermore, after the vehicle flow count is incremented by 1, the present invention also uses the maximum distance in the width dimension of the segmentation result of the current frame being processed as the width of the currently detected vehicle, and the maximum distance in the height dimension as the height of the currently detected vehicle. In this way, the height and width of the vehicle can be detected.

[0175] Figure 9 FIG1 shows an internal structure diagram of a device for real-time detection of pedestrian and vehicle flow based on a laser radar in one embodiment. Figure 9 As shown, the device for real-time detection of pedestrian and vehicle flow based on laser radar includes a processor, a memory and a network interface connected via a system bus. Among them, the memory includes a non-volatile storage medium and an internal memory. The non-volatile storage medium of the device for real-time detection of pedestrian and vehicle flow based on laser radar stores an operating system, and may also store a computer program. When the computer program is executed by the processor, the processor can implement a method for real-time detection of pedestrian flow based on laser radar and a method for real-time detection of vehicle flow based on laser radar. The internal memory may also store a computer program. When the computer program is executed by the processor, the processor can execute a method for real-time detection of pedestrian flow based on laser radar and a method for real-time detection of vehicle flow based on laser radar. Those skilled in the art will understand that Figure 9 The structure shown in the figure is only a block diagram of a part of the structure related to the scheme of the present application, and does not constitute a limitation on the device for real-time detection of pedestrian and vehicle flow based on lidar to which the scheme of the present application is applied. The specific device for real-time detection of pedestrian and vehicle flow based on lidar may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.

[0176] A device for real-time detection of pedestrian and vehicle traffic based on laser radar includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the following steps are implemented: obtaining a two-dimensional point cloud of a current frame, adding a time dimension to the two-dimensional point cloud, and obtaining a three-dimensional point cloud; clustering the three-dimensional point cloud to obtain multiple preliminary detection results; searching within the upper body range of the target detection result to obtain multiple searched head point sets, and searching within the lower body range of the target detection result to classify the point cloud within the lower body range of the target detection result into multiple head point sets based on the search results, thereby updating the multiple searched head point sets; A head point set of the target detection result is used as a current frame segmentation result, all the searched current frame segmentation results are used as the segmentation results of the current frame, and the segmentation result of the previous frame containing multiple previous frame segmentation results is obtained; based on the intersection between the segmentation result of the current frame and the segmentation result of the previous frame, a currently processed current frame segmentation result is matched with all the previous frame segmentation results; if the number of successful matches of the currently processed current frame segmentation result is greater than the preset successful match number threshold, the flow count is increased by 1; the next current frame segmentation result is used as the currently processed current frame segmentation result, and the step of matching a currently processed current frame segmentation result with all the previous frame segmentation results based on the intersection between the segmentation result of the current frame and the segmentation result of the previous frame and subsequent steps are returned to execute until all current frame segmentation results are traversed.

[0177] and performing the following steps: obtaining a two-dimensional point cloud of the current frame, adding a time dimension to the two-dimensional point cloud, and obtaining a three-dimensional point cloud; Clustering the three-dimensional point cloud to obtain multiple preliminary detection results; calculating the distance of all preliminary detection results in the width dimension and the distance in the time dimension, merging the preliminary detection results according to the distance in the width dimension and the distance in the time dimension, and determining the retained detection results; taking a retained detection result as a current frame segmentation result, taking all searched current frame segmentation results as the segmentation result of the current frame, and obtaining the segmentation result of the previous frame containing multiple previous frame segmentation results; matching a currently processed current frame segmentation result with all previous frame segmentation results based on the intersection between the current frame segmentation result and the previous frame segmentation result, if the number of successful matches of the currently processed current frame segmentation result is greater than a preset successful match number threshold, then adding 1 to the traffic flow count; taking the next current frame segmentation result as the currently processed current frame segmentation result, returning to execute the step of matching a currently processed current frame segmentation result with all previous frame segmentation results based on the intersection between the current frame segmentation result and the previous frame segmentation result and subsequent steps until all current frame segmentation results are traversed.

[0178] A computer-readable storage medium stores a computer program that, when executed by a processor, implements the following steps: obtaining a two-dimensional point cloud of a current frame, adding a time dimension to the two-dimensional point cloud, and obtaining a three-dimensional point cloud; clustering the three-dimensional point cloud to obtain multiple preliminary detection results; searching within the upper body range of a target detection result to obtain multiple searched head point sets, and searching within the lower body range of the target detection result to classify the point cloud within the lower body range of the target detection result into multiple head point sets based on the search results, thereby updating the multiple searched head point sets; A head point set of the target detection result is used as a current frame segmentation result, all the searched current frame segmentation results are used as the segmentation results of the current frame, and the segmentation result of the previous frame containing multiple previous frame segmentation results is obtained; based on the intersection between the segmentation result of the current frame and the segmentation result of the previous frame, a currently processed current frame segmentation result is matched with all the previous frame segmentation results; if the number of successful matches of the currently processed current frame segmentation result is greater than the preset successful match number threshold, the flow count is increased by 1; the next current frame segmentation result is used as the currently processed current frame segmentation result, and the step of matching a currently processed current frame segmentation result with all the previous frame segmentation results based on the intersection between the segmentation result of the current frame and the segmentation result of the previous frame and subsequent steps are returned to execute until all current frame segmentation results are traversed.

[0179] and performing the following steps: obtaining a two-dimensional point cloud of the current frame, adding a time dimension to the two-dimensional point cloud, and obtaining a three-dimensional point cloud; Clustering the three-dimensional point cloud to obtain multiple preliminary detection results; calculating the distance of all preliminary detection results in the width dimension and the distance in the time dimension, merging the preliminary detection results according to the distance in the width dimension and the distance in the time dimension, and determining the retained detection results; taking a retained detection result as a current frame segmentation result, taking all searched current frame segmentation results as the segmentation result of the current frame, and obtaining the segmentation result of the previous frame containing multiple previous frame segmentation results; matching a currently processed current frame segmentation result with all previous frame segmentation results based on the intersection between the current frame segmentation result and the previous frame segmentation result, if the number of successful matches of the currently processed current frame segmentation result is greater than a preset successful match number threshold, then adding 1 to the traffic flow count; taking the next current frame segmentation result as the currently processed current frame segmentation result, returning to execute the step of matching a currently processed current frame segmentation result with all previous frame segmentation results based on the intersection between the current frame segmentation result and the previous frame segmentation result and subsequent steps until all current frame segmentation results are traversed.

[0180] It should be noted that the above-mentioned method for real-time detection of pedestrian flow based on laser radar and the method, device, equipment and computer-readable storage medium for real-time detection of vehicle flow based on laser radar belong to a general inventive concept, and the contents of the embodiments of the method for real-time detection of pedestrian flow based on laser radar and the method, device, equipment and computer-readable storage medium for real-time detection of vehicle flow based on laser radar can be applied to each other.

[0181] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiment methods can be implemented by instructing the relevant hardware through a computer program, which can be stored in a non-volatile computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in many forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM (RDRAM).

[0182] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0183] The above embodiments merely illustrate several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art could make various modifications and improvements without departing from the spirit of the present invention, all of which fall within the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be determined by the appended claims.

Claims

1. A method for real-time detection of human flow based on laser radar, characterized in that: The method comprises: Obtaining a two-dimensional point cloud of the current frame, adding a time dimension to the two-dimensional point cloud, and obtaining a three-dimensional point cloud; wherein the two-dimensional point cloud includes a width dimension and a height dimension; Clustering the three-dimensional point cloud to obtain a plurality of preliminary detection results; wherein the preliminary detection result is a set of point clouds that are preliminarily determined to belong to the same class; Searching within the upper body range of the target detection result to obtain multiple head point sets searched, and searching within the lower body range of the target detection result, and according to the search result, classifying the point cloud within the lower body range of the target detection result into the multiple head point sets to update the multiple head point sets searched; wherein the target detection result is any one of the multiple preliminary detection results, and the head point set indicates a point cloud set corresponding to a single human body with the head as a judgment reference; Taking a head point set of the target detection result as a current frame segmentation result, taking all the searched current frame segmentation results as the current frame segmentation result, and obtaining the previous frame segmentation result including multiple previous frame segmentation results; Based on the intersection between the segmentation result of the current frame and the segmentation result of the previous frame, a currently processed current frame segmentation result is matched with all the segmentation results of the previous frame. If the number of successful matches of the currently processed current frame segmentation result is greater than the preset successful match number threshold, the traffic count is increased by 1; Taking the next current frame segmentation result as the current frame segmentation result currently being processed, returning to the step of performing the intersection between the current frame segmentation result and the previous frame segmentation result, matching the current frame segmentation result currently being processed with all previous frame segmentation results, and subsequent steps, until all current frame segmentation results are traversed; The step of searching within the upper body of the target detection result to obtain multiple head point sets includes: In the target detection result, a first point cloud above a first preset height, a preset upper body search height range, and a first search height step are obtained; wherein the minimum value of the upper body search height range is the upper body minimum height, and the maximum value of the upper body search height range is the upper body maximum height; Initialize the head point set to be empty; Starting from the maximum height of the upper body, based on the first search height step and the first current search step number, the first point cloud is searched for a point cloud whose height value is within the upper body search height range and higher than the first current search height, and the currently searched point cloud is used as the second point cloud; wherein the first current search height is equal to the difference between the maximum height of the upper body and the first search distance, the first search distance is equal to the product of the first search height step length and the first current search step number, and the initial value of the first current search step number is 1; During the search process, determining whether the second point cloud currently searched includes at least one head point set; If the currently searched second point cloud does not include the head point set, all currently searched second point clouds are used as the third point cloud; If the currently searched second point cloud includes at least one head point set, clustering the second point cloud that does not belong to the head point set based on the at least one head point set, and dividing the second point cloud that does not belong to the head point set into a third point cloud, a clustered point cloud, and a boundary point cloud based on the clustering result; wherein the third point cloud is not clustered into any head point set, the clustered point cloud is clustered into any head point set, and the boundary point cloud is clustered into at least two head point sets; If a third point cloud is currently searched, all third point clouds are clustered to obtain a suspected head point set, and the suspected head point set whose number of dimensions in the time dimension is greater than the preset dimension threshold and whose maximum width in the width dimension is greater than the preset width threshold is taken as the head point set; Let the first current search step number = the first current search step number + 1, return to execute the step and subsequent steps of searching for a point cloud in the first point cloud whose height value is within the upper body search height range and higher than the first current search height based on the first search height step number and the first current search step number, and using the currently searched point cloud as the second point cloud, until the first current search height is less than the minimum height of the upper body.

2. The method according to claim 1, characterized in that The searching is performed within the lower body range of the target detection result, and the point cloud within the lower body range of the target detection result is classified into the multiple head point sets according to the search result to update the searched multiple head point sets, including: Obtaining a preset lower body search height range and a second search height step; wherein the minimum value of the lower body search height range is the minimum height of the lower body, and the maximum value of the lower body search height range is the maximum height of the lower body; Starting from the maximum height of the lower body, based on the second search height step and the second current search step number, search the first point cloud for a point cloud whose height value is within the lower body search height range and higher than the second current search height, and use the currently searched point cloud as the fourth point cloud; wherein the second current search height is equal to the difference between the maximum height of the lower body and the second search distance, the second search distance is equal to the product of the second search height step and the second current search step number, and the initial value of the second current search step number is 1; clustering the fourth point cloud based on the at least one head point set to classify the fourth point cloud that meets the clustering condition into the at least one head point set; Let the second current search step number = the second current search step number + 1, return to execute the step and subsequent steps of searching for a point cloud in the first point cloud whose height value is within the lower body search height range and higher than the second current search height based on the second search height step number and the second current search step number, and use the currently searched point cloud as the fourth point cloud, until the second current search height is less than the minimum height of the lower body.

3. The method according to claim 1, characterized in that The method of matching a currently processed current frame segmentation result with all previous frame segmentation results based on the intersection between the current frame segmentation result and the previous frame segmentation result, and adding 1 to the traffic count if the number of successful matches of the currently processed current frame segmentation result is greater than a preset successful match number threshold, includes: Calculate the coverage area of ​​all current frame segmentation results in the coordinate system of width dimension and time dimension; Intersecting the coverage area of ​​the target current frame segmentation result with the coverage area of ​​the target previous frame segmentation result, calculating the intersection area between the target current frame segmentation result and the target previous frame segmentation result, obtaining and counting the intersection areas between all current frame segmentation results and previous frame segmentation results, and obtaining a statistically obtained overlap table; wherein the target current frame segmentation result is any one of all current frame segmentation results, the target previous frame segmentation result is any one of all previous frame segmentation results, and a row or a column of the overlap table indicates the intersection area of ​​a current frame segmentation result and all previous frame segmentation results; Reading an initial row or initial column in the overlap table, and determining the number of previous frame segmentation results in the initial row or initial column that meet a preset matching requirement; wherein the preset matching requirement is that the intersection area between the previous frame segmentation result and the initial current frame segmentation result corresponding to the initial row or initial column is greater than half the coverage area of ​​the initial current frame segmentation result; If the number of previous frame segmentation results that meet the preset matching requirements is 1, the match is deemed successful, and the area difference between the intersection area of ​​the successfully matched previous frame segmentation result and the initial current frame segmentation result and the coverage area of ​​the initial current frame segmentation result is calculated. If the area difference is less than or equal to the preset area difference threshold, the number of successful matches of the successfully matched previous frame segmentation result is increased by one, and the value is assigned to the number of successful matches of the initial current frame segmentation result; If the number of successful matches of the initial current frame segmentation result is greater than a preset successful match number threshold, the traffic count is increased by 1; The step of using the next current frame segmentation result as the current frame segmentation result currently being processed, returning to execute the step of matching the current frame segmentation result currently being processed with all previous frame segmentation results based on the intersection between the current frame segmentation result and the previous frame segmentation result, and subsequent steps until all current frame segmentation results are traversed, including: Let the initial row = initial row + 1, or let the initial column = initial column + 1, return to execute the step of determining the number of upper frame segmentation results that meet the preset matching requirements in the initial row or initial column and subsequent steps until all rows or all columns in the overlap table are read.

4. The method according to claim 1, wherein After the traffic count is increased by 1, the following steps are also included: Performing principal component analysis on the current frame segmentation result being processed, and obtaining a principal component direction vector corresponding to the current frame segmentation result being processed; Calculating the angle between the principal component direction vector and the set in-and-out direction vector, and determining the in-and-out direction of the current frame segmentation result being processed according to the calculated angle; Get the height dimension data of all point clouds in the height dimension and the time dimension data in the time dimension in the current frame segmentation result being processed; Calculating height mean data based on all height dimensional data, and decentralizing all height dimensional data based on the height mean data to obtain decentralized height dimensional data; Calculating time mean data based on all time dimension data, and decentralizing all time dimension data based on the time mean data to obtain decentralized time dimension data; A covariance matrix is ​​constructed based on the decentralized height dimension data and time dimension data, the eigenvector of the covariance matrix is ​​solved, and the principal component direction vector corresponding to the current frame segmentation result currently being processed is determined based on the solved eigenvector.

5. The method according to claim 1, wherein The step of obtaining a two-dimensional point cloud of the current frame and adding a time dimension to the two-dimensional point cloud to obtain a three-dimensional point cloud includes: Obtaining a two-dimensional point cloud of the current frame, performing foreground separation and ground registration on the two-dimensional point cloud, and obtaining a registered two-dimensional foreground point cloud; Adding a time dimension to the two-dimensional foreground point cloud to obtain a three-dimensional foreground point cloud; The three-dimensional point cloud is clustered to obtain a plurality of preliminary detection results, including: The three-dimensional foreground point clouds are clustered to obtain a plurality of preliminary detection results.

6. A method for real-time detection of vehicle flow based on laser radar, characterized in that: The method comprises: Obtaining a two-dimensional point cloud of the current frame, adding a time dimension to the two-dimensional point cloud, and obtaining a three-dimensional point cloud; wherein the two-dimensional point cloud includes a width dimension and a height dimension; Clustering the three-dimensional point cloud to obtain a plurality of preliminary detection results; wherein the preliminary detection result is a set of point clouds that are preliminarily determined to belong to the same class; Calculate the distance in the width dimension and the distance in the time dimension of all preliminary detection results, merge the preliminary detection results according to the distance in the width dimension and the distance in the time dimension, and determine the retained detection results; use one retained detection result as a current frame segmentation result, use all searched current frame segmentation results as the segmentation results of the current frame, and obtain the segmentation result of the previous frame containing multiple previous frame segmentation results; Based on the intersection between the segmentation result of the current frame and the segmentation result of the previous frame, a currently processed current frame segmentation result is matched with all the segmentation results of the previous frame. If the number of successful matches of the currently processed current frame segmentation result is greater than the preset successful match number threshold, the traffic flow count is increased by 1; Taking the next current frame segmentation result as the current frame segmentation result currently being processed, returning to the step of performing the intersection between the current frame segmentation result and the previous frame segmentation result, matching the current frame segmentation result currently being processed with all previous frame segmentation results, and subsequent steps, until all current frame segmentation results are traversed; The method of matching a currently processed current frame segmentation result with all previous frame segmentation results based on the intersection between the current frame segmentation result and the previous frame segmentation result, and adding 1 to the vehicle flow count if the number of successful matches of the currently processed current frame segmentation result is greater than a preset successful match number threshold, includes: Calculate the coverage area of ​​all current frame segmentation results in the coordinate system of width dimension and time dimension; Intersecting the coverage area of ​​the target current frame segmentation result with the coverage area of ​​the target previous frame segmentation result, calculating the intersection area between the target current frame segmentation result and the target previous frame segmentation result, obtaining and counting the intersection areas between all current frame segmentation results and previous frame segmentation results, and obtaining a statistically obtained overlap table; wherein the target current frame segmentation result is any one of all current frame segmentation results, the target previous frame segmentation result is any one of all previous frame segmentation results, and a row or a column of the overlap table indicates the intersection area of ​​a current frame segmentation result and all previous frame segmentation results; Reading an initial row or initial column in the overlap table, and determining the number of previous frame segmentation results in the initial row or initial column that meet a preset matching requirement; wherein the preset matching requirement is that the intersection area between the previous frame segmentation result and the initial current frame segmentation result corresponding to the initial row or initial column is greater than half the coverage area of ​​the initial current frame segmentation result; If the number of previous frame segmentation results that meet the preset matching requirements is 1, the match is deemed successful, and the area difference between the intersection area of ​​the successfully matched previous frame segmentation result and the initial current frame segmentation result and the coverage area of ​​the initial current frame segmentation result is calculated. If the area difference is less than or equal to the preset area difference threshold, the number of successful matches of the successfully matched previous frame segmentation result is increased by one, and the value is assigned to the number of successful matches of the initial current frame segmentation result; If the number of successful matches of the initial current frame segmentation result is greater than a preset successful match threshold, the vehicle flow count is increased by 1; The step of using the next current frame segmentation result as the current frame segmentation result currently being processed, returning to execute the step of matching the current frame segmentation result currently being processed with all previous frame segmentation results based on the intersection between the current frame segmentation result and the previous frame segmentation result, and subsequent steps until all current frame segmentation results are traversed, including: Let the initial row = initial row + 1, or let the initial column = initial column + 1, return to execute the step of determining the number of upper frame segmentation results that meet the preset matching requirements in the initial row or initial column and subsequent steps until all rows or all columns in the overlap table are read.

7. The method according to claim 6, characterized in that The merging of the preliminary detection results according to the distance in the width dimension and the distance in the time dimension and determining the retained detection results includes: If there are two or more preliminary detection results that meet the first merging requirement, the preliminary detection results that meet the first merging requirement are merged and retained, and the preliminary detection results that do not meet the first merging requirement are retained; wherein the first merging requirement is that the distance between the two preliminary detection results in the width dimension is less than or equal to a preset first width threshold, and the distance in the time dimension is less than or equal to a preset number of frames; All preliminary detection results that are located in the same lane and whose maximum distance in the width dimension is less than a preset second width threshold are merged. If the merged detection result meets the second merging requirement, it is retained; if the merged detection result does not meet the second merging requirement, the preliminary detection result before the merging is retained; wherein, the second merging requirement is that the maximum distance in the width dimension is less than a preset third width threshold; wherein, the third width threshold indicates the maximum vehicle width of the vehicle.

8. The method according to claim 6, characterized in that After the vehicle flow count is increased by 1, the method further includes: The maximum distance of the current frame segmentation result in the current processing in the width dimension is used as the width of the currently detected vehicle, and the maximum distance in the height dimension is used as the height of the currently detected vehicle.

9. A device for real-time detection of pedestrian and vehicle flow based on laser radar, comprising a memory and a processor, characterized in that: The memory stores a computer program, and when the computer program is executed by the processor, the processor is caused to perform the steps of the method according to any one of claims 1 to 8.

Images