Multi-information fusion speed measurement method and system

Abstract

The invention relates to the technical field of traffic monitoring, in particular to a multivariate information fusion speed measurement method and system, and the method comprises the following steps: installing an image collection device and a three-dimensional laser radar, obtaining an image of a monitoring area through the image collection device, and obtaining the point cloud data of the monitoring area through the three-dimensional laser radar; constructing a vehicle intelligent identification model, and identifying a vehicle in the monitoring area image by using the vehicle intelligent identification model to obtain a first vehicle identification result; preprocessing the monitoring area point cloud data, and identifying vehicles in the monitoring area point cloud data based on point cloud segmentation and point cloud clustering to obtain a second vehicle identification result; and determining the vehicle position according to the first vehicle identification result and the second vehicle identification result, and further obtaining the vehicle speed by tracking and monitoring the vehicle. According to the multi-information fusion speed measurement method provided by the invention, the speed measurement distance and the speed measurement precision in a complex environment can be improved, and optimization of traffic flow management and improvement of road traffic efficiency are facilitated.

Description

Technical Field

[0001] The present invention relates to the technical field of traffic monitoring, and in particular to a multi-information fusion speed measurement method and system. Background Art

[0002] Vehicle speed measurement technology is not only a fundamental means of obtaining vehicle speed but also crucial for ensuring traffic safety, optimizing traffic flow management, and improving road efficiency. Within this field, 3D LiDAR, with its high-precision, high-resolution ranging capabilities, demonstrates tremendous potential. It generates point cloud data containing (X, Y, Z) coordinates, providing powerful technical support for object detection, tracking, and speed measurement in three-dimensional space. Consequently, 3D LiDAR has gained widespread application in vehicle speed measurement.

[0003] However, despite its many advantages, 3D LiDAR also faces numerous challenges in practical applications. First, the ranging accuracy of 3D LiDAR is affected by atmospheric density, weather conditions, and light attenuation and scattering, factors that can lead to deviations in the measurement results. Second, the size, shape, and motion of the target can also interfere with the speed measurement accuracy of 3D LiDAR. For example, smaller targets may be difficult to accurately identify and their speed cannot be accurately measured. Given the limitations of 3D LiDAR in vehicle speed measurement, it is necessary to explore a more efficient and accurate speed measurement method. Summary of the Invention

[0004] In view of the defects in the prior art, the present invention provides a multi-information fusion speed measurement method and system.

[0005] In order to achieve the above-mentioned objectives, in a first aspect, the present invention provides a multi-information fusion speed measurement method, which includes the following steps: installing an image acquisition device and a three-dimensional laser radar on the roadside, and using the image acquisition device to obtain an image of the monitoring area, while using the three-dimensional laser radar to obtain point cloud data of the monitoring area; constructing a vehicle intelligent recognition model, and using the vehicle intelligent recognition model to identify vehicles in the image of the monitoring area to obtain a first vehicle recognition result; pre-processing the point cloud data of the monitoring area, and then identifying vehicles in the point cloud data of the monitoring area based on point cloud segmentation and point cloud clustering to obtain a second vehicle recognition result; determining the vehicle position based on the first vehicle recognition result and the second vehicle recognition result, and then obtaining the vehicle speed by tracking and monitoring the vehicle. The multi-information fusion speed measurement method provided by the present invention can improve the speed measurement distance and speed measurement accuracy in complex environments, which is conducive to optimizing traffic flow management and improving road traffic efficiency.

[0006] Optionally, constructing a vehicle intelligent recognition model and using the vehicle intelligent recognition model to identify vehicles in the monitoring area image to obtain a first vehicle recognition result includes the following steps: Construct a historical monitoring area image dataset and then build a vehicle intelligent recognition model based on the YOLOv5s target detection algorithm; The vehicle intelligent recognition model is used to identify vehicles in the monitoring area image to obtain a first vehicle recognition result.

[0007] The present invention builds a vehicle intelligent recognition model based on YOLOv5s, which not only improves the recognition efficiency but also reduces the requirements for hardware equipment for speed measurement, thereby improving the practicality of the method.

[0008] Optionally, constructing a historical monitoring area image dataset and then constructing a vehicle intelligent recognition model based on the YOLOv5s target detection algorithm includes the following steps: For the YOLOv5s object detection algorithm, the ordinary convolutions in the second and fourth C3 structures in its backbone network are replaced with dynamic snake convolutions to obtain the first vehicle recognition network; Introducing the CA attention mechanism between the backbone network and the neck network of the first vehicle recognition network to obtain a second vehicle recognition network; The loss function of the second vehicle identification network is optimized to obtain a third vehicle identification network, where the loss function of the third vehicle identification network satisfies the following relationship: in, For the overall loss, is the distance loss, is the shape loss, is the angle loss, is the area of ​​the real box, is the area of ​​the prediction box, w is the width of the prediction box, h is the height of the prediction box, is the width of the real frame, is the height of the real frame, is the height difference between the center point of the real box and the predicted box, and d is the distance between the center point of the real box and the predicted box; The historical monitoring area image dataset is constructed to complete the training and verification of the third vehicle recognition network, thereby obtaining the vehicle intelligent recognition model.

[0009] The present invention introduces dynamic snake convolution and CA attention mechanism into the YOLOv5s target detection algorithm and optimizes its loss function, thereby improving the algorithm's accuracy in recognizing long-distance targets and the algorithm's training speed, which is beneficial to improving the accuracy of speed measurement distance and results.

[0010] Optionally, preprocessing the monitoring area point cloud data, and then identifying vehicles in the monitoring area point cloud data based on point cloud segmentation and point cloud clustering to obtain a second vehicle identification result includes the following steps: Preprocessing the point cloud data of the monitoring area to obtain clean point cloud thickened data; performing point cloud segmentation on the clean point cloud thickening data to obtain segmented point cloud data; Point cloud clustering is performed on the segmented point cloud data to obtain a vehicle point cloud cluster of the vehicle, and abnormal point clouds in the vehicle point cloud cluster are eliminated to obtain the second vehicle recognition result.

[0011] The present invention obtains vehicle point cloud clusters by preprocessing, segmenting and clustering the point cloud data of the monitoring area, and eliminates abnormal point clouds in the vehicle point cloud clusters, thereby improving the accuracy of vehicle identification based on lidar, which is conducive to improving the accuracy of speed measurement distance and speed measurement results.

[0012] Optionally, the preprocessing of the monitoring area point cloud data to obtain clean point cloud thickened data comprises the following steps: Superimposing the monitoring area point cloud data of the current frame with the monitoring area point cloud data of the previous four frames to obtain superimposed point cloud data of the current frame; Voxel filtering is performed on the superimposed point cloud data to obtain the clean point cloud thickened data.

[0013] The present invention superimposes point clouds from different frames to increase the number of available point clouds, which can, to a certain extent, solve the problem of being unable to accurately identify vehicles at a long distance due to sparse point clouds, thereby increasing the speed measurement distance.

[0014] Optionally, performing point cloud segmentation on the clean point cloud densified data to obtain segmented point cloud data comprises the following steps: Use the random sampling consensus algorithm to preliminarily extract the road surface point cloud and obtain the road surface point cloud set; For a certain road point cloud in the road point cloud set, searching for its adjacent point clouds among all point clouds, and adding the adjacent point cloud to the point cloud cluster where the road point cloud is located if the adjacent point cloud meets the aggregation condition; The aggregation conditions satisfy the following relationship: in, 、 and are the coefficients of the planar model fitted by the random sampling consensus algorithm, is the constant term of the plane model fitted by the random sampling consensus algorithm, is the elevation difference between the road point cloud and its adjacent point clouds, n is the number of road point clouds, L is the horizontal distance between the road point cloud and its adjacent point clouds, is the constant term of the plane model where the i-th road point cloud is located, is a constant term of the plane model where the adjacent point clouds of the road point cloud are located; All road surface point clouds in the road surface point cloud set are traversed, and point cloud segmentation of the clean point cloud thickening data is performed to obtain the segmented point cloud data.

[0015] The present invention is based on a random sampling consensus algorithm and combines clustering conditions to achieve accurate segmentation of point cloud data, laying a solid and reliable foundation for subsequent identification of vehicle point cloud clusters through point cloud clustering.

[0016] Optionally, performing point cloud clustering on the segmented point cloud data to obtain a vehicle point cloud cluster of the vehicle, and removing abnormal point clouds in the vehicle point cloud cluster to obtain the second vehicle recognition result includes the following steps: The shape enclosed by four adjacent point clouds in two adjacent rows of point clouds is regarded as a rectangle, and the diagonal length of the rectangle is used as the minimum clustering radius of the DBSCAN algorithm, and then the segmented point cloud data is clustered using the DBSCAN algorithm to obtain the vehicle point cloud cluster of the vehicle; Calculating a first average distance from each point cloud to other point clouds to construct a point cloud distance dataset, and calculating an outlier parameter based on the point cloud distance dataset; In the vehicle point cloud cluster, calculating a second average distance between each vehicle point cloud and other vehicle point clouds, and calculating a mean and a standard deviation of the second average distance, and then calculating a distance threshold in combination with the outlier parameter; For any vehicle point cloud in the vehicle point cloud cluster, when its corresponding average distance is greater than the distance threshold, the vehicle point cloud is regarded as an abnormal point cloud in the vehicle point cloud cluster and is removed; After all abnormal point clouds are eliminated, the vehicle point cloud cluster is the second vehicle recognition result.

[0017] The present invention improves the DBSCAN algorithm to accurately obtain vehicle point cloud clusters and further eliminates abnormal point clouds in the vehicle point cloud clusters, thereby improving the accuracy of vehicle recognition results based on lidar, which is beneficial to improving the speed measurement distance and the accuracy of speed measurement results in complex environments.

[0018] Optionally, the outlier parameter and the distance threshold respectively satisfy the following relationship: in, is the outlier parameter, is the skewness coefficient of the point cloud distance dataset, is the kurtosis coefficient of the point cloud distance dataset, is the distance threshold, is the mean of the second average distance, is the standard deviation of the second mean distance.

[0019] Optionally, determining the vehicle position according to the first vehicle identification result and the second vehicle identification result, and then obtaining the vehicle speed by tracking and monitoring the vehicle includes the following steps: Determine a first coordinate of the vehicle in the image acquisition device coordinate system based on the first vehicle recognition result, and simultaneously determine a second coordinate of the vehicle in the radar coordinate system based on the second vehicle recognition result; Performing spatiotemporal synchronization on the image acquisition device and the three-dimensional laser radar, and then determining a final coordinate position of the vehicle based on the first coordinate and the second coordinate; The final coordinate position of the vehicle at different times is obtained by tracking and monitoring the vehicle, and then the vehicle speed is calculated.

[0020] The present invention locates the vehicle based on the vehicle position obtained by image acquisition equipment and three-dimensional laser radar, thereby improving the information perception dimension and further improving the accuracy and robustness of vehicle speed measurement in complex environments.

[0021] In a second aspect, the present invention provides a multi-information fusion speed measurement system, comprising: a data acquisition device, a data output device, a processor, and storage; the storage comprising a computer-readable storage medium storing a computer program, the computer program comprising program instructions that, when executed by the processor, cause the processor to implement a multi-information fusion speed measurement method provided by the present invention. This system is capable of stably executing the method provided by the present invention, thereby improving both speed measurement efficiency and practicality. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments. It should be understood that the following drawings only illustrate certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without creative work.

[0023] Figure 1 Schematic diagram of a flow chart of a multi-information fusion speed measurement method according to an embodiment of the present invention; Figure 2 Schematic diagram of four adjacent point clouds in two adjacent rows of point clouds according to an embodiment of the present invention; Figure 3 Schematic diagram of the framework of a multi-information fusion speed measurement system according to an embodiment of the present invention. DETAILED DESCRIPTION

[0024] Specific embodiments of the present invention will be described in detail below. It should be noted that the embodiments described herein are for illustrative purposes only and are not intended to limit the present invention. In the following description, numerous specific details are set forth to provide a thorough understanding of the present invention. However, it will be apparent to one of ordinary skill in the art that these specific details are not necessarily required to practice the present invention. In other instances, well-known circuits, software, or methods are not specifically described to avoid obscuring the present invention.

[0025] Throughout this specification, references to "one embodiment," "an embodiment," "an example," or "an example" mean that a particular feature, structure, or characteristic described in connection with the embodiment or example is included in at least one embodiment of the present invention. Therefore, appearances of the phrases "in one embodiment," "in an embodiment," "an example," or "an example" in various places throughout this specification are not necessarily all referring to the same embodiment or example. Furthermore, the particular features, structures, or characteristics may be combined in any suitable combinations and / or subcombinations in one or more embodiments or examples. Furthermore, those of ordinary skill in the art will appreciate that the figures provided herein are for illustrative purposes only and are not necessarily drawn to scale.

[0026] It should be noted in advance that, in an optional embodiment, except for independent explanations, the same symbols or letters appearing in all formulas have the same meanings and values.

[0027] In an alternative embodiment, see Figure 1 This embodiment provides a multi-information fusion speed measurement method, which includes the following steps: S1. Install an image acquisition device and a three-dimensional laser radar on the roadside, and use the image acquisition device to obtain images of the monitoring area, and use the three-dimensional laser radar to obtain point cloud data of the monitoring area.

[0028] Specifically, in this embodiment, a certain intersection is used as a monitoring area, and an image acquisition device is used to obtain an image of the monitoring area, and a three-dimensional laser radar is used to obtain point cloud data of the monitoring area.

[0029] Furthermore, the image acquisition device is specifically a high-definition camera. The models of the high-definition camera and the three-dimensional laser radar can be selected according to actual conditions and are not limited here. In other optional embodiments, the image acquisition device can also be a high-definition camera.

[0030] S2. Construct a vehicle intelligent recognition model, and use the vehicle intelligent recognition model to identify vehicles in the monitoring area image to obtain a first vehicle recognition result.

[0031] Wherein, step S2 specifically includes the following steps: S21. Build a historical monitoring area image dataset, and then build a vehicle intelligent recognition model based on the YOLOv5s target detection algorithm.

[0032] The YOLOv5s target detection algorithm has the advantages of high precision, high efficiency, and lightweight. Building a vehicle intelligent recognition model based on the YOLOv5s target detection algorithm can not only improve vehicle recognition efficiency, but also reduce the hardware requirements for speed measurement, thereby improving the practicality of this method. Step S21 specifically includes the following steps: S211. For the YOLOv5s target detection algorithm, the ordinary convolutions in the second and fourth C3 structures in its backbone network are replaced with dynamic snake convolutions to obtain the first vehicle recognition network.

[0033] Specifically, in this embodiment, the backbone network of the YOLOv5s target detection algorithm includes four C3 structures. By introducing dynamic snake convolution into the second and fourth C3 structures in its backbone network, the algorithm can improve the detection of small local features and thereby improve the algorithm's ability to recognize long-distance vehicles.

[0034] S212. Introduce a CA attention mechanism between the backbone network and the neck network of the first vehicle recognition network to obtain a second vehicle recognition network.

[0035] Specifically, in this embodiment, the output of the SPP structure in the backbone network of the first vehicle recognition network is used as the input of the CA attention mechanism, and the output of the CA attention mechanism is used as the input of the neck network of the first vehicle recognition network. Introducing the CA attention mechanism in the first vehicle recognition network can improve the network's focus on key features, improve the accuracy of vehicle recognition, and thus improve the accuracy of speed measurement.

[0036] S213: Optimize the loss function of the second vehicle identification network to obtain a third vehicle identification network.

[0037] Specifically, in this embodiment, the loss function of the third vehicle identification network satisfies the following relationship: in, For the overall loss, is the distance loss, is the shape loss, is the angle loss, is the area of ​​the real box, is the area of ​​the prediction box, w is the width of the prediction box, h is the height of the prediction box, is the width of the real frame, is the height of the real frame, is the height difference between the center point of the real box and the predicted box, and d is the distance between the center point of the real box and the predicted box.

[0038] The loss function of the third vehicle recognition network takes into account the distance loss, shape loss and angle loss between the predicted box and the real box, effectively reducing the freedom of parameter selection during the training of the third vehicle recognition network, improving the convergence speed and target detection accuracy during the training of the third vehicle recognition network, and helping to improve the accuracy of the speed measurement distance and speed measurement results.

[0039] S214: Construct the historical monitoring area image dataset to complete the training and verification of the third vehicle recognition network, and then obtain the vehicle intelligent recognition model.

[0040] Specifically, in this embodiment, an image acquisition device is used to collect historical monitoring images of the monitoring area, and all the collected historical monitoring images are used to construct a historical monitoring area image dataset, and then the historical monitoring area image dataset is used to complete the training and verification of the third vehicle recognition network, and finally a vehicle intelligent recognition model is obtained.

[0041] S22: Use the vehicle intelligent recognition model to identify vehicles in the monitoring area image to obtain a first vehicle recognition result.

[0042] S3. Preprocess the point cloud data of the monitoring area, and then identify vehicles in the point cloud data of the monitoring area based on point cloud segmentation and point cloud clustering to obtain a second vehicle identification result.

[0043] This embodiment preprocesses the monitoring area point cloud data, performs point cloud segmentation and point cloud clustering to obtain vehicle point cloud clusters, and removes abnormal point clouds in the vehicle point cloud clusters, thereby improving the accuracy of vehicle identification based on lidar, which is conducive to improving the accuracy of speed measurement distance and speed measurement results. Step S3 specifically includes the following steps: S31 . Preprocess the point cloud data of the monitoring area to obtain clean point cloud thickened data.

[0044] Wherein, step S31 specifically includes the following steps: S311 , superimposing the monitoring area point cloud data of the current frame with the monitoring area point cloud data of the previous four frames to obtain superimposed point cloud data of the current frame.

[0045] Specifically, in this embodiment, one of the factors affecting the speed measurement distance of a 3D LiDAR is that the point cloud becomes increasingly sparse as distance increases, so that the number of point clouds is insufficient to accurately calculate vehicle speed. To address this issue, this embodiment superimposes the monitoring area point cloud data of the current frame with the monitoring area point cloud data of the previous four frames to increase the number of available point clouds, making the recognition target more prominent. This solves the problem of being unable to accurately identify vehicles at a distance due to sparse point clouds and improves the speed measurement distance.

[0046] S312: Perform voxel filtering on the superimposed point cloud data to obtain the clean point cloud thickened data.

[0047] Specifically, in this embodiment, after step S311, the number of point clouds in the current frame will increase significantly, which greatly increases the data processing volume. Therefore, this embodiment performs voxel filtering on the superimposed point cloud data, thereby reducing the point cloud density without changing the approximate shape and geometric structure of the point cloud, reducing the data processing volume, and increasing the vehicle recognition speed, thereby improving the speed measurement efficiency.

[0048] S32: performing point cloud segmentation on the clean point cloud thickening data to obtain segmented point cloud data.

[0049] Wherein, step S32 specifically includes the following steps: S321. Use a random sampling consensus algorithm to preliminarily extract a road surface point cloud to obtain a road surface point cloud set.

[0050] Specifically, in this embodiment, a random sampling consensus algorithm is used to process the clean point cloud thickening data, and then a road surface point cloud is preliminarily extracted to obtain a road surface point cloud set.

[0051] More specifically, the random sampling consensus algorithm is the RANSAC algorithm. The road point cloud initially extracted is the point cloud corresponding to the plane model fitted using the RANSAC algorithm. When the road surface has a certain slope, the road point cloud obtained using the RANSAC algorithm is usually only a part of all road point clouds.

[0052] S322 . For a certain road point cloud in the road point cloud set, search for its adjacent point cloud in all point clouds, and add the adjacent point cloud to the point cloud cluster where the road point cloud is located if the adjacent point cloud meets the aggregation condition.

[0053] Specifically, in this embodiment, for a certain road point cloud o in the road point cloud set, its adjacent point clouds are searched among all the point clouds contained in the clean point cloud thickening data. If one of its adjacent point clouds p meets the aggregation condition, the adjacent point cloud p is added to the point cloud cluster where the road point o cloud is located, that is, the adjacent point cloud p is another road point cloud adjacent to the road point cloud o.

[0054] More specifically, the aggregation conditions satisfy the following relationship: in, 、 and are the coefficients of the planar model fitted by the random sampling consensus algorithm, is the constant term of the plane model fitted by the RANSAC algorithm, is the elevation difference between the road point cloud and its adjacent point clouds, n is the number of road point clouds, L is the horizontal distance between the road point cloud and its adjacent point clouds, is the constant term of the plane model where the i-th road point cloud is located, is a constant term of the plane model of the adjacent point clouds of the road surface point cloud. The plane where the road surface point cloud is located is parallel to the plane where its adjacent point clouds are located, and the plane where the road surface point cloud is located is parallel to the plane fitted by the random sampling consensus algorithm.

[0055] Furthermore, the first relational expression in the aggregation condition actually reflects whether the vertical distance between a road point cloud in the road point cloud set and its adjacent points is less than a certain threshold, and the second relational expression in the aggregation condition actually reflects whether the distance between a road point cloud in the road point cloud set and the plane containing its adjacent points is less than a certain threshold. By searching for adjacent point clouds in the road point cloud set and adding adjacent point clouds to the point cloud cluster if the adjacent point clouds meet the aggregation condition, we can find as many road point clouds as possible in the clean point cloud densification data, thereby improving the accuracy of point cloud segmentation.

[0056] S323 , traversing all road point clouds in the road point cloud set, and then performing point cloud segmentation on the clean point cloud thickened data to obtain the segmented point cloud data.

[0057] Specifically, in this embodiment, all road point clouds in the road point cloud set are processed according to the description of step S322, and then all road point clouds are extracted to achieve accurate segmentation of the clean point cloud thickening data to obtain segmented point cloud data.

[0058] This embodiment is based on the random sampling consensus algorithm and combines the clustering conditions to achieve accurate segmentation of point cloud data, laying a solid and reliable foundation for the subsequent identification of vehicle point cloud clusters through point cloud clustering.

[0059] S33 , performing point cloud clustering on the segmented point cloud data to obtain a vehicle point cloud cluster of the vehicle, and removing abnormal point clouds in the vehicle point cloud cluster to obtain the second vehicle recognition result.

[0060] This step improves the DBSCAN algorithm to accurately obtain vehicle point cloud clusters and further eliminates abnormal point clouds in the vehicle point cloud clusters, thereby improving the accuracy of vehicle recognition results based on lidar, thereby improving the accuracy of speed measurement distance and speed measurement results in complex environments. Step S33 specifically includes the following steps: S331. The shape enclosed by four adjacent point clouds in two adjacent rows of point clouds is regarded as a rectangle, and the diagonal length of the rectangle is used as the minimum clustering radius of the DBSCAN algorithm. Then, the DBSCAN algorithm is used to cluster the segmented point cloud data to obtain the vehicle point cloud cluster of the vehicle.

[0061] Specifically, in this embodiment, the three-dimensional laser radar scans the vehicle by rotating the laser beam. The closer the position is to the three-dimensional laser radar, the denser the point cloud is, and the farther the position is from the three-dimensional laser radar, the sparser the point cloud is.

[0062] See Figure 2 For the scanned vehicle, in the four adjacent point clouds a1, a2, a3, and a4 in two adjacent rows of point clouds, point clouds a1 and a2 are in the same row, point clouds a3 and a4 are in the same row, and the distance between point clouds a1 and a2 is greater than the distance between point clouds a3 and a4, that is, the point clouds in the rows where a1 and a2 are located are sparser than the point clouds in the rows where a3 and a4 are located. Therefore, the distance between a1 and a2 and the 3D lidar is greater than the distance between a3 and a4 and the 3D lidar. However, considering that the distance between adjacent point clouds is usually small, the distance between point clouds a1, a2, a3, and a4 and the 3D lidar is considered to be equal. If the distance between point clouds a1, a2, a3, and a4 and the 3D lidar is considered to be equal, then the shape enclosed by point clouds a1, a2, a3, and a4 can be considered as a rectangle.

[0063] Furthermore, if the shape enclosed by point clouds a1, a2, a3, and a4 is regarded as a rectangle, then the distance between point clouds a1 and a4 will be proportional to the distance between the rectangle and the three-dimensional lidar. If the distance between point clouds a1 and a4 is set as the minimum clustering radius of the DBSCAN algorithm, then the minimum clustering radius of the DBSCAN algorithm can be adaptively adjusted according to the distance between the three-dimensional lidar and the vehicle, thereby solving the problem of poor point cloud clustering effect of distant vehicles, improving the recognition accuracy of vehicles, and thus improving the accuracy of speed measurement results.

[0064] After setting the minimum clustering radius in the DBSCAN algorithm, the segmented point cloud data can be clustered using the DBSCAN algorithm to obtain vehicle point cloud clusters. All vehicle point clouds in a vehicle point cloud cluster come from scanning the same vehicle.

[0065] S332 : Calculate a first average distance from each point cloud to other point clouds to construct a point cloud distance dataset, and calculate an outlier parameter based on the point cloud distance dataset.

[0066] Specifically, in this embodiment, the Euclidean distance from a point cloud to all other point clouds included in the clean point cloud densified data can be calculated, and then the first average distance from this point cloud to the other point clouds can be calculated, that is, the mean of the Euclidean distances from this point cloud to the other point clouds. Furthermore, the first average distance from each point cloud included in the clean point cloud densified data to the other point clouds can be calculated, and then the first average distances of all point clouds can be used to construct a point cloud distance dataset.

[0067] Furthermore, after obtaining the point cloud distance dataset, the skewness coefficient and kurtosis coefficient of the point cloud distance dataset are calculated to verify the degree to which the first average distance in the point cloud distance dataset is close to the normal distribution, so as to verify in which value range the first average distance is concentrated, and then determine the outlier parameter by referring to the characteristics of the normal distribution data. The outlier parameter specifically satisfies the following relationship: in, is the outlier parameter, is the skewness coefficient of the point cloud distance dataset, is the kurtosis coefficient of the point cloud distance dataset. The skewness coefficient and kurtosis coefficient satisfy the following relationship: Among them, m is the number of the first average distance in the point cloud distance dataset, is the jth first average distance in the point cloud distance dataset, is the mean of the first average distance in the point cloud distance dataset, and s is the standard deviation of the first average distance in the point cloud distance dataset. The closer the skewness coefficient is to 0, the better the first average distance is. The better the symmetry; the closer the kurtosis coefficient is to 3, the closer the steepness of the change in the first average distance is to the normal distribution.

[0068] S333: Calculate a second average distance between each vehicle point cloud and other vehicle point clouds in the vehicle point cloud cluster, calculate a mean and a standard deviation of the second average distance, and then calculate a distance threshold in combination with the outlier parameter.

[0069] Specifically, in this embodiment, for all vehicle point clouds in a certain vehicle point cloud cluster, their respective second average distances can be calculated in the same way as the first average distance, and then the mean and standard deviation of the second average distances are calculated. Finally, the distance threshold is calculated in combination with the outlier parameter. The distance threshold satisfies the following relationship: in, is the distance threshold, is the mean of the second average distance, is the standard deviation of the second mean distance.

[0070] S334 . For any vehicle point cloud in the vehicle point cloud cluster, when its corresponding average distance is greater than the distance threshold, the vehicle point cloud is regarded as an abnormal point cloud in the vehicle point cloud cluster and is removed.

[0071] S335: After all abnormal point clouds are eliminated, the vehicle point cloud cluster is the second vehicle recognition result.

[0072] S4. Determine the vehicle position according to the first vehicle identification result and the second vehicle identification result, and then obtain the vehicle speed by tracking and monitoring the vehicle.

[0073] Wherein, step S4 specifically includes the following steps: S41. Determine a first coordinate of the vehicle in an image acquisition device coordinate system according to the first vehicle recognition result, and simultaneously determine a second coordinate of the vehicle in a radar coordinate system according to the second vehicle recognition result.

[0074] Specifically, in this embodiment, after determining the position of the image acquisition device, an image acquisition device coordinate system can be established with the optical center of the image acquisition device as the origin, and then the first coordinate of the vehicle in the image acquisition device coordinate system can be obtained. The process of obtaining the first coordinate also involves calibration of the image acquisition device's intrinsic and extrinsic parameters, image distortion correction, and conversion from the pixel coordinate system to the image acquisition device coordinate system. However, these are existing technical means and will not be described in detail here.

[0075] Furthermore, after determining the position of the 3D laser radar, a corresponding radar coordinate system can be established. Since the second vehicle recognition result is a vehicle point cloud cluster, the coordinates of the center point of the circumscribed rectangle of the second vehicle recognition result can be used as the second coordinates of the vehicle in the radar coordinate system.

[0076] S42. Perform spatiotemporal synchronization on the image acquisition device and the three-dimensional laser radar, and then determine the final coordinate position of the vehicle according to the first coordinate and the second coordinate.

[0077] Specifically, in this embodiment, the image acquisition device and 3D LiDAR are spatiotemporally synchronized and then jointly calibrated. This allows the second coordinate in the LiDAR coordinate system to be converted to the image acquisition device coordinate system, and the conversion result is recorded as the third coordinate. Spatiotemporal synchronization and joint calibration of different sensors are conventional techniques and will not be described in detail here.

[0078] Furthermore, the midpoint of the line connecting the first and third coordinates is used as the final coordinate position of the vehicle. This embodiment determines the vehicle's position based on the vehicle's position acquired by the image acquisition device and the 3D LiDAR, thereby enhancing the dimension of information perception and thereby improving the accuracy and robustness of vehicle speed measurement in complex environments.

[0079] S43. Tracking and monitoring the vehicle to obtain the final coordinate position of the vehicle at different times, and then calculating the vehicle speed.

[0080] Specifically, in this embodiment, the vehicle is tracked based on the unscented Kalman filter and the Hungarian algorithm to obtain the final coordinate position of the vehicle at two different times, and then the vehicle speed is calculated, that is, .in, is the vehicle speed, for The final coordinate position of the vehicle at time The distance between the final coordinate positions of the vehicle at time .

[0081] It should be noted that, in some cases, the actions described in the specification can be performed in a different order and still achieve the desired results. In this embodiment, the order of steps given is only to make the embodiment appear clearer and easier to explain, rather than to limit it.

[0082] In an alternative embodiment, see Figure 2 In order to improve the speed measurement efficiency and the practicality of the present method, the present embodiment further provides a multi-information fusion speed measurement system, which includes: a data acquisition device 1, a data output device 2, a processor 3 and a storage 4, wherein the storage 4 includes a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by the processor 3, the processor 3 implements the contents described in steps S1 to S4.

[0083] Specifically, in this embodiment, the data acquisition device 1 includes an image acquisition device and a three-dimensional laser radar. The data output device 2 includes a digital display screen for outputting and displaying speed measurement results.

[0084] In summary, this method first obtains the vehicle's first coordinate based on image information captured by an image acquisition device, then obtains the vehicle's second coordinate based on point cloud information collected by a three-dimensional lidar. Finally, the vehicle's speed is calculated based on the first and second coordinates, thereby improving the accuracy and robustness of vehicle speed measurement in complex environments. In the process of obtaining the first coordinate, this method improves the YOLOv5s target detection algorithm to construct a vehicle intelligent recognition model, improving the algorithm's accuracy in long-range target recognition and its training speed. This improves the accuracy of the speed measurement distance and results, reduces the hardware requirements for speed measurement, and enhances the practicality of this method. In the process of obtaining the second coordinate, this method superimposes point clouds from different frames to increase the number of available point clouds. This can, to a certain extent, solve the problem of being unable to accurately identify vehicles at a distance due to sparse point clouds, thereby improving the speed measurement distance. Then, based on the random sampling consensus algorithm, combined with the clustering conditions, accurate segmentation of the point cloud data is achieved. Finally, by improving the DBSCAN algorithm to accurately obtain vehicle point cloud clusters, and further eliminating abnormal point clouds in the vehicle point cloud clusters, the speed measurement distance in complex environments and the accuracy of the speed measurement results are improved. In addition, the present invention also provides a system compatible with this method. This system can stably operate the method provided by the present invention, which can not only improve the speed measurement efficiency, but also improve the practicality of the present method.

[0085] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or make equivalent replacements for some or all of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the scope of the technical solutions of the embodiments of the present invention, and they should all be included in the scope of the claims and description of the present invention.

Claims

1. A multi-information fusion speed measurement method, characterized in that: The steps include: Install an image acquisition device and a three-dimensional laser radar on the roadside, and use the image acquisition device to obtain images of the monitoring area, and use the three-dimensional laser radar to obtain point cloud data of the monitoring area; Constructing a vehicle intelligent recognition model, and using the vehicle intelligent recognition model to recognize vehicles in the monitoring area image to obtain a first vehicle recognition result; Preprocessing the point cloud data of the monitoring area, and then identifying vehicles in the point cloud data of the monitoring area based on point cloud segmentation and point cloud clustering to obtain a second vehicle identification result; The vehicle position is determined according to the first vehicle identification result and the second vehicle identification result, and the vehicle speed is then acquired by tracking and monitoring the vehicle.

2. A multi-information fusion speed measurement method according to claim 1, characterized in that: The step of constructing a vehicle intelligent recognition model and using the vehicle intelligent recognition model to recognize vehicles in the monitoring area image to obtain a first vehicle recognition result comprises the following steps: Construct a historical monitoring area image dataset, and then build a vehicle intelligent recognition model based on the YOLOv5s target detection algorithm; The vehicle intelligent recognition model is used to recognize the vehicle in the monitoring area image to obtain a first vehicle recognition result.

3. A multi-information fusion speed measurement method according to claim 2, characterized in that: The construction of the historical monitoring area image dataset and then the construction of the vehicle intelligent recognition model based on the YOLOv5s target detection algorithm includes the following steps: For the YOLOv5s target detection algorithm, the ordinary convolutions in the second and fourth C3 structures in its backbone network are replaced with dynamic snake convolutions to obtain the first vehicle recognition network; Introducing a CA attention mechanism between the backbone network and the neck network of the first vehicle recognition network to obtain a second vehicle recognition network; The loss function of the second vehicle identification network is optimized to obtain a third vehicle identification network, wherein the loss function of the third vehicle identification network satisfies the following relationship: in, For the overall loss, is the distance loss, is the shape loss, is the angle loss, is the area of ​​the real box, is the area of ​​the prediction box, w is the width of the prediction box, h is the height of the prediction box, is the width of the real frame, is the height of the real frame, is the height difference between the center point of the real box and the predicted box, and d is the distance between the center point of the real box and the predicted box; The historical monitoring area image data set is constructed to complete the training and verification of the third vehicle recognition network, thereby obtaining the vehicle intelligent recognition model.

4. The multi-information fusion speed measurement method according to claim 1, characterized in that: The step of preprocessing the point cloud data of the monitoring area, and then identifying vehicles in the point cloud data of the monitoring area based on point cloud segmentation and point cloud clustering to obtain a second vehicle identification result comprises the following steps: Preprocessing the point cloud data of the monitoring area to obtain clean point cloud thickened data; Performing point cloud segmentation on the clean point cloud thickening data to obtain segmented point cloud data; Point cloud clustering is performed on the segmented point cloud data to obtain a vehicle point cloud cluster of the vehicle, and abnormal point clouds in the vehicle point cloud cluster are eliminated to obtain the second vehicle recognition result.

5. A multi-information fusion speed measurement method according to claim 4, characterized in that: The preprocessing of the monitoring area point cloud data to obtain clean point cloud thickened data comprises the following steps: Superimposing the monitoring area point cloud data of the current frame with the monitoring area point cloud data of the previous four frames to obtain superimposed point cloud data of the current frame; Voxel filtering is performed on the superimposed point cloud data to obtain the clean point cloud thickened data.

6. A multi-information fusion speed measurement method according to claim 4, characterized in that: The step of performing point cloud segmentation on the clean point cloud thickening data to obtain segmented point cloud data comprises the following steps: A road surface point cloud is initially extracted using a random sampling consensus algorithm to obtain a road surface point cloud set; For a road point cloud in the road point cloud set, searching for its adjacent point cloud in all point clouds, and adding the adjacent point cloud to the point cloud cluster where the road point cloud is located when the adjacent point cloud meets the aggregation condition; The aggregation conditions satisfy the following relationship: in, , and are the coefficients of the plane model fitted by the random sampling consensus algorithm, is the constant term of the plane model fitted by the random sampling consensus algorithm, is the elevation difference between the road point cloud and its adjacent point clouds, n is the number of road point clouds, L is the horizontal distance between the road point cloud and its adjacent point clouds, is the constant term of the plane model where the i-th road point cloud is located, is a constant term of the plane model where the adjacent point clouds of the road point cloud are located; All road surface point clouds in the road surface point cloud set are traversed, and point cloud segmentation of the clean point cloud thickening data is then performed to obtain the segmented point cloud data.

7. A multi-information fusion speed measurement method according to claim 4, characterized in that: The step of performing point cloud clustering on the segmented point cloud data to obtain a vehicle point cloud cluster of the vehicle, and removing abnormal point clouds in the vehicle point cloud cluster to obtain the second vehicle recognition result comprises the following steps: The shape enclosed by four adjacent point clouds in two adjacent rows of point clouds is regarded as a rectangle, and the diagonal length of the rectangle is used as the minimum clustering radius of the DBSCAN algorithm, and then the segmented point cloud data is clustered using the DBSCAN algorithm to obtain the vehicle point cloud cluster of the vehicle; Calculating a first average distance from each point cloud to other point clouds to construct a point cloud distance data set, and calculating an outlier parameter based on the point cloud distance data set; In the vehicle point cloud cluster, calculating a second average distance between each vehicle point cloud and other vehicle point clouds, and calculating a mean and a standard deviation of the second average distance, and then calculating a distance threshold in combination with the outlier parameter; For any vehicle point cloud in the vehicle point cloud cluster, when the corresponding average distance is greater than the distance threshold, the vehicle point cloud is regarded as an abnormal point cloud in the vehicle point cloud cluster and is removed; After all abnormal point clouds are eliminated, the vehicle point cloud cluster is the second vehicle recognition result.

8. A multi-information fusion speed measurement method according to claim 7, characterized in that: The outlier parameter and the distance threshold respectively satisfy the following relationship: in, is the outlier parameter, is the skewness coefficient of the point cloud distance dataset, is the kurtosis coefficient of the point cloud distance dataset, is the distance threshold, is the mean of the second average distance, is the standard deviation of the second mean distance.

9. A multi-information fusion speed measurement method according to claim 8, characterized in that: Determining the vehicle position according to the first vehicle recognition result and the second vehicle recognition result, and then obtaining the vehicle speed by tracking and monitoring the vehicle comprises the following steps: Determine a first coordinate of the vehicle in the image acquisition device coordinate system according to the first vehicle recognition result, and determine a second coordinate of the vehicle in the radar coordinate system according to the second vehicle recognition result; Performing spatiotemporal synchronization on the image acquisition device and the three-dimensional laser radar, and then determining a final coordinate position of the vehicle according to the first coordinate and the second coordinate; The final coordinate position of the vehicle at different times is obtained by tracking and monitoring the vehicle, and then the vehicle speed is calculated.

10. A multi-information fusion speed measurement system, characterized in that: The multi-information fusion speed measurement system includes: a data acquisition device, a data output device, a processor and a storage device, the storage device includes a computer-readable storage medium, the computer-readable storage medium stores a computer program, the computer program includes program instructions, and when the program instructions are executed by the processor, the processor implements the multi-information fusion speed measurement method as described in any one of claims 1-9.

Images