A highway scene event monitoring system and method

Abstract

The application discloses a highway scene event monitoring system and method, which specifically comprises the following steps: collecting point cloud data of a highway scene of a driving area of a target vehicle through a laser radar carried on the target vehicle in the highway scene; an edge processor receives the point cloud data of the highway scene collected by the corresponding laser radar, pre-processes the point cloud data of the highway scene, and transmits the pre-processed point cloud data to a cloud server; the cloud server receives the pre-processed point cloud data of the highway scene transmitted by each edge processor, processes all the point cloud data of the highway scene, including target detection and tracking, parabolic detection and visibility detection, and feeds back the detection results to the edge processor, so that the edge processor visualizes the detection results.

Description

Technical Field

[0001] This invention relates to the field of intelligent transportation technology, and in particular to a highway scene event monitoring system and method. Background Technology

[0002] With the development of autonomous driving technology, LiDAR technology is becoming increasingly important because it can provide high-resolution, accurate, and comprehensive information, and it complements other sensors such as radar and cameras. LiDAR occupies an increasingly important position among sensor devices due to its unique technology, providing reliable, efficient, and high-precision 3D data. Traditional vision sensors, such as monocular cameras, are easily affected by environmental interference such as light and weather, leading to errors in capturing traffic scene features. However, LiDAR has extremely strong resistance to environmental interference and can accurately measure the relative distance between the edges of objects in the field of view and the device. This contour information forms a point cloud and creates a 3D environmental map with centimeter-level accuracy, thus improving measurement precision. The application of LiDAR in autonomous driving encompasses various functions such as localization, drivable area detection, lane marking detection, obstacle detection, dynamic object tracking, and obstacle classification and recognition.

[0003] In existing technologies, for vehicle target detection, traditional visual sensors, such as monocular cameras, can collect massive amounts of detailed information at close range. For vehicles in images, features such as edges, shape, symmetry, shadows, and position can be well identified and distinguished. Therefore, vehicle detection can be performed using one or several of these features in combination. This method has high accuracy and a relatively low technical threshold. However, monocular cameras are highly dependent on the environment; changes in the environment can significantly affect the camera's perception and detection accuracy, especially in strong light or rainy / snowy weather, where detection performance is extremely poor, requiring complex calculations to ensure accuracy. Regarding projectile detection, projectiles are also a significant factor affecting highway traffic safety. Current highway scene monitoring methods do not address projectile detection, and obstacle avoidance methods based on obstacle detection, both domestically and internationally, are currently insufficient to avoid such sudden and short-response events. In terms of visibility detection, the commonly used method is the image-based dark channel prior algorithm, which has a good defogging effect. However, under other severe weather conditions with poor visibility, such as rain and snow, the dark channel prior method is not as effective. In addition, its detection method and processing are relatively complicated and the technical threshold is also high. Summary of the Invention

[0004] In view of this, the present invention proposes a highway scene event monitoring system and method, which can realize target vehicle detection and tracking, object throwing from vehicle windows detection, vehicle speed measurement and visibility detection in highway scenes.

[0005] The first aspect of this invention provides a highway scene event monitoring system, comprising: at least one lidar, each lidar mounted on a target vehicle in a highway scene, for collecting point cloud data of the highway scene in the area where the target vehicle is traveling; at least one edge processor, each edge processor coupled to a corresponding lidar, for receiving the point cloud data of the highway scene collected by the corresponding lidar, preprocessing the point cloud data of the highway scene, and transmitting the preprocessed point cloud data to a cloud server; and a cloud server coupled to the edge processor, for receiving the preprocessed point cloud data of the highway scene transmitted by each edge processor, processing all the point cloud data of the highway scene, determining the point cloud data of all target vehicles, and generating a 3D bounding box for each target vehicle.

[0006] Furthermore, the cloud server processes all point cloud data of the highway scene to determine the point cloud data of all target vehicles and generate a 3D bounding box for each target vehicle, including: acquiring preprocessed point cloud data of the highway scene from each edge server; clustering and segmenting all acquired point cloud data of the highway scene, dividing the point cloud data into different point cloud sets based on density or geometric shape; extracting features of the target vehicle point clouds from different point cloud sets to determine the point cloud data of all target vehicles; generating a 3D preselected bounding box for each target vehicle in the highway scene based on the point cloud data of all target vehicles; calculating the center point of the 3D preselected bounding box for each target vehicle, standardizing and adjusting the position of the 3D preselected bounding box for each target vehicle, obtaining the standardized center point coordinates of each target vehicle, and generating multiple high-confidence 3D bounding boxes for each target vehicle; removing low-confidence 3D bounding boxes and overlapping boxes based on the ratio of the intersection and union of the target vehicle's true bounding box and the generated 3D bounding boxes, and generating the 3D bounding box for each target vehicle.

[0007] Furthermore, the cloud server is also used to: perform conditional filtering on all point cloud data of the highway scene to obtain point cloud data of the area in front of all target vehicles; and detect objects thrown from vehicle windows in the highway scene based on the point cloud data of the area in front of all target vehicles.

[0008] Furthermore, the cloud server is further used to: acquire point cloud data of all target objects in the preprocessed highway scene, extract the reflection intensity data of the point cloud of all target objects, and record the reflection intensity data of target objects at different locations; the target objects include target vehicles and traffic information signs; preprocess all reflection intensity data of each target object; perform angle normalization correction on all reflection intensity data of each preprocessed target object; construct an echo intensity calculation formula; calculate the atmospheric attenuation coefficient corresponding to all target objects based on the corrected reflection intensity data of each target object and the echo intensity calculation formula; calculate the average value of the atmospheric attenuation coefficient corresponding to all target objects to obtain the atmospheric attenuation coefficient of the highway scene; and calculate the atmospheric visibility information of the highway scene under different weather conditions based on the atmospheric attenuation coefficient of the highway scene.

[0009] A second aspect of the present invention provides a method for monitoring events in a highway scene. The method includes: a lidar acquiring point cloud data of a highway scene in the driving area of ​​a target vehicle; an edge processor receiving the point cloud data of the highway scene acquired by the lidar, preprocessing the point cloud data of the highway scene, and transmitting the preprocessed point cloud data to a cloud server; the cloud server processing all the preprocessed point cloud data of the highway scene to determine the point cloud data of all target vehicles and generating a 3D bounding box for each target vehicle.

[0010] Furthermore, the method also includes: the cloud server performing conditional filtering on all point cloud data of the highway scene to obtain point cloud data of the area in front of all target vehicles; clustering and segmenting the point cloud data of the area in front of all target vehicles to obtain multiple different point cloud sets; extracting the geometric features of the target objects in the multiple different point cloud sets to obtain point cloud data of all target objects; obtaining the three-dimensional coordinate data of the point clouds in the point cloud data of all target objects, and calculating the centroid coordinates of the point clouds of all target objects; calculating the distance between each target object and the corresponding target vehicle based on the centroid coordinates of each target object; detecting target objects with abrupt distance changes within a set time period, and calculating the trajectory of the target object in the vertical direction; constructing a free fall trajectory equation; fitting the trajectory of the target object with abrupt distance changes detected within the set time period to the free fall trajectory equation, calculating the variance and covariance of the trajectory points and free fall points of the target object, and determining whether the variance and covariance are within a set threshold range. If they are, the target object is determined to be a projectile thrown from a vehicle window.

[0011] Furthermore, the method also includes: the cloud server acquiring point cloud data of all target objects in the preprocessed highway scene, extracting reflection intensity data of the point clouds of all target objects, and recording the reflection intensity data of target objects at different locations; the target objects include target vehicles and traffic information signs; preprocessing all reflection intensity data of each target object; performing angle normalization correction on all reflection intensity data of each preprocessed target object; constructing an echo intensity calculation formula; and calculating the atmospheric attenuation coefficient corresponding to all target objects based on the corrected reflection intensity data of each target object and the echo intensity calculation formula.

[0012] The average atmospheric attenuation coefficient for all target objects is calculated to obtain the atmospheric attenuation coefficient for the highway scene. Based on the atmospheric attenuation coefficient for the highway scene, atmospheric visibility information for the highway scene under different weather conditions is calculated.

[0013] The aforementioned highway scene event monitoring system and method uses an array-type lidar to collect 3D point cloud data of highway scenes. Based on the basic 3D point cloud data, weather visibility detection and filtering are added to accurately extract the contour information of target vehicles and effectively prevent visual interference such as rain and fog. In addition, deep learning target detection methods are used to detect and track vehicles, which can accurately extract the feature information of cars, and the accuracy is higher than that of traditional target tracking methods. At the same time, a parabolic detection algorithm is added, which can detect some dangerous vehicles throwing objects in traffic scenes and avoid them, and can further predict sudden events such as landslides and rockfalls. Attached Figure Description

[0014] For illustrative and not limiting purposes, the invention will now be described with reference to preferred embodiments thereof, particularly the accompanying drawings, in which:

[0015] Figure 1 This is a schematic diagram of the structure of a highway scene event monitoring system provided in an embodiment of the present invention;

[0016] Figure 2 This is a flowchart of a highway scene event monitoring method provided in another embodiment of the present invention;

[0017] Figure 3 This is a flowchart of target detection provided in another embodiment of the present invention;

[0018] Figure 4 This is a flowchart of parabolic detection provided in another embodiment of the present invention;

[0019] Figure 5 This is a flowchart of visibility detection provided in another embodiment of the present invention. Detailed Implementation

[0020] To better understand the above-mentioned objects, features, and advantages of the present invention, the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that, unless otherwise specified, the embodiments of the present invention and the features thereof can be combined with each other.

[0021] Numerous specific details are set forth in the following description to provide a thorough understanding of the invention. The described embodiments are merely some, not all, of the embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.

[0022] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used herein in the description of the invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention.

[0023] Figure 1 This is a schematic diagram of a highway scene event monitoring system according to an embodiment of the present invention. The highway scene event monitoring system provided in this embodiment will be described in detail below with reference to the accompanying drawings.

[0024] This highway scene event monitoring system is used to collect and process point cloud data of highway scenes, enabling target vehicle detection and tracking, projectile detection, and visibility detection in highway scenes. Please refer to [link / reference]. Figure 1 The highway scene event monitoring system includes:

[0025] At least one lidar 100, each lidar 100 is mounted on a target vehicle in a highway scene, and is used to collect point cloud data of the highway scene in the area where the target vehicle is traveling.

[0026] At least one edge processor 200 is provided, each edge processor 200 being coupled to a corresponding LiDAR 100 and a display module 400. It is used to receive point cloud data of a highway scene collected by the corresponding LiDAR 100, preprocess and convert the point cloud data of the highway scene, and transmit the preprocessed and converted point cloud data to a cloud server 300. It is also used to receive the detection results fed back by the cloud server 300, perform visualization processing on the detection results, and transmit the visualization processing results to the corresponding display module 400.

[0027] The cloud server 300 is coupled to the edge processor 200 to receive the pre-processed point cloud data of the highway scene transmitted by each edge processor 200, process all the point cloud data of the highway scene, including target detection and tracking, parabolic detection and visibility detection, and feed back the detection results to the corresponding edge processor 200.

[0028] At least one display module 400, each display module 400 being coupled to a corresponding edge processor 200, is used to receive and display the visualization processing results fed back by the corresponding edge processor 200.

[0029] In some embodiments, the lidar 100 is an area-array 3D lidar mounted on a target vehicle in a highway scene. When the target vehicle passes through a drivable area of ​​the highway scene, the lidar 100 collects point cloud data of the highway scene. This point cloud data includes the 3D coordinates of the point cloud and its reflection intensity data. The lidar 100 is communicatively connected to the edge processor 200 and can send point cloud data to the edge processor 200 in real time.

[0030] Since there are several target vehicles in the highway scene, each target vehicle is equipped with a corresponding LiDAR 100. Point cloud data of the highway scene passing through the drivable area is collected through the LiDAR 100 on each target vehicle.

[0031] In some embodiments, the edge processor 200 primarily consists of a high-performance motherboard containing CPU and GPU computing chips. The edge processor 200's casing has multiple screw holes for mounting brackets at various angles, and it can also be integrated into a vehicle system.

[0032] The edge processor 200 is primarily used to receive and preprocess point cloud data of the highway scene collected by the LiDAR 100 in real time, convert the point cloud data file format, and send the preprocessed point cloud data of the highway scene to the cloud server 300 in real time. The cloud server 300 then processes the point cloud data, including target detection and tracking, parabolic detection, and visibility detection. The cloud server 300 also receives the preprocessed point cloud data to create a dataset, trains a model using the training set, and sends the trained model to the corresponding edge processor 200. The cloud server 300 also feeds back the processed detection results to the corresponding edge processor 200. The edge processor 200 also receives the detection results processed by the cloud server 300, visualizes them, and feeds back the visualization results to the driver in real time through the display module 400 on the target vehicle.

[0033] In this embodiment, the point cloud data collected by the lidar 100 is preprocessed by the edge processor 200, including ground filtering and Gaussian noise filtering, to remove redundant point cloud information such as outliers and anomalies, thereby improving the accuracy of target detection.

[0034] Because the cloud server 300 has different requirements for the format of point cloud data, the edge processor 200 also converts the preprocessed PCD format point cloud data into the format required by the cloud server 300, such as PLY format, to remove redundant point cloud information and facilitate direct transmission to the cloud server 300 for processing.

[0035] In some embodiments, the cloud server 300 is composed of a high-performance industrial control unit running Ubuntu 18.04. It is primarily responsible for processing the large volumes of point cloud data transmitted from all edge processors 200, and further processing the point cloud data using its onboard object detection and tracking, vehicle speed measurement, parabolic object detection, and visibility detection algorithms. The cloud server 300 contains the Kitti point cloud dataset and corresponding .txt format tags, suitable for the aforementioned algorithms.

[0036] The cloud server 300 is used to process large batches of point cloud data. It is mainly responsible for training the dataset and processing the point cloud data, including object detection, object tracking, parabolic detection and visibility detection. It receives the preprocessed data transmitted back from each edge processor 200 in real time and sends the detection results to the edge processor 200.

[0037] In some embodiments, the edge processor 200 acquires point cloud data of the highway scene during the target vehicle's journey collected by the corresponding LiDAR 100 and performs preprocessing. This preprocessing includes ground filtering and Gaussian noise filtering. Ground filtering identifies which data points in the disordered and irregular 3D discrete point cloud originate from the ground surface and which originate from ground features. Ground filtering removes ground point cloud data from the background, preventing misselection when calculating the target vehicle's 3D bounding box. Gaussian noise filtering is a signal filter used for signal smoothing and eliminating Gaussian noise. Gaussian filtering removes outlier and abnormal point sets from the point cloud data, reducing computational workload and improving operational efficiency.

[0038] In some embodiments, the cloud server 300 receives preprocessed point cloud data of the highway scene transmitted by each edge processor 200, processes all point cloud data of the highway scene, determines the point cloud data of all target vehicles, and generates a 3D bounding box for each target vehicle to achieve the detection of target vehicles in the highway scene.

[0039] Specifically, the cloud server 300 processes the point cloud data of all highway scenes, determines the point cloud data of all target vehicles, and generates the 3D bounding box of each target vehicle. The specific implementation method is as follows:

[0040] Acquire 200 pre-processed point cloud data of the highway scene from each edge server;

[0041] Clustering and segmentation are performed on all point cloud data of the acquired highway scene, and the point cloud data is divided into different point cloud sets based on density or geometric shape.

[0042] Extract the features of the target vehicle point cloud from different point cloud sets to determine the point cloud data of all target vehicles and the point cloud data of non-target vehicles.

[0043] Based on the point cloud data of all target vehicles, generate a 3D preselection box for each target vehicle within the field of view of the LiDAR in the highway scene;

[0044] Calculate the center point of the 3D preselection box for each target vehicle, standardize and adjust the position of the 3D preselection box for each target vehicle, obtain the normalized center point coordinates of each target vehicle, and generate multiple high-confidence 3D bounding boxes for each target vehicle.

[0045] Based on the Intersection over Union (IoU) ratio, low-confidence 3D bounding boxes and overlapping boxes are removed to generate a unique 3D bounding box for the target vehicle.

[0046] This embodiment targets the highway scenario, reducing the confidence level of the vehicle point cloud data of the interference items, and only using it to capture the point cloud data of the target vehicle to generate the 3D bounding box of the target vehicle.

[0047] In this embodiment, the cloud server 300 uses the PoinNet++ network structure to extract the features of each point cloud dataset. The features of the point cloud include the normal vector direction, geometric features, local features, and global features. The geometric features include three-dimensional dimensions, etc.; the local features involve treating a point in the point cloud and its neighborhood as an ellipsoid and determining the direction of the shortest minor axis of the ellipsoid; the global features involve projecting the entire point cloud onto a plane and calculating the projection variance and covariance, etc.

[0048] In this embodiment, the Intersection over Union (IoU) is the ratio of the intersection and union of the true bounding box of the target vehicle and the predicted 3D bounding box. The cloud server 300 removes low-confidence 3D bounding boxes and overlapping boxes based on the IoU value, generating a unique 3D bounding box for the vehicle.

[0049] The cloud server 300 is also further used for target tracking based on the target vehicle's 3D bounding box.

[0050] This embodiment uses the AB3DMOT network framework and Kalman filter algorithm for target tracking. After training the deep learning network, a usable target detection model is obtained. The model is then imported into AB3DMOT to convert static point cloud data into dynamic point cloud data and perform target tracking.

[0051] Specifically, the cloud server 300 performs target tracking based on the target vehicle's 3D bounding box as follows:

[0052] Capture the 3D bounding box of the target vehicle;

[0053] The 3D bounding box of the target vehicle in the previous frame is predicted to the bounding box of the target vehicle in the current frame using a 3D Kalman filter.

[0054] Match the predicted bounding box of the target vehicle with the current bounding box and calculate the loss function;

[0055] The bounding box of the target vehicle is updated based on the matching results using a 3D Kalman filter, and the prediction algorithm is optimized to obtain a high-confidence tracking result.

[0056] Remove the predicted trajectory and 3D bounding box of the target vehicle that has lost sight of the target vehicle, and update the results in real time.

[0057] The cloud server 300 is also used to count the number of 3D bounding boxes of vehicles that have lost their field of vision within a certain period of time, and to obtain the traffic flow during that period of time.

[0058] This embodiment can obtain the traffic flow within any time period by counting the number of 3D bounding boxes of vehicles that have lost their field of vision within any time period, thus completing the traffic flow monitoring work within any time period.

[0059] In this embodiment, the cloud server 300 performs target detection on point cloud data and obtains the 3D bounding boxes of target vehicles within the field of view of the LiDAR in the highway scene. Based on the obtained 3D bounding boxes and Kalman filters, the cloud server 300 can dynamically optimize the tracking results of target vehicles frame by frame while moving, realize the real-time generation of each new 3D bounding box of the target vehicle, and thus realize the tracking of the target vehicle. It can complete the work of traffic flow monitoring, assisted driving, and avoidance of some traffic accidents.

[0060] In some embodiments, the cloud server 300 is further configured to: perform conditional filtering on all point cloud data of the highway scene to obtain point cloud data of the area in front of all target vehicles; and detect objects thrown from vehicle windows in the highway scene based on the point cloud data of the area in front of all target vehicles.

[0061] The point cloud data collected by the LiDAR 100 includes the three-dimensional coordinate information and reflection intensity information of the point cloud. Based on the three-dimensional coordinate information of the point cloud, the cloud server 300 can calculate the geometric distance of the target point cloud relative to the LiDAR 100, thereby enabling parabolic detection based on the point cloud data. In addition to the relative geometric distance information, the effective distance of parabolic detection and the effective size of the target object also depend on the angular resolution of the LiDAR 100 and the frame rate of point cloud acquisition. For the target object, ignoring its horizontal trajectory, its vertical trajectory can be approximated as free fall. Therefore, when performing parabolic detection, abnormal point clouds with abrupt changes in geometric distance between consecutive frames are first detected. The trajectory of the abnormal point cloud in the vertical direction is calculated, and the trajectory detected in a short period of time is fitted with the free fall motion equation. If there is no error or the error is small, it can be determined as a parabolic object from a car window.

[0062] Specifically, the cloud server 300 detects objects thrown from vehicle windows in highway scenarios based on point cloud data of the area in front of all target vehicles, as follows:

[0063] Clustering and segmentation are performed on the point cloud data of the area in front of all target vehicles to obtain multiple different point cloud sets;

[0064] Extract the geometric features of the target objects from multiple different point cloud sets to obtain the point cloud data of all target objects;

[0065] Obtain the 3D coordinate data of the point cloud data of all target detection objects, and calculate the centroid coordinates of the point cloud of all target detection objects;

[0066] Based on the centroid coordinates of each target object, calculate the distance between each target object and the corresponding target vehicle;

[0067] Detect target objects that exhibit sudden changes in distance within a set time period and calculate the trajectory of the target object in the vertical direction;

[0068] Construct the equation of free fall trajectory;

[0069] The trajectory of a target object whose distance changes abruptly within a set time period is fitted with the equation of free fall motion, and the target object with no error or small error is identified as a projectile thrown from a car window.

[0070] In this embodiment, the cloud server 300 obtains point cloud data of the area in front of all target vehicles by conditionally filtering the point cloud based on all point cloud data of the highway scene acquired by the lidar 100. Then, it performs clustering and segmentation on the point cloud data of the area in front of all target vehicles and obtains parabolic point cloud data, i.e., the point cloud set with the smallest geometric scale, by using the geometric features of the segmented point cloud set.

[0071] In this embodiment, the method for constructing the trajectory equation of free fall is as follows:

[0072] Establish a three-dimensional Cartesian coordinate system with the lidar as the origin. Let the direction perpendicular to the ground be the z-axis, the direction the lidar is pointing be the y-axis, and the direction perpendicular to the lidar on the same horizontal plane be the x-axis. Let time be t. Ignore the motion of the parabola along the x and y axes, and only consider the coordinate transformation in the vertical direction. Then, let the coordinates of the center of mass of the parabola be (x, y, z). i Given an initial coordinate system (x, y, z0) and an initial time of t0, the trajectory of the free fall motion is as follows:

[0073]

[0074] In this embodiment, the cloud server 300 will fit the trajectory of the target object whose distance changes suddenly within a set time period with the free fall motion equation, and calculate the variance and covariance of the trajectory points and free fall points of the target object. When the variance and covariance of the target object are within a certain threshold range, it indicates that the target object has no error or the error is small, and the target object can be identified as a projectile thrown from a car window.

[0075] In this embodiment, the lidar 100 operates at a frame rate of 7 frames per second and an angular resolution of 0.12°, enabling it to detect projectiles the size of an aluminum can at a distance of 50 meters. 50 meters is precisely the safest driving distance in a highway scenario. In addition to projectile detection, the cloud server 300 can also detect obstacles.

[0076] In some embodiments, the cloud server 300 is further configured to: detect road obstacles in a highway scene based on point cloud data of the area in front of all target vehicles.

[0077] When there are fixed obstacles on the road surface, such as roadblocks or potholes, the cloud server 300 detects road obstacles in the highway scene based on point cloud data of the area in front of all target vehicles.

[0078] Based on the three-dimensional coordinate data of the point cloud, the vertical coordinates of the ground point cloud should fluctuate within a general range. When a point cloud set with a vertical coordinate significantly lower or higher than the ground point cloud coordinate is found, it is marked as an abnormal point set. When the abnormal point set gets closer as the vehicle moves, it is determined to be a road obstacle and is avoided.

[0079] Specifically, the cloud server 300 detects road obstacles in a highway scene based on point cloud data of the area in front of all target vehicles in the following way:

[0080] Clustering and segmentation are performed on the point cloud data of the area in front of all target vehicles to obtain multiple different point cloud sets;

[0081] Extract the geometric features of all target objects from multiple different point cloud sets to obtain point cloud data of all target objects;

[0082] Based on the three-dimensional coordinate data of the point cloud data of all target detection objects, the centroid coordinates of the point cloud of all target detection objects are obtained.

[0083] Based on the centroid coordinates of each target object, the distance between each target object and the corresponding target vehicle is calculated;

[0084] A preset danger distance threshold is set. Targets within the danger distance threshold range are detected, identified as road obstacles, and avoided.

[0085] This embodiment predetermines a danger distance. When a target detection point cloud of a certain size is detected within the danger distance, that is, a point cloud whose z-direction coordinates are different from the road surface coordinates, and the target detection point cloud is different from the target vehicle point cloud, as its distance continuously approaches the target vehicle, it is determined to be a foreign object on the road and is avoided.

[0086] In some embodiments, the cloud server 300 is further used to: detect naturally falling objects in a highway scene based on point cloud data of the area in front of all target vehicles. For example, a rockfall at a highway tunnel entrance. The detection process for a rockfall at a highway tunnel entrance consists of two parts: the falling process and the process after it lands until it comes to rest. The rockfall can be considered as free fall. Because the rockfall is relatively large, the detection distance is also greater. Its trajectory is calculated, and the trajectory is fitted with radar detection data. A high confidence level indicates it is a rockfall. After the rockfall comes to rest, the aforementioned road obstacle detection algorithm is used for further detection and avoidance.

[0087] Specifically, the cloud server 300 detects naturally falling objects in a highway scene based on point cloud data of the area in front of all target vehicles, as follows:

[0088] Clustering and segmentation are performed on the point cloud data of the area in front of all target vehicles to obtain multiple different point cloud sets;

[0089] Extract the geometric features of the target objects from multiple different point cloud sets to obtain the point cloud data of all target objects;

[0090] Based on the three-dimensional coordinate data of the point cloud data of all target detection objects, the centroid coordinates of the point cloud of all target detection objects are obtained.

[0091] Based on the three-dimensional coordinate information of each target object, the distance between each target object and the corresponding target vehicle is calculated;

[0092] Detect target objects that exhibit sudden changes in distance within a set time period and calculate the trajectory of the target object in the vertical direction;

[0093] The motion trajectory of the target object is fitted with the actual detection data of the lidar. If the confidence level is high, it can be judged as a falling rock.

[0094] The data detected by the lidar is the true three-dimensional coordinates of the naturally falling object. The trajectory of the target object is the calculated free-fall trajectory. By fitting the free-fall trajectory equation mentioned above, the calculated z-axis is determined. i and the actual z obtained i The difference, and multiple z i If both variances between the differences are within a set threshold, the confidence level is considered high.

[0095] In some embodiments, the cloud server 300 is further configured to calculate the speed of all target vehicles based on the point cloud data of all target vehicles.

[0096] Specifically, the cloud server 300 calculates the speed of all target vehicles based on the point cloud data of all target vehicles in the following way:

[0097] A coordinate system is established with the location of the lidar as the origin. The coordinates of the center point of the target vehicle are calculated based on the three-dimensional coordinate data of the point cloud data of the target vehicle. The displacement distance of the center point of the target vehicle in two consecutive frames is calculated, and the speed of the target vehicle when traveling in a straight line is calculated in combination with the lidar frame rate.

[0098] Let the time of each frame be t, the coordinates of the center point of the target vehicle in the previous frame be (x1, y1, z1), the coordinates of the center point of the target vehicle in the next frame be (x2, y2, z2), and the average speed of the target vehicle between the two frames be u. Then the calculation formula is:

[0099]

[0100] Meanwhile, when the target vehicle is turning, its trajectory can be approximated as a circle. By connecting the coordinates of the center point of the target vehicle in three consecutive frames, the coordinates of the center of its trajectory can be fitted, and the angular velocity of the target vehicle when turning can be calculated by combining its translational velocity.

[0101] In this embodiment, the cloud server 300 processes all point cloud data of the highway scene to realize the detection of objects thrown from vehicle windows, road obstacles, natural falling objects, and vehicle speed in the highway scene.

[0102] Visibility refers to the maximum distance at which a person with normal vision can distinguish a target object from its background. In other words, during the day, against a background of the sky near the horizon, one can clearly see the outline of a dark object on the ground with a viewing angle greater than 20 degrees and identify what it is. At night, one can clearly see the luminous point of a target light. The degree of visibility is mainly determined by two factors: ① the difference in brightness between the target object and the background supporting it; the greater (smaller) the difference, the greater (smaller) the visible distance, but this difference in brightness usually does not change much; ② atmospheric transparency.

[0103] This embodiment uses the reflection intensity information carried by the point cloud data obtained by the lidar 100 scanning the target object to calculate visibility.

[0104] Currently, visibility measurements are mainly performed using transmission and forward scattering methods. Both methods assume a uniform meteorological environment around the installation location and use a single-point measurement to represent the visibility over a large area. Both methods have limitations; they can only obtain visibility in a fixed horizontal direction and cannot make accurate and timely judgments and provide feedback on weather phenomena that affect navigation safety.

[0105] In some embodiments, the cloud server 300 is further configured to perform visibility detection on the highway scene based on point cloud data of all target vehicles and traffic information signs.

[0106] The reflection intensity value collected by the lidar 100 can be regarded as a discrete integer value obtained after certain signal processing of the echo light power of the target vehicle received by the lidar. In this embodiment, based on the reflection intensity value received by the lidar and the system parameters of the lidar, various parameters and influencing factors required for the weather visibility method are derived. The echo intensity calculation formula is derived based on its echo equation, and then the echo intensity calculation formula is further fitted based on the obtained reflection intensity data to obtain the required parameter, namely the atmospheric attenuation coefficient.

[0107] The echo intensity of a lidar system can be primarily characterized by the lidar equation. According to the lidar equation, the echo power of a lidar system is determined by the laser's emission power, the parameters of the transmitting optical system, the atmospheric laser transmission characteristics, the parameters of the receiving optical system, and the target's reflection characteristics.

[0108] The lidar equation is as follows:

[0109]

[0110] Among them, P r It is the echo power received by the lidar, P LR is the transmitted optical power of the lidar, R is the target reflectivity, and η0 is the transmission and reception efficiency of the lidar optical system. e is the ratio of the receiver area to the emitted beam spot area. -2αS For the ambient light path Lambertian attenuation model, θ represents the incident angle of the target point relative to the radar.

[0111] The reflection intensity value of a 3D scanning lidar can be viewed as an intensity value related to the received optical power, returned to the user after the lidar has processed the received target echo light power through a series of comprehensive processes, including photoelectric signal conversion and amplification, subsequent correction processing, and digital quantization. This intensity value is typically a discrete integer value, similar to grayscale pixel values ​​in an image. In this embodiment, a function h is used to characterize the subsequent signal processing of the lidar, and the lidar's intensity return value is denoted as I, and the target reflected echo intensity is denoted as Pr.

[0112] The expression for the echo intensity of the lidar is:

[0113] I = h(p) (3)

[0114] Different LiDAR models may have different calculation methods, therefore, it is necessary to establish a formula for calculating echo intensity through data modeling. The specific modeling approach involves first analyzing the real-world factors affecting the magnitude of echo power or echo intensity based on relevant theories; then analyzing all influencing factors one by one using experimental methods with controlled variables; and finally establishing the echo intensity calculation formula through data fitting, with the correlation coefficient of the fitted data characterizing the model's reliability. The final established echo intensity calculation formula is shown below:

[0115] I=h(p)=Aexp(-2αs)cosθ (4)

[0116] Among them, coefficient A contains all the constant terms in equation (2), s is the distance, and α is the atmospheric attenuation coefficient to be obtained.

[0117] Specifically, the cloud server 300 performs visibility detection on highway scenes based on point cloud data of all target vehicles and traffic information signs as follows:

[0118] Acquire the point cloud data of all target objects in the entire highway scene after preprocessing, and extract the reflection intensity data of the point cloud of all target objects, and record the reflection intensity data of target objects at different locations; the target objects include target vehicles and traffic information signs.

[0119] Preprocess all reflection intensity data for each target object;

[0120] Angle normalization correction is applied to all reflection intensity data of each target object after preprocessing.

[0121] An echo intensity calculation formula is constructed, as shown in formula (3);

[0122] Based on all the reflection intensity data and echo intensity calculation formula of each target object after correction, calculate the atmospheric attenuation coefficient α corresponding to all target objects;

[0123] The atmospheric attenuation coefficient α corresponding to all target objects is calculated to obtain the atmospheric attenuation coefficient of the highway scene.

[0124] Based on the atmospheric attenuation coefficient of the highway scene, atmospheric visibility information of the highway scene under different weather conditions is calculated.

[0125] In this embodiment, the cloud server 300 preprocesses all reflection intensity data of each target vehicle, including median filtering and SOR spatial filtering, thereby reducing measurement errors.

[0126] In this embodiment, all reflection intensity data for each target vehicle are normalized by angle, that is, the reflection intensity data is corrected to the intensity information at a 0° incident angle, thereby normalizing and mapping the echo intensity to the same reference frame, thus eliminating the influence of other variables on data fitting.

[0127] In the same highway scenario, the reflection intensity data of point clouds of multiple moving target vehicles can be recorded. Multiple target vehicles can generate multiple atmospheric attenuation coefficients α. The multiple target vehicles allow more point cloud reflection intensity data to be incorporated into the calculation, and averaging multiple atmospheric attenuation coefficients α can ensure a more accurate atmospheric attenuation coefficient.

[0128] In this embodiment, the cloud server 300 calculates the atmospheric visibility information of the highway scene under different weather conditions based on the atmospheric attenuation coefficient of the highway scene as follows:

[0129] V=3·912 / α(5)

[0130] Where α is the atmospheric attenuation coefficient; V is the atmospheric visibility information.

[0131] The aforementioned highway scene event monitoring system can detect and track target vehicles, detect objects thrown from vehicle windows, measure vehicle speed, and detect visibility in highway scenes.

[0132] Figure 2 This is a flowchart of a highway scene event monitoring method according to another embodiment of the present invention. The highway scene event monitoring system provided in this embodiment will be described in detail below with reference to the accompanying drawings.

[0133] Please see Figure 2The highway scene event monitoring method includes the following steps:

[0134] S100, the LiDAR 100, collects point cloud data of highway scenes.

[0135] The lidar 100 is mounted on a target vehicle in a highway scene to collect point cloud data of the highway scene in the area where the target vehicle is traveling.

[0136] S200, the edge processor 200 receives point cloud data of the highway scene collected by LiDAR, preprocesses and converts the point cloud data of the highway scene, and transmits the preprocessed and converted point cloud data to the cloud server 300.

[0137] In this embodiment, since there are multiple target vehicles in the highway scenario, each target vehicle is equipped with a LiDAR 100. The edge processor 200 is coupled to a corresponding LiDAR 100 to receive and preprocess the point cloud data collected by the LiDAR 100.

[0138] S300, cloud server 300 receives preprocessed point cloud data of highway scene transmitted by edge processor 200, processes all point cloud data of highway scene, including target detection and tracking, parabolic detection and visibility detection, and feeds back the detection results to the corresponding edge processor 200.

[0139] S400, the edge processor 200 receives the detection results fed back by the cloud server 300, performs visualization processing on the detection results, and transmits the visualization processing results to the corresponding display module 400 for display.

[0140] In this embodiment, the cloud server 300 is coupled to at least one edge processor 200 to receive preprocessed point cloud data from the edge processor 200 and process the point cloud data to achieve target vehicle detection and tracking, object throwing from vehicle windows detection, and visibility detection in highway scenarios.

[0141] In some embodiments, the edge processor 200 acquires point cloud data of the highway scene during the journey of the target vehicle collected by the corresponding lidar 100 and performs preprocessing. The preprocessing includes ground filtering, Gaussian noise filtering, etc., to remove redundant point cloud information such as outliers and anomalies, thereby improving the accuracy of target detection.

[0142] In some embodiments, step S300, where the cloud server 300 processes all point cloud data of the highway scene, includes:

[0143] S310, the cloud server 300 receives the preprocessed point cloud data of the highway scene transmitted by the edge processor 200, processes all the point cloud data of the highway scene, determines the point cloud data of all target vehicles, and generates a 3D bounding box for each target vehicle to realize the detection of target vehicles in the highway scene.

[0144] Figure 3 This is a flowchart of target detection provided in another embodiment of the present invention. Please refer to [link / reference]. Figure 3 The cloud server 300 processes the point cloud data of all highway scenes, determines the point cloud data of all target vehicles, and generates the 3D bounding box of each target vehicle. The specific implementation method is as follows:

[0145] S311, acquire point cloud data of the highway scene after preprocessing by edge server 200;

[0146] S312, cluster and segment all point cloud data of the acquired highway scene, and divide the point cloud data into different point cloud sets based on density or geometric shape;

[0147] S313, extract the features of the target vehicle point cloud from different point cloud sets, and determine the point cloud data of all target vehicles and the point cloud data of non-target vehicles.

[0148] S314, Based on the point cloud data of all target vehicles, generate a 3D pre-selection box for each target vehicle within the field of view of the LiDAR in the highway scene;

[0149] S315, calculate the center point of the 3D preselection box for each target vehicle, standardize and adjust the position of the 3D preselection box for each target vehicle, obtain the standardized center point coordinates, and generate multiple high-confidence vehicle 3D bounding boxes for each target vehicle.

[0150] S316, based on the Intersection over Union (IoU) ratio, remove low-confidence 3D bounding boxes and overlapping boxes to generate a unique 3D bounding box for the target vehicle.

[0151] This embodiment targets the highway scenario, reducing the confidence level of the vehicle point cloud data of the interference items, and only using it to capture the point cloud data of the target vehicle to generate the 3D bounding box of the target vehicle.

[0152] In some embodiments, the method further includes:

[0153] S317 performs target tracking based on the target vehicle's 3D bounding box.

[0154] This embodiment uses the AB3DMOT network framework and Kalman filter algorithm for target tracking. After training the deep learning network, a usable target detection model is obtained. The model is then imported into AB3DMOT to convert static point cloud data into dynamic point cloud data and perform target tracking.

[0155] Specifically, the cloud server 300 performs target tracking based on the target vehicle's 3D bounding box as follows:

[0156] (1) Capture the 3D bounding box of the target vehicle;

[0157] (2) Predict the state of the previous frame to the current frame using a 3D Kalman filter;

[0158] (3) Match the predicted bounding box with the current bounding box and calculate the loss function;

[0159] (4) The bounding box of the target vehicle is updated based on the matching result using a 3D Kalman filter, and the prediction algorithm is optimized to obtain a high-confidence tracking result;

[0160] (5) Delete the predicted trajectory and 3D bounding box of the target vehicle that has lost the field of vision, and update the results in real time.

[0161] In some embodiments, the method further includes:

[0162] S318, cloud server 300 counts the number of 3D bounding boxes of vehicles that have lost their field of vision within a certain period of time, and obtains the traffic flow during that period of time.

[0163] This embodiment can obtain the traffic flow within any time period by counting the number of 3D bounding boxes of vehicles that have lost their field of vision within any time period, thus completing the traffic flow monitoring work within any time period.

[0164] In this embodiment, the cloud server 300 performs target detection on point cloud data and obtains the 3D bounding boxes of target vehicles within the field of view of the LiDAR in the highway scene. Based on the obtained 3D bounding boxes and Kalman filters, the cloud server 300 can dynamically optimize the tracking results of target vehicles frame by frame while moving, realize the real-time generation of each new 3D bounding box of the target vehicle, and thus realize the tracking of the target vehicle. It can complete the work of traffic flow monitoring, assisted driving, and avoidance of some traffic accidents.

[0165] In some embodiments, step S300, where the cloud server 300 processes all point cloud data of the highway scene, further includes:

[0166] The S320 cloud server 300 performs conditional filtering on all point cloud data in the highway scene to obtain point cloud data of the area in front of all target vehicles; based on the point cloud data of the area in front of all target vehicles, it detects objects thrown from vehicle windows in the highway scene.

[0167] Figure 4 This is a flowchart of a vehicle window projectile detection method according to another embodiment of the present invention. Please refer to [link / reference]. Figure 4 The cloud server 300 detects objects thrown from vehicle windows in highway scenarios based on point cloud data of the area in front of all target vehicles, as follows:

[0168] S321, cluster and segment the point cloud data of the area in front of all target vehicles to obtain multiple different point cloud sets;

[0169] S322, extract the geometric features of the target objects from multiple different point cloud point sets to obtain the point cloud data of all target objects;

[0170] S323, Obtain the three-dimensional coordinate data of the point cloud in the point cloud data of all target detection objects, and calculate the centroid coordinates of the point cloud of all target detection objects;

[0171] S324, Calculate the distance between each target object and its corresponding target vehicle based on the centroid coordinates of each target object;

[0172] S325, detects target objects that experience sudden changes in distance within a set time period, and calculates the trajectory of the target object in the vertical direction;

[0173] S326, Construct the equation for the trajectory of free fall;

[0174] S327 will fit the trajectory of the target object that is detected to have a sudden change in distance within a set time period with the equation of free fall motion, and determine the target object with no error or small error as a projectile thrown from the car window.

[0175] In this embodiment, the cloud server 300 will fit the trajectory of the target object whose distance changes suddenly within a set time period with the free fall motion equation, and calculate the variance and covariance of the trajectory points and free fall points of the target object. When the variance and covariance of the target object are within a certain threshold range, it indicates that the target object has no error or the error is small, and the target object can be identified as a projectile thrown from a car window.

[0176] In some embodiments, step S300, where the cloud server 300 processes all point cloud data of the highway scene, further includes:

[0177] The S330 and cloud server 300 detect road obstacles in highway scenarios based on point cloud data of the area in front of all target vehicles.

[0178] Based on the three-dimensional coordinate data of the point cloud, the vertical coordinates of the ground point cloud should fluctuate within a general range. When a point cloud set with a vertical coordinate significantly lower or higher than the ground point cloud coordinate is found, it is marked as an abnormal point set. When the abnormal point set gets closer as the vehicle moves, it is determined to be a road obstacle and is avoided.

[0179] Specifically, the cloud server 300 detects road obstacles in a highway scene based on point cloud data of the area in front of all target vehicles in the following way:

[0180] S331, cluster and segment the point cloud data of the area in front of all target vehicles to obtain multiple different point cloud sets;

[0181] S332, extract the geometric features of all target objects from multiple different point cloud point sets to obtain point cloud data of all target objects;

[0182] S333, based on the three-dimensional coordinate data of the point cloud in the point cloud data of all target detection objects, calculate the centroid coordinates of the point cloud of all target detection objects;

[0183] S334, based on the centroid coordinates of each target object, calculate the distance between each target object and the corresponding target vehicle;

[0184] S335, preset danger distance threshold, detect target objects within the danger distance threshold range, identify the target object as a road obstacle, and avoid it.

[0185] This embodiment predetermines a danger distance. When a target detection point cloud of a certain size is detected within the danger distance, that is, a point cloud whose z-direction coordinates are different from the road surface coordinates, and the target detection point cloud is different from the target vehicle point cloud, as its distance continuously approaches the target vehicle, it is determined to be a foreign object on the road and is avoided.

[0186] In some embodiments, step S300, where the cloud server 300 processes all point cloud data of the highway scene, further includes:

[0187] The S340 and cloud server 300 detect naturally falling objects in highway scenarios based on point cloud data of the area in front of all target vehicles.

[0188] Specifically, the cloud server 300 detects naturally falling objects in a highway scene based on point cloud data of the area in front of all target vehicles, as follows:

[0189] S341, cluster and segment the point cloud data of the area in front of all target vehicles to obtain multiple different point cloud sets;

[0190] S342, extract the geometric features of the target objects from multiple different point cloud point sets to obtain the point cloud data of all target objects;

[0191] S343, Based on the three-dimensional coordinate data of the point cloud in the point cloud data of all target detection objects, calculate the centroid coordinates of the point cloud of all target detection objects;

[0192] S344, based on the three-dimensional coordinate information of each target object, calculates the distance between each target object and the corresponding target vehicle;

[0193] S345, detects target objects that experience sudden changes in distance within a set time period, and calculates the trajectory of the target object in the vertical direction;

[0194] S346: Fit the trajectory of the detected target to the lidar detection data; if the confidence level is high, it can be determined to be a falling rock.

[0195] In some embodiments, step S300, where the cloud server 300 processes all point cloud data of the highway scene, further includes:

[0196] The S350 and cloud server 300 calculate the speed of all target vehicles based on the point cloud data of all target vehicles.

[0197] Specifically, the cloud server 300 calculates the speed of all target vehicles based on the point cloud data of all target vehicles in the following way:

[0198] A coordinate system is established with the location of the lidar as the origin. The coordinates of the center point of the target vehicle are calculated based on the three-dimensional coordinate data of the point cloud data of the target vehicle. The displacement distance of the center point of the target vehicle in two consecutive frames is calculated, and the speed of the target vehicle when traveling in a straight line is calculated in combination with the lidar frame rate.

[0199] Meanwhile, when the target vehicle is turning, its trajectory can be approximated as a circle. By connecting the coordinates of the center point of the target vehicle in three consecutive frames, the coordinates of the center of its trajectory can be fitted, and the angular velocity of the target vehicle when turning can be calculated by combining its translational velocity.

[0200] The cloud server 300 processes all point cloud data in the highway scene to achieve detection of objects thrown from vehicle windows, road obstacles, falling objects, and vehicle speed in the highway scene.

[0201] In some embodiments, step S300, where the cloud server 300 processes all point cloud data of the highway scene, further includes:

[0202] The S360 and cloud server 300 are further used to perform visibility detection on highway scenes based on point cloud data of all target vehicles.

[0203] Figure 5 This is a flowchart of visibility detection provided in an embodiment of the present invention. Please refer to [link / reference]. Figure 5 The cloud server 300 performs visibility detection on highway scenes based on point cloud data of all target vehicles in the following specific manner:

[0204] S361, acquire the point cloud data of all target objects in the entire highway scene after preprocessing, extract the reflection intensity data of the point cloud of all target objects, and record the reflection intensity data of the target objects at different locations; the target objects include target vehicles and traffic information signs.

[0205] S362, preprocesses all reflection intensity data for each target object;

[0206] S363 performs angle normalization correction on all reflection intensity data of each target object after preprocessing.

[0207] S364, Construct the echo intensity calculation formula, which is as follows:

[0208] I=h(p)=Aexp(-2αs)cosθ;

[0209] S365, based on all the reflection intensity data and echo intensity calculation formula of each target object after correction, calculate the atmospheric attenuation coefficient α corresponding to all target objects;

[0210] S366, calculate the average value of the atmospheric attenuation coefficient α for all target objects to obtain the atmospheric attenuation coefficient of the highway scene;

[0211] S367 calculates atmospheric visibility information for highway scenes under different weather conditions based on the atmospheric attenuation coefficient of the highway scene.

[0212] In the same highway scenario, the reflection intensity data of point clouds of multiple moving target vehicles can be recorded. Multiple target vehicles can generate multiple atmospheric attenuation coefficients α. The multiple target vehicles allow more point cloud reflection intensity data to be incorporated into the calculation, and averaging multiple atmospheric attenuation coefficients α can ensure a more accurate atmospheric attenuation coefficient.

[0213] In this embodiment, the cloud server 300 calculates the atmospheric visibility information of the highway scene under different weather conditions based on the atmospheric attenuation coefficient of the highway scene as follows:

[0214] V=3·912 / α(5)

[0215] Where α is the atmospheric attenuation coefficient; V is the atmospheric visibility information.

[0216] The aforementioned highway scene event monitoring method can detect and track target vehicles, detect objects thrown from vehicle windows, measure vehicle speed, and detect visibility in highway scenes.

[0217] In summary, the aforementioned highway scene event monitoring system and method utilizes area array LiDAR to collect 3D point cloud data of highway scenes. By incorporating weather visibility detection and filtering processing into the basic 3D point cloud data, the system can accurately extract the contour information of target vehicles and effectively prevent visual interference from rain, fog, and other factors. Furthermore, the use of deep learning target detection methods for vehicle detection and tracking can accurately extract vehicle feature information, achieving higher accuracy than traditional target tracking methods. Additionally, the system incorporates a parabolic object detection algorithm, enabling it to detect and avoid dangerous objects thrown from vehicles in traffic scenes and further predict emergencies such as landslides and rockfalls.

[0218] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can occur depending on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A highway scene event monitoring system, characterized in that, include: At least one lidar, each lidar being mounted on a target vehicle in a highway scene, is used to collect point cloud data of the highway scene in the area where the target vehicle is traveling; At least one edge processor, each edge processor being coupled to a corresponding LiDAR, is used to receive point cloud data of the highway scene collected by the corresponding LiDAR, preprocess the point cloud data of the highway scene, and transmit the preprocessed point cloud data to the cloud server. A cloud server, coupled to an edge processor, receives preprocessed point cloud data of a highway scene transmitted from each edge processor, processes all point cloud data of the highway scene, determines the point cloud data of all target vehicles, and generates a 3D bounding box for each target vehicle. The cloud server processes all point cloud data of the highway scene, determines the point cloud data of all target vehicles, and generates a 3D bounding box for each target vehicle, including: Acquire point cloud data of the highway scene after preprocessing at each edge server; Clustering and segmentation are performed on all point cloud data of the acquired highway scene, and the point cloud data is divided into different point cloud sets based on density or geometric shape. Extract the features of the target vehicle point cloud from different point cloud sets to determine the point cloud data of all target vehicles; Based on the point cloud data of all target vehicles, generate a 3D preselection box for each target vehicle in the highway scene; Calculate the center point of the 3D preselection box for each target vehicle, standardize and adjust the position of the 3D preselection box for each target vehicle, obtain the normalized center point coordinates of each target vehicle, and generate multiple high-confidence 3D bounding boxes for each target vehicle. Based on the ratio of the intersection and union of the target vehicle's true bounding box and the generated 3D bounding box, low-confidence 3D bounding boxes and overlapping boxes are removed, and the target vehicle's 3D bounding box is generated.

2. The highway scene event monitoring system according to claim 1, characterized in that, The cloud server is further used for: Capture the 3D bounding box of the target vehicle; A Kalman filter is used to predict the 3D bounding box of the target vehicle in the previous frame to the bounding box of the target vehicle in the current frame. Match the predicted bounding box of the target vehicle with the current bounding box and calculate the loss function; Update the bounding box of the target vehicle based on the matching results, and delete the 3D bounding box of the target vehicle that has been lost from view.

3. The highway scene event monitoring system according to claim 1, characterized in that, The cloud server is further used for: Conditional filtering is applied to all point cloud data in the highway scene to obtain point cloud data of the area in front of all target vehicles; based on the point cloud data of the area in front of all target vehicles, objects thrown from vehicle windows in the highway scene are detected.

4. The highway scene event monitoring system according to claim 3, characterized in that, The cloud server performs object throwing from vehicle windows in highway scenarios based on point cloud data of the area in front of all target vehicles, including: Clustering and segmentation are performed on the point cloud data of the area in front of all target vehicles to obtain multiple different point cloud sets; Extract the geometric features of the target objects from multiple different point cloud sets to obtain the point cloud data of all target objects; Obtain the 3D coordinate data of the point cloud data of all target detection objects, and calculate the centroid coordinates of the point cloud of all target detection objects; Based on the centroid coordinates of each target object, calculate the distance between each target object and the corresponding target vehicle; Detect target objects that exhibit sudden changes in distance within a set time period and calculate the trajectory of the target object in the vertical direction; Construct the equation of free fall trajectory; The trajectory of the target object that is detected to have a sudden change in distance within a set time period is fitted with the equation of the free fall trajectory. The variance and covariance of the trajectory points and free fall points of the target object are calculated. It is then determined whether the variance and covariance are within a set threshold range. If they are, the target object is identified as a projectile thrown from a car window.

5. The highway scene event monitoring system according to claim 1, characterized in that, The cloud server is further used for: The point cloud data of all target objects in the preprocessed highway scene is acquired, and the reflection intensity data of the point cloud of all target objects is extracted. The reflection intensity data of the target objects at different locations is recorded. The target objects include target vehicles and traffic information signs. Preprocess all reflection intensity data for each target object; Angle normalization correction is applied to all reflection intensity data of each target object after preprocessing. Construct a formula for calculating echo intensity; Based on all the reflection intensity data and echo intensity calculation formulas of each target object after correction, calculate the atmospheric attenuation coefficient corresponding to all target objects; The average atmospheric attenuation coefficient for all target objects is calculated to obtain the atmospheric attenuation coefficient for the highway scene. Based on the atmospheric attenuation coefficient of the highway scene, atmospheric visibility information of the highway scene under different weather conditions is calculated.

6. A method for monitoring events in a highway scene, characterized in that, The method includes: LiDAR collects point cloud data of the highway scene in the area where the target vehicle is traveling; The edge processor receives point cloud data of the highway scene collected by LiDAR, preprocesses the point cloud data of the highway scene, and transmits the preprocessed point cloud data to the cloud server. The cloud server processes all point cloud data of the pre-processed highway scene, determines the point cloud data of all target vehicles, and generates a 3D bounding box for each target vehicle; The cloud server processes all point cloud data of the preprocessed highway scene, determines the point cloud data of all target vehicles, and generates a 3D bounding box for each target vehicle, including: Acquire point cloud data of the highway scene after preprocessing at each edge server; Clustering and segmentation are performed on all point cloud data of the acquired highway scene, and the point cloud data is divided into different point cloud sets based on density or geometric shape. Extract the features of the target vehicle point cloud from different point cloud sets to determine the point cloud data of all target vehicles; Based on the point cloud data of all target vehicles, generate a 3D preselection box for each target vehicle in the highway scene; Calculate the center point of the 3D preselection box for each target vehicle, standardize and adjust the position of the 3D preselection box for each target vehicle, obtain the normalized center point coordinates of each target vehicle, and generate multiple high-confidence 3D bounding boxes for each target vehicle. Based on the ratio of the intersection and union of the target vehicle's true bounding box and the generated 3D bounding box, low-confidence 3D bounding boxes and overlapping boxes are removed, and the target vehicle's 3D bounding box is generated.

7. The highway scene event monitoring method according to claim 6, characterized in that, The method further includes: The cloud server performs conditional filtering on all point cloud data of the highway scene to obtain point cloud data of the area in front of all target vehicles. Clustering and segmentation are performed on the point cloud data of the area in front of all target vehicles to obtain multiple different point cloud sets; Extract the geometric features of the target objects from multiple different point cloud sets to obtain the point cloud data of all target objects; Obtain the 3D coordinate data of the point cloud data of all target detection objects, and calculate the centroid coordinates of the point cloud of all target detection objects; Based on the centroid coordinates of each target object, calculate the distance between each target object and the corresponding target vehicle; Detect target objects that exhibit sudden changes in distance within a set time period and calculate the trajectory of the target object in the vertical direction; Construct the equation of free fall trajectory; The trajectory of the target object that is detected to have a sudden change in distance within a set time period is fitted with the equation of the free fall trajectory. The variance and covariance of the trajectory points and free fall points of the target object are calculated. It is then determined whether the variance and covariance are within a set threshold range. If they are, the target object is identified as a projectile thrown from a car window.

8. The highway scene event monitoring method according to claim 6, characterized in that, The method further includes: The cloud server acquires the point cloud data of all target objects in the preprocessed highway scene, extracts the reflection intensity data of the point cloud of all target objects, and records the reflection intensity data of target objects at different locations; the target objects include target vehicles and traffic information signs. Preprocess all reflection intensity data for each target object; Angle normalization correction is applied to all reflection intensity data of each target object after preprocessing. Construct a formula for calculating echo intensity; Based on all the reflection intensity data and echo intensity calculation formulas of each target object after correction, calculate the atmospheric attenuation coefficient corresponding to all target objects; The average atmospheric attenuation coefficient for all target objects is calculated to obtain the atmospheric attenuation coefficient for the highway scene. Based on the atmospheric attenuation coefficient of the highway scene, atmospheric visibility information of the highway scene under different weather conditions is calculated.

Images