Intelligent expressway real-time accident intelligent early warning and active management method based on big data

Abstract

This invention provides a smart highway real-time accident intelligent early warning and proactive control method based on big data, relating to the field of highway drone inspection. The method includes the following steps: acquiring sensor data collected by the onboard sensors of the inspection drone to generate a perception dataset; calculating vehicle driving parameters based on the perception dataset to extract traffic state features; acquiring environmental state features through the inspection drone; inputting the traffic state features and environmental state features into a classification model to output lane driving state labels; generating dynamic judgment threshold groups; and outputting multi-level preliminary early warning signals based on the traffic state features and the dynamic judgment threshold groups; performing multi-frame temporal consistency verification; outputting the final early warning signal and executing corresponding control actions. Using this method, routine automatic drone inspections can be supported. By leveraging the mobile inspection characteristics of drones to supplement the coverage blind spots of fixed roadside monitoring, labor costs are reduced, average working time per person is reduced, and accurate early warnings are achieved.

Description

Technical Field

[0001] This invention relates to the field of high-speed unmanned aerial vehicle (UAV) inspection, specifically to a smart highway real-time accident intelligent early warning and proactive control method based on big data. Background Technology

[0002] With the continuous expansion of my country's expressway network and the sustained growth of traffic volume, proactive early warning and dynamic proactive management of traffic accidents have become the core direction of smart expressway construction and a key means to improve road traffic safety and operational efficiency. Currently, the main approach relies on fixed video surveillance equipment deployed along the roadside to achieve routine traffic monitoring on regular road sections. This is supplemented by manually operated inspection drones to conduct additional patrols on key road sections. Combined with statistical analysis of backend traffic data, this enables the identification, early warning issuance, and control and response of traffic accidents.

[0003] However, existing technical solutions still have many unresolved issues in practical applications: fixed roadside monitoring is limited by deployment locations and equipment viewing angles, generally resulting in blind spots in perception coverage, and it is difficult to flexibly adjust the monitoring range. While manually operated inspection drones have the advantage of maneuverability and flexibility, their inspection efficiency is constrained by personnel skill levels, labor costs, and working hours, making it impossible to achieve routine drone inspections. Furthermore, after an accident, it takes a considerable amount of time for personnel to reach the scene for communication and handling.

[0004] Therefore, there is an urgent need for a smart highway real-time accident intelligent early warning and proactive control method based on big data and drone inspection. Summary of the Invention

[0005] To overcome existing technical problems, this invention provides a smart highway real-time accident intelligent early warning and proactive control method based on big data.

[0006] The present invention adopts the following technical solution.

[0007] A big data-based intelligent real-time accident early warning and proactive control method for highways includes the following steps: S1. Acquire the sensing data collected by the airborne sensors of the inspection drone, extract the spatial positioning data of each vehicle, and construct the coordinate transformation relationship by combining the absolute three-dimensional anchor points of the high-precision map to convert the spatial positioning data into physical mileage coordinates in order to generate a sensing dataset. S2. Calculate vehicle driving parameters based on the perception dataset to extract traffic state features, obtain environmental state features through inspection drones, input the traffic state features and environmental state features into the classification model, and output lane driving state labels. S3. Select the corresponding safety benchmark threshold group based on the lane driving status label, select the bias coefficient that represents the historical accident performance of the area corresponding to the physical mileage coordinate, adjust the safety benchmark threshold group through the bias coefficient to generate a dynamic judgment threshold group, and output multi-level preliminary warning signals based on traffic state characteristics and dynamic judgment threshold group. S4. Perform multi-frame timing consistency verification on multi-level preliminary warning signals, output the final warning signal and execute the corresponding control actions.

[0008] As a further improvement of the present invention, the airborne sensor is a camera, the sensing data is a dynamic video stream, and the spatial positioning data is two-dimensional pixel coordinates. The specific steps for extracting spatial positioning data from each vehicle and constructing coordinate transformation relationships using absolute 3D anchor points from a high-precision map to convert spatial positioning data to physical mileage coordinates include: S101. The target region of interest is obtained by processing according to the preset safety offset angle or the preset cropping area. Based on the target region of interest mapping, the boundary line feature points of the high-speed lane in each frame of the dynamic video stream and the grounding points of each vehicle chassis are extracted as vehicle pixel coordinates. S102. Based on electronic video stabilization technology, continuously perform two-dimensional affine compensation on images of different frames in the dynamic video stream, and continuously acquire pose data of the inspection drone containing three-dimensional coordinates and attitude angles for global benchmark correction, so as to calibrate the mapping relationship between the image center point and the coordinates of the real physical world. S103. Based on the three-dimensional coordinates of the inspection drone, retrieve the absolute three-dimensional anchor points of each lane line in the corresponding road segment in the high-precision map, extract the corresponding two-dimensional ground physical coordinates based on the absolute three-dimensional anchor points, and construct a set of matching point pairs between the boundary line feature points and the two-dimensional ground physical coordinates based on the pose data. S104. The dynamic two-dimensional homography perspective transformation matrix is ​​solved by the direct linear transformation algorithm based on the matching point pair set. S105. Map all vehicle pixel coordinates to obtain at least one physical mileage coordinate based on the two-dimensional homography perspective transformation matrix.

[0009] As a further improvement of the present invention, the airborne sensor is a solid-state lidar, the sensing data is three-dimensional point cloud data, and the spatial positioning data is three-dimensional spatial coordinates. The specific steps for extracting spatial positioning data from each vehicle and constructing coordinate transformation relationships using absolute 3D anchor points from a high-precision map to convert spatial positioning data to physical mileage coordinates include: S111. Filter and cluster the 3D point cloud data, and extract the boundary point of each vehicle as the 3D laser space coordinates relative to the coordinate system of the lidar itself. S112. Continuously acquire pose data of the inspection drone, including three-dimensional coordinates and attitude angles, and map the three-dimensional laser space coordinates to three-dimensional space coordinates. S113. Retrieve the absolute three-dimensional anchor points of each lane line in the current road segment from the high-precision map, match the nearest lane line for the three-dimensional spatial coordinates of each vehicle, project the three-dimensional spatial coordinates onto the lane line along the normal direction, and extract the longitudinal arc length of the projection point on the corresponding lane line and the lateral offset of the three-dimensional spatial coordinates from the lane line to obtain the physical mileage coordinates of the corresponding vehicle.

[0010] As a further improvement of the present invention, the airborne sensor includes a camera and a solid-state lidar, both of which have an operating state and a non-operating state.

[0011] As a further improvement of the present invention, the specific steps for calculating vehicle driving parameters based on a perception dataset to extract traffic state features include: S21. Track the corresponding vehicle through the target tracking algorithm, generate a continuous physical mileage coordinate time series of each vehicle, perform abnormal mutation preprocessing on the physical mileage coordinate time series, and use a fixed duration sliding window to perform moving average filtering to obtain a mileage coordinate sequence of each vehicle containing multiple smooth mileage coordinates. S22. Based on the mileage coordinate sequence of each vehicle, obtain the smoothed mileage coordinates of each vehicle at the current time and the previous time, calculate the average speed of the lane at different times, and the speed of each vehicle at different times. Obtain smoothed mileage coordinates of adjacent vehicles in the same lane and calculate the longitudinal distance between vehicles at different times. Obtain the lane-changing trajectory of each vehicle as it crosses lane lines; Based on the vehicle speeds at different times, the frequency of emergency braking per unit time is calculated. S23. Integrate the vehicle speed, longitudinal distance between vehicles and lane change trajectory of each vehicle at different times, as well as the average speed of the lane at different times and the frequency of emergency braking per unit time to obtain traffic state characteristics.

[0012] As a further improvement of the present invention, the specific steps of acquiring environmental state features through an inspection drone, inputting traffic state features and environmental state features into a classification model, and outputting lane driving state labels include: S24. Based on the three-dimensional coordinate reading of the inspection drone, the real-time rainfall and wind speed data of the weather station are read. By integrating the visibility and water accumulation values ​​uploaded by the operator, the environmental status characteristics are obtained. S25. Integrate environmental and traffic state features and input them into a pre-trained classification model to output a lane driving state label containing any one of free flow, saturated flow, congestion shock wave, or high-risk warning.

[0013] As a further improvement of the present invention, the step of generating the bias coefficient includes: dividing a certain highway segment into multiple discrete spatial grids according to a preset longitudinal mileage division, obtaining the traffic accident database of the corresponding highway segment in a preset accident reference year, calculating the total frequency of historical accidents for each spatial grid, calculating the average frequency and standard deviation of historical accidents for all spatial grids, calculating the risk heat index for each spatial grid, and mapping the risk heat index to obtain the bias coefficient through normalization.

[0014] As a further improvement of the present invention, the multi-level preliminary warning signal includes a corresponding warning level, a corresponding lane driving status label and a corresponding risk mileage coordinate, with the warning levels being Level 1 warning, Level 2 warning and Level 3 warning, which are respectively of increasing severity. When a Level 2 or Level 3 warning is issued, the inspection mission of the inspection drone is suspended and hovered, with the onboard sensors oriented towards the risk mileage coordinates.

[0015] As a further improvement of the present invention, the specific steps of multi-frame timing consistency verification include: establishing a sliding verification window according to a preset sliding verification number, obtaining the mode of the warning level within the sliding verification window, and calculating the jump variance. If the variance of the jump is lower than or equal to the preset fluctuation threshold, or if the warning level first increases monotonically and then remains stable, the warning level corresponding to the mode is output as the final warning signal. Otherwise, the inspection drone is moved closer to the risk mileage coordinate, or the case is transferred to manual remote review, or the highest warning level in the sliding verification window is used as the final warning signal.

[0016] The beneficial effects of this invention are as follows: It supports routine automatic inspections by drones. Leveraging the mobile inspection capabilities of drones to supplement the coverage blind spots of fixed roadside monitoring, it can essentially cover the entire highway section. Secondly, it reduces reliance on manual operation, significantly lowering labor costs and reducing average working time per person. Furthermore, by fusing and evaluating lane driving status labels, accurate early warnings are achieved based on lane driving status labels, bias coefficients, and multi-frame temporal consistency checks. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a flowchart illustrating the present invention; Figure 2 This is a schematic flowchart of the first embodiment of the present invention, S1; Figure 3 This is a schematic flowchart of the second embodiment of the present invention, S1. Detailed Implementation

[0019] The accompanying drawings are for illustrative purposes only and should not be construed as limiting the scope of this patent. To better illustrate this embodiment, some parts in the drawings may be omitted, enlarged, or reduced, and do not represent the actual dimensions of the product.

[0020] It will be understood by those skilled in the art that certain well-known structures and their descriptions may be omitted in the accompanying drawings. The technical solution of the present invention will be further described below with reference to the accompanying drawings and embodiments.

[0021] A big data-based intelligent real-time accident early warning and proactive control method for highways includes the following steps: S1. Acquire the sensing data collected by the airborne sensors of the inspection drone, extract the spatial positioning data of each vehicle, and construct the coordinate transformation relationship by combining the absolute three-dimensional anchor points of the high-precision map to convert the spatial positioning data into physical mileage coordinates in order to generate a sensing dataset. The inspection method for inspection drones already exists in the current protection of forest land, and it is not the focus of this invention, so this invention will not elaborate on it further.

[0022] As a first embodiment of the present invention: The airborne sensor is a camera, the sensing data is a dynamic video stream, and the spatial positioning data is two-dimensional pixel coordinates. The specific steps for extracting spatial positioning data from each vehicle and constructing coordinate transformation relationships using absolute 3D anchor points from a high-precision map to convert spatial positioning data to physical mileage coordinates include: S101. The target region of interest is obtained by processing according to the preset safety offset angle or the preset cropping area. Based on the target region of interest mapping, the boundary line feature points of the high-speed lane in each frame of the dynamic video stream and the grounding points of each vehicle chassis are extracted as vehicle pixel coordinates. Because the physical depth distortion is more severe closer to the viewpoint, exacerbating errors, it is necessary to process the dynamic video stream to obtain the target region of interest for vehicle identification in the near and middle ranges. Specifically, the preset cropping region can be based on a preset image height, such as cropping the upper 30%-40% of the image. However, as a preferred embodiment of this invention, based on the approximate trapezoidal distribution of highways in the image, the cropping region is defined as a trapezoidal region. The upper boundary of the trapezoidal region is at 30%-40% of the image height, and the left and right boundary points of the upper boundary are shrunk inward by 10%-15% of the image width. Pixels located within the trapezoidal region are assigned a weight of 1, and the rest are assigned a weight of 0, thus obtaining the target region of interest.

[0023] Of course, another way to obtain the target region of interest in this invention is to extract the real-time pitch angle of the camera gimbal and the vertical field of view of the camera, calculate the pixel row coordinates of the physical horizon in the current image through the camera intrinsic parameters, and according to the preset safety offset angle, such as shifting downward by 5 degrees, it can be understood as the pixel distance corresponding to shifting 5° from the horizon position downwards in the image. Calculate the row coordinates of the far-end truncated pixels, assign a weight of 0 to all pixels in the image coordinate system whose vertical coordinate is lower than the row coordinates of the far-end truncated pixels, and assign a weight of 1 to the rest.

[0024] It is important to note that since at least four pairs of non-collinear matching points are required to solve the two-dimensional homography perspective transformation matrix, whether the above-mentioned preset safety offset angle or preset clipping region is used for processing, the determination of the safety offset angle and the determination of the clipping region must ensure that at least four pairs of non-collinear matching points exist.

[0025] For boundary line feature point acquisition, semantic segmentation algorithms are used to extract the endpoints of dashed lane lines at high speeds or discrete points sampled at fixed pixel intervals on the lane lines to obtain boundary line feature points. The reason for using the vehicle chassis ground contact point as the vehicle pixel coordinate is that if locations such as the roof, license plate, or side of the vehicle are selected, these points have ground clearance, which does not meet the prerequisite for subsequent two-dimensional homography perspective transformation matrix calculations. Although the influence of ground clearance can be removed by projecting onto the ground, the projected ground coordinates will have errors. Only the chassis ground contact point, i.e., the location where the tires are in contact with the ground, yields relatively accurate vehicle pixel coordinates. It should be noted that only the ground contact point of one tire needs to be obtained. Before obtaining the vehicle chassis ground contact point, each vehicle is first identified using a vehicle recognition algorithm. This is widely used in fixed video surveillance equipment deployed on the roadside, so it will not be elaborated further here.

[0026] Of course, if the vehicle pixel coordinates cannot be obtained through the chassis grounding point, the prior size of the vehicle target detection is introduced, such as matching the standard wheelbase and vehicle height based on the vehicle type classification, constructing a local three-dimensional pseudo-box, and calculating the physical mileage coordinates of its bottom center point through back projection using camera intrinsic and extrinsic parameters, so as to eliminate the perspective projection error caused by the ground clearance.

[0027] S102. Based on electronic video stabilization technology, continuously perform two-dimensional affine compensation on images of different frames in the dynamic video stream, and continuously acquire pose data of the inspection drone containing three-dimensional coordinates and attitude angles for global benchmark correction, so as to calibrate the mapping relationship between the image center point and the coordinates of the real physical world. Electronic video stabilization is a common technique in drone gimbals. It involves extracting high-contrast feature points from frames t and t-1, such as the boundary line feature points mentioned above. Using fast optical flow methods, such as the UCAS-Kanade algorithm, the pixel displacement vectors of these feature points are calculated. Based on this displacement vector, the two-dimensional affine transformation matrix between the two frames is solved. The image of frame t is then subjected to reverse translation and rotation compensation to absorb some of the high-frequency mechanical vibrations generated by the drone rotor. The remaining jitter is negligible relative to the distance traveled at the vehicle speed.

[0028] Because coordinate drift gradually occurs during prolonged operation of electronic video stabilization, the system can read the UAV's RTK spatial coordinates and IMU attitude angles (i.e., the aforementioned three-dimensional coordinates and attitude angles) at a lower frequency, such as 1Hz to 5Hz. Using this pose data as a global reference, the mapping between the image center point and the real-world coordinates is recalibrated, eliminating the cumulative drift caused by electronic video stabilization. By employing high-frequency electronic video stabilization and low-frequency global reference correction, the concurrent processing load on the system bus can be significantly reduced.

[0029] S103. Based on the three-dimensional coordinates of the inspection drone, retrieve the absolute three-dimensional anchor points of each lane line in the corresponding road segment in the high-precision map, extract the corresponding two-dimensional ground physical coordinates based on the absolute three-dimensional anchor points, and construct a set of matching point pairs between the boundary line feature points and the two-dimensional ground physical coordinates based on the pose data. Since the image center point of the UAV has a mapping relationship with the coordinates of the real physical world, the true physical location of the corresponding road segment in the image can be calculated when the UAV's 3D coordinates are known. The absolute 3D anchor points of each lane line of the corresponding road segment can then be retrieved from the high-precision map. Ignoring their elevation, the 2D ground physical coordinates are obtained, facilitating the subsequent calculation of the 2D homography perspective transformation matrix. Using the UAV's pose data, the 2D ground physical coordinates are roughly projected onto the image coordinate system. Then, using the Euclidean distance minimization principle or graph matching algorithms, the closest point pairs that conform to the lane line topological order are found, forming a set of matching point pairs.

[0030] S104. The dynamic two-dimensional homography perspective transformation matrix is ​​solved by the direct linear transformation algorithm based on the matching point pair set. For any pair of matching points in the set of matching points, according to the homography mapping formula, it can be expanded into two independent homogeneous linear equations. It should be noted here that, due to 3... A matrix H of size 3 has eight degrees of freedom (size factor normalization, usually by setting the bottom right element). Therefore, theoretically, at least four pairs of non-collinear matching points are needed to solve this problem.

[0031] The system substitutes k pairs of matching points, where k is greater than or equal to 4, and constructs 2k combinations. The overdetermined coefficient matrix A of 9 transforms the problem into an optimization problem A·h=0, where h is the coefficient of 9 after the matrix H is flattened. 1. Column vector. Perform singular value decomposition on matrix A, extract the right singular vector corresponding to its smallest singular value, and then use it to obtain 3. The matrix H is the optimal homography matrix in the least squares sense.

[0032] Furthermore, the present invention can also employ a random sampling consensus algorithm on this basis, randomly selecting four pairs of matching points each time to calculate the optimal homography matrix H and using the remaining matching points to test the projection error of the optimal homography matrix H, and taking the optimal homography matrix H with the smallest error as the final optimal homography matrix H, which can enhance the mapping stability of coordinates.

[0033] It's important to note that while the mathematical definition of a two-dimensional homography perspective transformation matrix is ​​a mapping between two two-dimensional planes, if the terrain exhibits longitudinal slope variations (up and down), the road surface is no longer a perfect two-dimensional plane. However, this is precisely where the ingenuity of this invention lies. The technical solution is specifically designed for highway scenarios. Highways not only have the clearest lane markings, but also adhere to strict road engineering design specifications. Their longitudinal slope is typically limited to 3%–5%, and their cross slope is usually within 2%, with minimal curvature variation and virtually no longitudinal slope changes. For mountain roads, highways generally have monotonous upward or downward slopes, without any up-and-down longitudinal slope variations. Furthermore, by setting a target region of interest, the present invention can crop out the distant scene. Within the target region of interest at near to mid-range distances, the road surface can be strictly approximated as a two-dimensional local plane, which can greatly reduce the computational redundancy of three-dimensional mapping transformation. Moreover, each time the two-dimensional homography perspective transformation matrix is ​​updated, the current two-dimensional homography perspective transformation matrix can fit the local plane slope of the current image. During the gradual update, it can realize the calculus calculation thinking of approximating curves with straight lines, implicitly absorbing the changes in slope.

[0034] S105. Map all vehicle pixel coordinates to obtain at least one physical mileage coordinate based on the two-dimensional homography perspective transformation matrix.

[0035] It should be further explained here that the camera setup allows for remote manual intervention and control when the final warning signal is uploaded.

[0036] As a second embodiment of the present invention: The airborne sensor is a solid-state lidar, the sensing data is three-dimensional point cloud data, and the spatial positioning data is three-dimensional spatial coordinates. The specific steps for extracting spatial positioning data from each vehicle and constructing coordinate transformation relationships using absolute 3D anchor points from a high-precision map to convert spatial positioning data to physical mileage coordinates include: S111. Filter and cluster the 3D point cloud data, and extract the boundary point of each vehicle as the 3D laser space coordinates relative to the coordinate system of the lidar itself. A ray-slope ground filtering algorithm can be used to identify large, continuous point clouds representing highway surface features. Point clouds taller than a preset maximum vehicle height within these large, continuous point clouds are removed, thus eliminating objects such as overhead poles. Point clouds outside the left and right boundaries of these large, continuous point clouds are also removed, further eliminating objects such as trees and buildings. Finally, the large, continuous point clouds are removed. A spatial distance clustering threshold is set for the remaining point clouds. Discrete point clouds with a spatial distance less than this threshold are grouped into the same cluster, with each cluster representing an individual vehicle. The spatial distance clustering threshold can be set to 0.3 meters or as needed. For a specific corner boundary point, the corner closest to the inspection drone can be selected.

[0037] S112. Continuously acquire pose data of the inspection drone, including three-dimensional coordinates and attitude angles, and map the three-dimensional laser space coordinates to three-dimensional space coordinates. By using the external parameter matrix between the factory-calibrated lidar and the UAV flight control center, which includes the external parameter matrix of rotation and translation vectors, the three-dimensional laser space coordinates in the lidar's own coordinate system are converted into three-dimensional body coordinates in the UAV's body coordinate system. Based on the pose data, the three-dimensional body coordinates in the UAV's body coordinate system are mapped to three-dimensional space coordinates in the real world coordinate system.

[0038] S113. Retrieve the absolute three-dimensional anchor points of each lane line in the current road segment from the high-precision map, match the nearest lane line for the three-dimensional spatial coordinates of each vehicle, project the three-dimensional spatial coordinates onto the lane line along the normal direction, and extract the longitudinal arc length of the projection point on the corresponding lane line and the lateral offset of the three-dimensional spatial coordinates from the lane line to obtain the physical mileage coordinates of the corresponding vehicle.

[0039] The technical principle of the above-mentioned technical solution is also known as Frenet coordinate system transformation, which is a commonly used technical means in intelligent driving of vehicles. Therefore, this invention will not elaborate further.

[0040] The disadvantage of the second embodiment is that when the final warning signal is uploaded and manual remote takeover is initiated, the human operator cannot directly make judgments and controls based on the intuitive visuals.

[0041] Therefore, as a third embodiment of the present invention: Airborne sensors include cameras and solid-state LiDAR, both of which have active and inactive states. Combining the two allows for the activation of the solid-state LiDAR in challenging environments (where physical mileage coordinates cannot be obtained for extended periods), or for automated inspections, or for remote manual intervention by activating the camera, all of which fall within the scope of this invention.

[0042] S2. Calculate vehicle driving parameters based on the perception dataset to extract traffic state features, obtain environmental state features through inspection drones, input the traffic state features and environmental state features into the classification model, and output lane driving state labels. As a further improvement of the present invention, the specific steps for calculating vehicle driving parameters based on a perception dataset to extract traffic state features include: S21. Track the corresponding vehicle through the target tracking algorithm, generate a continuous physical mileage coordinate time series of each vehicle, perform abnormal mutation preprocessing on the physical mileage coordinate time series, and use a fixed duration sliding window to perform moving average filtering to obtain a mileage coordinate sequence of each vehicle containing multiple smooth mileage coordinates. Since real-world vehicles have speed limits, significant mileage jumps are unlikely to occur in a very short time. Therefore, a maximum reasonable displacement threshold at the physical level can be set to eliminate abnormal abrupt changes. For example, the theoretical maximum speed of a vehicle can be set to 150km / h~200km / h. The maximum mileage change is obtained by multiplying the time interval between two physical mileage coordinate acquisitions by the theoretical maximum speed. If the absolute value of the difference between the two physical mileage coordinates is greater than the maximum mileage change, it is considered an abnormal abrupt change and is eliminated. Eliminated abnormal abrupt changes can be obtained by linear interpolation between two valid physical mileage coordinates. Moving average filtering is a basic processing method, so it will not be elaborated on further in this invention.

[0043] S22. Based on the mileage coordinate sequence of each vehicle, obtain the smoothed mileage coordinates of each vehicle at the current time and the previous time, calculate the average speed of the lane at different times, and the speed of each vehicle at different times. The system obtains smoothed mileage coordinates of adjacent vehicles within the same lane and calculates the longitudinal distance between vehicles at different times. It's worth noting that knowing whether two physical mileage coordinates are in the same lane is a standard technical method, currently achievable by navigation and autonomous driving systems. Therefore, it's also possible to obtain the lane-changing trajectory of vehicles crossing lane lines.

[0044] Obtain the lane-changing trajectory of each vehicle as it crosses lane lines; Based on the vehicle speeds at different times, the frequency of emergency braking per unit time is calculated; a braking deceleration threshold is set, which can be set to 4~6. This is distinct from normal deceleration and coasting deceleration. If the vehicle's deceleration calculated from several consecutive changes in physical mileage coordinates exceeds the braking deceleration threshold, and the vehicle's brake lights illuminate (if the inspection drone is behind the vehicle), it can be considered a single instance of emergency braking. The preset unit time can be within ten seconds, three seconds, or five seconds.

[0045] S23. Integrate the vehicle speed, longitudinal distance between vehicles and lane change trajectory of each vehicle at different times, as well as the average speed of the lane at different times and the frequency of emergency braking per unit time to obtain traffic state characteristics.

[0046] As a further improvement of the present invention, the specific steps of acquiring environmental state features through an inspection drone, inputting traffic state features and environmental state features into a classification model, and outputting lane driving state labels include: S24. Based on the three-dimensional coordinate reading of the inspection drone, the real-time rainfall and wind speed data of the weather station are read. By integrating the visibility and water accumulation values ​​uploaded by the operator, the environmental status characteristics are obtained. The visibility and water accumulation values ​​of this invention can be calculated manually using meteorological-grade lidar, which is already equipped in some local meteorological bureaus.

[0047] Regarding visibility values, rain and fog particles cause backscattering of the laser, leading to attenuation of the received echo intensity. This is based on the lidar equation. The density of suspended noise points in mid-air was statistically analyzed. The natural logarithm of both sides of the lidar equation was taken to transform the exponential relationship into a linear one. Suspended noise points at different distances were analyzed, with the distance between the target point and the lidar as the x-axis. Using the ordinate as the vertical axis, a straight line is fitted using the least squares method, and the atmospheric extinction coefficient is obtained by inverse solution. . A higher value indicates more severe scattering and lower visibility. The atmospheric extinction coefficient is calculated under clear, clean weather conditions. As a benchmark value for the extinction coefficient with a visibility level of 1, the atmospheric extinction coefficient is measured when visibility is judged to be very low. The maximum extinction coefficient is used to obtain the visibility value through a normalization formula. The visibility value can be updated every half hour.

[0048] , It is the maximum value of the extinction coefficient. It is the baseline value for the extinction coefficient.

[0049] The degree of water accumulation can be manually assessed based on rainfall. For example, the current rainfall and the average slipperiness of the highway can be evaluated each time it rains. When the rainfall exceeds a certain amount, the degree of water accumulation is 1.

[0050] S25. Integrate environmental and traffic state features and input them into a pre-trained classification model to output a lane driving state label containing any one of free flow, saturated flow, congestion shock wave, or high-risk warning.

[0051] To facilitate understanding of the general idea behind the classification model, an example is provided here. Since road conditions and driving habits vary across different regions, adjustments must be made based on actual road conditions. When the average lane speed exceeds 80% of the highway's maximum speed limit, longitudinal vehicle spacing exceeds the preset safe distance threshold, the frequency of emergency braking is less than or equal to 1, lane-changing trajectories are irregular, and real-time rainfall, wind speed data, visibility values, and water accumulation values ​​are all less than the corresponding first danger value, then the lane driving status label can be considered free-flowing.

[0052] When the average speed of vehicles in a lane is less than the maximum speed limit of the highway, the variance of longitudinal vehicle spacing is less than a certain stable threshold, and the frequency of emergency braking is less than or equal to 1, the lane driving status label can be regarded as saturated flow. When the average speed per lane is below the low-speed threshold and the longitudinal distance between vehicles is close to the safety limit threshold, the lane driving status label can be considered a congestion shock wave. It's important to note that the reason for not using the number of vehicles detected here to determine if it's a congestion shock wave is to avoid situations where some highways are not congested overall, but congestion occurs in a merging or diverging lane.

[0053] When the frequency of emergency braking gradually increases, or lane-changing trajectories become predictable (e.g., vehicles in one lane all change lanes to another), or any of the real-time rainfall, wind speed, visibility, or water accumulation values ​​exceeds the corresponding second danger value, the lane driving status label can be considered a high-risk warning. Furthermore, if the lane driving status label is considered a high-risk warning due to predictable lane-changing trajectories, then the S3 level three warning can be skipped directly.

[0054] It should be reiterated that the classification model does not make classifications based on rigid threshold conditions, but rather classifies the lane driving state label based on the specific features of environmental and traffic conditions.

[0055] S3. Select the corresponding safety benchmark threshold group based on the lane driving status label, select the bias coefficient that represents the historical accident performance of the area corresponding to the physical mileage coordinate, adjust the safety benchmark threshold group through the bias coefficient to generate a dynamic judgment threshold group, and output multi-level preliminary warning signals based on traffic state characteristics and dynamic judgment threshold group. As mentioned above, due to differences in road conditions, traffic control levels, and driving habits across different regions, the safety benchmark threshold set must be adjusted and determined based on actual road conditions. As a specific example of this invention: For the free-flow safety benchmark threshold group, Level 1 warning (congestion risk) is when the average vehicle speed in the lane gradually decreases from more than 100 km / h to 85 km / h within M consecutive seconds, and the average longitudinal distance between vehicles is less than 80 meters; Level 2 warning (rear-end collision risk) is when the frequency of emergency braking is greater than 1, and the average longitudinal distance between vehicles is less than 50 meters; Level 3 warning (accident risk) is when the speed of a single vehicle is less than 10 km / h within M consecutive seconds.

[0056] For the safety benchmark threshold group for saturated flow, Level 1 warning (congestion risk) is when the average speed of a lane is less than 60 km / h for M consecutive seconds; Level 2 warning (rear-end collision risk) is when the frequency of emergency braking is greater than or equal to 3 and the average longitudinal distance between vehicles is less than 25 meters; Level 3 warning (accident risk) is when the speed of a single vehicle is less than 10 km / h for M consecutive seconds.

[0057] For the safety baseline threshold group for congestion shock waves, the default is Level 1 warning (congestion risk); Level 2 warning (rear-end collision risk) is when the maximum speed difference within the same lane exceeds the rear-end collision risk threshold; Level 3 warning (accident risk) is when the speed of a single vehicle is less than 5 km / h for a preset dwell time. The preset dwell time can be set to one minute.

[0058] For the safety baseline threshold group of high-risk warnings, there is no Level 1 warning (congestion risk); Level 2 warning (rear-end collision risk) is when the longitudinal distance between vehicles is less than 50 meters; Level 3 warning (accident risk) is when the speed of a single vehicle is less than 10 km / h for M consecutive seconds.

[0059] As a further improvement of the present invention, the step of generating the bias coefficient includes: dividing a certain highway segment into multiple discrete spatial grids according to a preset longitudinal mileage division, obtaining the traffic accident database of the corresponding highway segment in a preset accident reference year, calculating the total frequency of historical accidents for each spatial grid, calculating the average frequency and standard deviation of historical accidents for all spatial grids, calculating the risk heat index for each spatial grid, and mapping the risk heat index to obtain the bias coefficient through normalization.

[0060] The expression for the risk heat index: , It is the average frequency of historical accidents. It is the standard deviation. This represents the total historical accident frequency for grid C. The default accident reference year is 3 years.

[0061] The expression for the bias coefficient: , It is a cutoff factor. It is the upper limit of the reference bias. It is a sensitivity smoothing factor.

[0062] Specifically, as a preferred embodiment of the present invention, the cutoff factor is limited to between 0.6 and 0.8, with 0.65 being recommended. The maximum tightening of the safety baseline threshold group is also limited to within 35%. The upper limit of the baseline bias is fixed at 1, and the sensitivity smoothing factor is recommended to be 1.2, which is used to control the degree of attenuation smoothness of the safety baseline threshold group and prevent abrupt changes.

[0063] It's important to note that adjusting the safety baseline threshold set using the bias coefficient to generate a dynamic judgment threshold set doesn't involve directly multiplying the bias coefficient by all the thresholds in the safety baseline threshold set. Instead, it involves targeted adjustments. Specifically, for example, the average longitudinal vehicle spacing and rear-end collision risk threshold are multiplied by the bias coefficient, because accident-prone areas require greater safety distances than accident-free areas.

[0064] As a further improvement of the present invention, the multi-level preliminary warning signal includes a corresponding warning level, a corresponding lane driving status label and a corresponding risk mileage coordinate, with the warning levels being Level 1 warning, Level 2 warning and Level 3 warning, which are respectively of increasing severity. When a Level 2 or Level 3 warning is issued, the inspection mission of the inspection drone is suspended and hovered, with the onboard sensors oriented towards the risk mileage coordinates.

[0065] Since multi-level preliminary warning signals will undergo multi-frame timing consistency verification in the future, hovering the drone and aligning it with the risk mileage coordinates can improve the accuracy of the warning when a level 2 or level 3 warning is output.

[0066] S4. Perform multi-frame timing consistency verification on multi-level preliminary warning signals, output the final warning signal and execute the corresponding control actions.

[0067] During Level 1, Level 2, and Level 3 alerts, the inspection drone uploads the final alert signal to allow for remote human intervention to review and implement control measures, such as broadcast guidance. During a Level 3 alert, the inspection drone approaches and hovers near the risk mileage coordinates, flashing warning lights in the direction of oncoming vehicles or guiding them to turn, extending the alert distance from a few dozen meters to over a hundred meters, while awaiting remote human intervention to review and implement control measures, such as broadcast guidance.

[0068] As a further improvement of the present invention, the specific steps of multi-frame timing consistency verification include: establishing a sliding verification window according to a preset sliding verification number, obtaining the mode of the warning level within the sliding verification window, and calculating the jump variance. If the variance of the jump is lower than or equal to the preset fluctuation threshold, or if the warning level first increases monotonically and then remains stable, the warning level corresponding to the mode is output as the final warning signal. Otherwise, the inspection drone is moved closer to the risk mileage coordinate, or the case is transferred to manual remote review, or the highest warning level in the sliding verification window is used as the final warning signal.

[0069] The fluctuation threshold is determined based on the size of the sliding check window. The sliding check window can be set to the judgment time of 15 multi-level preliminary warning signals, and the fluctuation threshold can be set to 0.5. In this case, 1 to 2 false detections can be tolerated. When the jump variance is large.

[0070] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the implementation of the present invention. Those skilled in the art can make other variations or modifications based on the above description. It is neither necessary nor possible to exhaustively describe all embodiments here. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the claims of the present invention.

Claims

1. A smart highway real-time accident intelligent early warning and proactive control method based on big data, characterized in that, Includes the following steps: S1. Acquire the sensing data collected by the airborne sensors of the inspection drone, extract the spatial positioning data of each vehicle, and construct the coordinate transformation relationship by combining the absolute three-dimensional anchor points of the high-precision map. Convert the spatial positioning data to physical mileage coordinates to generate a sensing dataset. S2. Calculate vehicle driving parameters based on the perception dataset to extract traffic state features, obtain environmental state features through inspection drones, input the traffic state features and environmental state features into a classification model, and output lane driving state labels. S3. Select the corresponding safety benchmark threshold group according to the lane driving status label, select the bias coefficient that represents the historical accident performance of the area corresponding to the physical mileage coordinate, adjust the safety benchmark threshold group through the bias coefficient to generate a dynamic judgment threshold group, and output multi-level preliminary warning signals according to traffic state characteristics and dynamic judgment threshold group. S4. Perform multi-frame timing consistency verification on the multi-level preliminary warning signals, output the final warning signal and execute the corresponding control actions.

2. The intelligent real-time accident early warning and proactive control method for smart highways based on big data as described in claim 1, characterized in that, The airborne sensor is a camera, the sensing data is a dynamic video stream, and the spatial positioning data is two-dimensional pixel coordinates; The specific steps for extracting the spatial positioning data of each vehicle and constructing coordinate transformation relationships by combining the absolute three-dimensional anchor points of a high-precision map to convert the spatial positioning data to physical mileage coordinates include: S101. Obtain the target region of interest by processing according to the preset safety offset angle or the preset cropping area, and extract the boundary line feature points of the high-speed lane of each frame image in the dynamic video stream and the grounding points of each vehicle chassis as vehicle pixel coordinates based on the target region of interest mapping. S102. Based on the electronic video stabilization technology, two-dimensional affine compensation is continuously performed on the images of different frames in the dynamic video stream, and the pose data of the inspection drone containing three-dimensional coordinates and attitude angles is continuously acquired for global reference correction, so as to calibrate the mapping relationship between the image center point and the coordinates of the real physical world. S103. Based on the three-dimensional coordinates of the inspection drone, retrieve the absolute three-dimensional anchor points of each lane line in the corresponding road segment in the high-precision map, extract the corresponding two-dimensional ground physical coordinates based on the absolute three-dimensional anchor points, and construct a set of matching point pairs between the boundary line feature points and the two-dimensional ground physical coordinates based on the pose data. S104. Based on the set of matching points, the dynamic two-dimensional homography perspective transformation matrix is ​​solved by the direct linear transformation algorithm. S105. Based on the two-dimensional homography perspective transformation matrix, map all the vehicle pixel coordinates to obtain at least one physical mileage coordinate.

3. The intelligent real-time accident early warning and proactive control method for smart highways based on big data as described in claim 1, characterized in that, The airborne sensor is a solid-state lidar, the sensing data is three-dimensional point cloud data, and the spatial positioning data is three-dimensional spatial coordinates. The specific steps for extracting the spatial positioning data of each vehicle and constructing coordinate transformation relationships by combining the absolute three-dimensional anchor points of a high-precision map to convert the spatial positioning data to physical mileage coordinates include: S111. Filter and cluster the three-dimensional point cloud data, and extract the boundary point of one corner of each vehicle as the three-dimensional laser space coordinates relative to the coordinate system of the lidar itself. S112. Continuously acquire pose data of the inspection drone, including three-dimensional coordinates and attitude angles, and map the three-dimensional laser space coordinates to three-dimensional space coordinates; S113. Retrieve the absolute three-dimensional anchor points of each lane line in the current road segment from the high-precision map, match the nearest lane line for the three-dimensional spatial coordinates of each vehicle, project the three-dimensional spatial coordinates onto the lane line along the normal direction, and extract the longitudinal arc length of the projection point on the corresponding lane line and the lateral offset of the three-dimensional spatial coordinates from the lane line to obtain the physical mileage coordinates of the corresponding vehicle.

4. The intelligent real-time accident early warning and proactive control method for smart highways based on big data as described in claim 1, characterized in that, The airborne sensor includes a camera and a solid-state lidar, both of which have active and inactive states.

5. The intelligent real-time accident early warning and proactive control method for smart highways based on big data as described in claim 1, characterized in that, The specific steps for calculating vehicle driving parameters and extracting traffic state features based on the aforementioned perception dataset include: S21. Track the corresponding vehicle through the target tracking algorithm, generate a continuous physical mileage coordinate time series of each vehicle, perform abnormal mutation preprocessing on the physical mileage coordinate time series, and use a fixed duration sliding window to perform moving average filtering to obtain a mileage coordinate sequence of each vehicle containing multiple smooth mileage coordinates. S22. Based on the mileage coordinate sequence of each vehicle, obtain the smoothed mileage coordinates of each vehicle at the current time and the previous time, and calculate the average lane speed and the speed of each vehicle at different times. Obtain smoothed mileage coordinates of adjacent vehicles in the same lane and calculate the longitudinal distance between vehicles at different times. Obtain the lane-changing trajectory of each vehicle as it crosses lane lines; Based on the vehicle speeds at different times, the frequency of emergency braking per unit time is calculated. S23. Integrate the vehicle speed, longitudinal distance between vehicles and lane change trajectory of each vehicle at different times, as well as the average speed of the lane at different times and the frequency of emergency braking per unit time to obtain traffic state characteristics.

6. The intelligent real-time accident early warning and proactive control method for smart highways based on big data as described in claim 1, characterized in that, The specific steps for acquiring environmental state features through inspection drones, inputting the traffic state features and environmental state features into a classification model, and outputting lane driving state labels include: S24. Based on the three-dimensional coordinates of the inspection drone, read the real-time rainfall and wind speed data of the weather station, and integrate the real-time rainfall, wind speed data, visibility value and water accumulation value through manually uploaded visibility and water accumulation values ​​to obtain environmental status characteristics. S25. Integrate the environmental state features and the traffic state features and input them into the pre-trained classification model to output a lane driving state label containing any one of free flow, saturated flow, congestion shock wave, and high-risk warning.

7. The intelligent real-time accident early warning and proactive control method for smart highways based on big data as described in claim 1, characterized in that, The steps for generating the bias coefficient include: dividing a certain highway segment into multiple discrete spatial grids according to a preset longitudinal mileage division; obtaining the traffic accident database of the corresponding highway segment within a preset accident reference year; calculating the total historical accident frequency of each spatial grid; calculating the average historical accident frequency and standard deviation of all spatial grids; calculating the risk heat index of each spatial grid; and mapping the risk heat index to obtain the bias coefficient through normalization.

8. The intelligent real-time accident early warning and proactive control method for smart highways based on big data as described in claim 1, characterized in that, The multi-level preliminary warning signal includes a corresponding warning level, a corresponding lane driving status label, and a corresponding risk mileage coordinate. The warning levels are Level 1 warning, Level 2 warning, and Level 3 warning, which are progressively more severe. When a Level 2 or Level 3 warning is issued, the inspection mission of the inspection drone is paused and it hovers, with the onboard sensors oriented toward the risk mileage coordinates.

9. The intelligent real-time accident early warning and proactive control method for smart highways based on big data as described in claim 8, characterized in that, The specific steps of multi-frame timing consistency verification include: establishing a sliding check window based on a preset sliding check number, obtaining the mode of the warning level within the sliding check window, and calculating the jump variance; If the variance of the jump is lower than or equal to the preset fluctuation threshold, or if the warning level first increases monotonically and then remains stable, the warning level corresponding to the mode is output as the final warning signal. Otherwise, the inspection drone is moved closer to the risk mileage coordinate, or the process is switched to manual remote review, or the highest warning level in the sliding verification window is used as the final warning signal.

Images