A high-risk road section identification system and method based on driving video

Abstract

The present invention discloses a high-risk road section identification system and method based on driving video, which belongs to the field of traffic safety technology. The method includes: using a drone to obtain driving video; performing registration processing on shaking frame images according to the obtained driving video; marking and continuously tracking vehicles according to the registered driving video to obtain continuous speed and driving trajectory data corresponding to the vehicle and time; calibrating lane lines, and extracting cross-sectional data of each lane line pile number according to the vehicle speed and driving trajectory data; obtaining vehicle driving characteristics including the lateral distance between the vehicle and the lane line, continuous acceleration, lateral acceleration and cross-sectional vehicle speed gradient according to the cross-sectional data; identifying high-risk road sections according to the vehicle driving characteristics, and completing the identification of high-risk road sections based on driving video. The present invention solves the problem that it is difficult to achieve stable, continuous and high-precision extraction of the distance between the vehicle and the intended lane line in the prior art.

Description

Technical Field

[0001] The present invention belongs to the technical field of traffic safety, and in particular relates to a high-risk road section identification system and method based on driving video. Background Art

[0002] At present, the evaluation of road safety is mainly based on the predicted running speed. The indicators of this method are too single, and the predicted running speed is difficult to represent the actual driving conditions. In scientific research, according to different data sources, it is mainly divided into several methods such as driving simulation, highway section data extraction, and data obtained by experimenters driving real vehicles. The data obtained by these methods have their limitations. It is impossible to obtain the continuous speed and trajectory data of multiple vehicles in the natural flow under the real environment, which leads to great limitations in the scientific phenomena reflected by such data, and it is difficult to accurately describe the real road conditions. At the same time, some studies use natural flow data sets such as NGSIM and HighD. The road environment of such data sets is too single and is not the road environment of my country, so it cannot represent the actual driving conditions in my country. Such data are often used in the research of following models, lane change models and macro traffic flow, and are not very practical in road safety analysis.

[0003] (1) There are relatively mature machine vision technologies that can identify vehicles in videos based on videos. In the first stage of this patent, vehicle identification technology based on Yolov5 is used to achieve the vehicle detection goal. Then Deepsort technology is used to achieve continuous vehicle tracking, which is also a relatively mature technology. However, this technology can only realize the extraction of vehicle speed and pixel coordinates at this stage, and cannot realize the real-time extraction of the lateral distance from the lane line. The semantic segmentation and recognition technology based on lane lines is difficult. Secondly, when there are occlusions, broken lines, and multiple lines overlap in the lane line, it is difficult to accurately calculate the continuous and accurate distance between the vehicle and the lane line to be judged. Based on the extraction of vehicle speed and coordinates in aerial videos based on deep learning, this technology originally uses Latitude software to realize lane line annotation, and then uses Python programming for data processing. It can realize the accurate calculation of the continuous distance and speed between any intended lane line and each vehicle, and is not affected by the occlusion, continuity, and overlap of the lane line.

[0004] (2) At present, the identification of high-risk sections of highways is mostly based on the judgment of pure linear shape, signs and other road conditions, or based on driving simulation data, or artificially arranged vehicles to drive on the spot to collect continuous actual driving speed trajectory data of a single or a small number of vehicles, or collect driving videos of multiple naturally driven vehicles at fixed sections of the road; there is almost no traffic safety research based on continuous high-precision driving videos of multiple vehicles under natural flow traffic conditions. At the same time, the speed and trajectory data used in the identification of high-risk sections of roads at this stage are the average speed or approximate trajectory of the section. This method establishes a new set of high-risk section identification indicators and identification methods through continuous high-precision speed and trajectory data. Summary of the invention

[0005] In view of the above-mentioned deficiencies in the prior art, the present invention provides a high-risk road section identification system and method based on driving video, which solves the problem in the prior art that it is difficult to extract the distance between the vehicle and the intended lane line stably, continuously and with high precision.

[0006] In order to achieve the above-mentioned purpose of the invention, the technical solution adopted by the present invention is: a method for identifying high-risk road sections based on driving videos, comprising the following steps:

[0007] S1. Use drones to obtain driving videos;

[0008] S2, taking the first frame of the driving video as a reference, performing registration processing on the shaking frame images;

[0009] S3. Mark and track the vehicle according to the registered driving video to obtain the continuous speed and driving trajectory data of the vehicle corresponding to time;

[0010] S4, calibrating the lane line, and extracting the cross-sectional data of each lane line number including the cross-sectional vehicle speed, acceleration, the lateral distance between the lane line and the vehicle, and the lateral speed according to the continuous speed and driving trajectory data of the vehicle;

[0011] S5. Obtaining vehicle driving characteristics including lateral distance between the vehicle and the lane line, continuous acceleration, lateral acceleration, and sectional vehicle speed gradient according to the cross-sectional data;

[0012] S6. Identify high-risk road sections according to the vehicle driving characteristics, and complete the identification of high-risk road sections based on the driving video.

[0013] The beneficial effects of the present invention are as follows: the high-risk road section identification method provided by the present invention includes the acquisition of video data with less jitter by a drone; the image registration based on the SIFT algorithm is the registration of the shaking frame picture, which ensures that the pixel coordinates of the main part of the video will not change, the relative position of the vehicle and the lane line will not change with the shaking of the lens, and the registration has no effect on the subsequent data collection. Compared with the current unregistered method, after registration, the trajectory error can be improved from 3m to about 20cm; vehicle data extraction based on deep learning, using Yolov5 to mark the vehicle and then continuously tracking the vehicle based on the Deepsort algorithm to obtain the vehicle's driving trajectory data. The deep learning-based method can stably achieve 100% vehicle detection and continuous tracking, with significantly better accuracy than vehicle detection methods under traditional optical technologies such as the inter-frame difference method and the optical flow method; then the lane lines are calibrated. This method can effectively and stably calibrate the lane lines, solving the problems of the current image-based lane line recognition algorithm, in which the recognition accuracy is seriously dependent on the lane line state, the error is large, and the algorithm is complex; then the cross-sectional speed and the lateral distance between the lane line and the vehicle are extracted based on the calibrated lane lines; according to data analysis and through indicators such as the distribution characteristics of the lateral distance to the lane line, continuous acceleration, lateral acceleration, and the gradient of the cross-sectional speed, high-risk sections can be identified, which can more effectively and accurately extract the continuous speed and trajectory data of multiple vehicles under the natural flow state; compared with the current methods based on cross-sectional speed measurement, data obtained from single-vehicle driving, and experience judgment, high-risk sections of the road can be identified more scientifically.

[0014] Furthermore, step S3 includes the following steps:

[0015] S301, marking the vehicle according to the registered driving video, and extracting a training set, a verification set and a test set from the registered driving video for training a vehicle detection model;

[0016] S302, calibrating the pixel distance and the actual distance according to the marking line of known length on the actual road;

[0017] S303, detecting the marked vehicle using a vehicle detection model;

[0018] S304, continuously tracking the detected vehicle using a multi-target tracking algorithm Deepsort;

[0019] S305: Based on the tracking result, the vehicle driving trajectory is obtained, and according to the calibration value of the actual distance and the pixel distance, the continuous speed of the vehicle corresponding to the time is obtained by calculating the time interval.

[0020] The beneficial effect of the above further scheme is that the present invention significantly improves the accuracy of vehicle trajectory by registering and stabilizing aerial video. Deep learning technology is used to mark vehicles and continuously track vehicles, thereby extracting high-precision vehicle speed and driving trajectory data, and the data accuracy and comprehensiveness are better than traditional optical methods and traditional cross-sectional speed measurement, single-vehicle driving and other vehicle data acquisition methods.

[0021] Furthermore, obtaining the calibrated lane line in step S4 includes the following steps:

[0022] A1. Extract an image from the registered driving video, import it into the CAD software with the upper left corner at the origin of the coordinates, and scale it so that the coordinate values ​​of the length and width of the image are the same as the pixel length of the image, that is, the CAD coordinates of any point in the image are the same as the pixel coordinates of the image in terms of X-axis coordinates, and the Y-axis coordinates are opposite;

[0023] A2. Design a design line that is the same as the lane line in the picture, and output a stake-by-stake coordinate table of the design line every 10 cm on the actual road;

[0024] A3. The Y-axis coordinate value of the pile-by-pile coordinate table is taken as a negative value, so that the lane line CAD coordinate table and the pixel coordinate of the lane line are the same;

[0025] A4. According to the coordinate calibration file of the lane line pixels, the calibration coordinate data of the lane line is obtained to complete the calibration of the lane line, wherein the calibration of the lane line is in the same coordinate system as the coordinate of the vehicle detection in step S3.

[0026] The beneficial effect of the above further solution is: obtaining the calibration file of the lane line to be studied, avoiding the problem that the lane line recognition based on image vision is highly dependent on the lane line state.

[0027] Furthermore, obtaining the cross-sectional vehicle speed in step S4 includes the following steps:

[0028] B1. Traverse the numbers of all vehicles and obtain the continuous pixel coordinates and speed values ​​of each vehicle in the video;

[0029] B2. Traverse each extracted section pile number of the calibrated lane line, and use each section pile number and continuous pixel coordinates to calculate the distance to obtain the two coordinate points with the minimum distance between the vehicle and the section. The two coordinate numbers are the collection points before and after the vehicle in the section;

[0030] B3. Calculate the distance between the two collection points before and after the vehicle on the cross section according to the two smallest coordinate numbers;

[0031] B4. Take the minimum two values ​​of the distance between the two collection points before and after the vehicle in the section, where the minimum two values ​​are the two positions before and after the i-th vehicle in the k-th section;

[0032] B5. Based on the speed value, the speed of the i-th vehicle at the k-th section is calculated according to the speeds of the two positions before and after the section and the distance difference between the two points of the section;

[0033] B6. Determine whether the speed of each vehicle in each section has been traversed. If so, calculate the speed of all vehicles in each section based on the speed of the i-th vehicle in the k-th section. Otherwise, return to step B1.

[0034] The beneficial effect of the above further solution is that by calculating the cross-sectional vehicle speed, the vehicle speed value of each vehicle passing through each stake position of the calibrated lane line is obtained.

[0035] Furthermore, in step B5, the speed of the i-th vehicle at the k-th section is expressed as follows:

[0036]

[0037]

[0038] Among them, v ik represents the speed of the i-th vehicle in the k-th section, v1 represents the speed of the collection point after the section, v2 represents the speed of the collection point before the section, I1 represents the horizontal distance between the section point and the collection point before the section, I'1 represents the vertical distance between the section point and the collection point before the section, I represents the horizontal distance between the collection points before and after the section, and I' represents the vertical distance between the collection points before and after the section.

[0039] Furthermore, obtaining the lane line and the lateral distance of the vehicle in step S4 includes the following steps:

[0040] C1, forming a triangle based on the two collection points before and after the vehicle and the cross section k;

[0041] C2. The height of the triangular section k is used as the lateral distance between the lane line and the vehicle at the section k.

[0042] Furthermore, the expression for forming a triangle in step C1 is as follows:

[0043]

[0044] Among them, l k represents the distance between the kth section and the jth collection point of the ith vehicle, represents the X coordinate of the kth section of the lane line, carx ij represents the X coordinate of the jth collection point of the i-th vehicle, represents the Y coordinate of the kth section of the lane line, cary ijIndicates the Y coordinate of the jth collection point of the i-th vehicle.

[0045] The present invention also provides a high-risk road section identification system based on driving video, comprising:

[0046] Aerial video acquisition module, used to acquire driving video using drones;

[0047] The image registration module is used to register the shaking frames based on the first frame of the driving video;

[0048] The vehicle data extraction module is used to mark and track the vehicle based on the registered driving video to obtain the continuous speed and driving trajectory data of the vehicle corresponding to time;

[0049] The cross-sectional driving video extraction module is used to calibrate the lane lines and extract the cross-sectional data of each lane line number including the cross-sectional vehicle speed, acceleration, the lateral distance between the lane line and the vehicle, and the lateral speed according to the continuous speed and driving trajectory data of the vehicle;

[0050] A data processing module, for obtaining vehicle driving characteristics including a lateral distance between the vehicle and the lane line, continuous acceleration, lateral acceleration and a sectional vehicle speed gradient according to the sectional data;

[0051] The analysis and identification module is used to identify high-risk road sections according to the vehicle driving characteristics and complete the high-risk road section identification based on the driving video.

[0052] The beneficial effects of the present invention are: using a drone to obtain driving video; performing registration processing on the shaking frame images according to the obtained driving video; using deep learning technology to mark and continuously track the vehicle according to the registered driving video, and obtaining the continuous speed and driving trajectory data corresponding to the vehicle and time; calibrating the lane line, and extracting the cross-sectional data of each pile number of the lane line according to the vehicle speed and driving trajectory data; obtaining the vehicle driving characteristics including the lateral distance between the vehicle and the lane line, continuous acceleration, lateral acceleration and cross-sectional vehicle speed gradient according to the cross-sectional data; identifying the high-risk road section according to the vehicle driving characteristics, and completing the identification of the high-risk road section based on the driving video. The present invention solves the problem that it is difficult to achieve stable, continuous and high-precision extraction of the distance between the vehicle and the intended lane line in the prior art, improves the accuracy and comprehensiveness of the real vehicle data extraction under the natural flow state, and increases the scientificity and reliability of the identification of high-risk road sections. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 The figure is a flow chart of the method of the present invention.

[0054] Figure 2 This is a schematic diagram of offset comparison of video acquisition according to the present invention.

[0055] Figure 3 This is a schematic diagram of the effect of video offset processing collected by the present invention.

[0056] Figure 4 This is a schematic diagram of the vehicle training frame selection of the present invention.

[0057] Figure 5 It is a schematic diagram of the vehicle marking inspection result of the present invention.

[0058] Figure 6 This is a flow chart of vehicle tracking and speed extraction of the present invention.

[0059] Figure 7 This is a schematic diagram of the conversion between the Cartesian coordinate system and the Frenet coordinate system used in the present invention.

[0060] Figure 8 This is a schematic diagram of the lane line marking in the video according to the present invention.

[0061] Fig. 9 It is a schematic diagram of extracting the vehicle lateral clearance according to the present invention.

[0062] Fig.10 This is a graph showing the results of extracting data on the vehicle speed and lateral distance relative to the lane line on the experimental section.

[0063] Fig.11 Schematic diagram of steering obstacle avoidance in this embodiment.

[0064] Fig.12 The figure is a schematic diagram of the area enclosed by the actual vehicle speed and the average vehicle speed within the mileage in this embodiment.

[0065] Fig.13 It is a schematic diagram of the system structure of the present invention. DETAILED DESCRIPTION

[0066] The specific implementation modes of the present invention are described below so that those skilled in the art can understand the present invention. However, it should be clear that the present invention is not limited to the scope of the specific implementation modes. For those of ordinary skill in the art, as long as various changes are within the spirit and scope of the present invention as defined and determined by the attached claims, these changes are obvious, and all inventions and creations utilizing the concept of the present invention are protected.

[0067] Example 1

[0068] like Figure 1 As shown, the present invention provides a method for identifying high-risk road sections based on driving videos, and the implementation method is as follows:

[0069] S1. Use drones to obtain driving videos;

[0070] In this embodiment, Figure 2 As shown in the figure, a professional DJI drone is used to collect videos by vertically shooting at a fixed point at an altitude of 200 to 250 meters. This ensures that a sufficient length of road is captured and that vehicles block the drone's shooting angle. On the other hand, the shooting range should not be too large, the vehicle should not be too small, and the features should not disappear, which will affect the accuracy of vehicle recognition in the later machine vision processing.

[0071] S2, taking the first frame of the driving video as a reference, performing registration processing on the shaking frame images;

[0072] In this embodiment, Figure 2 As shown in the figure, due to the influence of wind when capturing video from drones, the drone will shake slightly, and the lens image will be offset in the horizontal range, resulting in significant errors in trajectory extraction. SIFT algorithm is used to perform affine transformation through Python programming to process the original video. The reference video is f(x,y), and the image to be registered is g(x,y). In addition, the points on the reference image are In the image to be registered, it is (x, y), k is the scale parameter, θ is the rotation angle, and Δx and Δy are the translation amounts of the two coordinate axes respectively. Figure 3 As shown in the figure, according to the SIFT algorithm, the road can be corrected to the same position as the first frame based on more than 4 feature points. Due to image registration, the picture has a little black edge, which makes the image consistent with the first frame, ensuring that the pixel coordinates of the main part of the video will not change, and the relative position of the vehicle and the lane line will not change with the camera shake. The camera shake is very small, and the black edge generated by the registration is acceptable, and it has no effect on subsequent data collection.

[0073] S3. Mark and track the vehicle according to the registered driving video to obtain the continuous speed and driving trajectory data of the vehicle corresponding to time. The implementation method is as follows:

[0074] S301, marking the vehicle according to the registered driving video, and extracting a training set, a verification set and a test set from the registered driving video for training a vehicle detection model;

[0075] S302, calibrating the pixel distance and the actual distance according to the marking line of known length on the actual road;

[0076] S303, detecting the marked vehicle using a vehicle detection model;

[0077] S304, continuously tracking the detected vehicle using a multi-target tracking algorithm Deepsort;

[0078] S305: Based on the tracking result, the vehicle driving trajectory is obtained, and according to the calibration value of the actual distance and the pixel distance, the continuous speed of the vehicle corresponding to the time is obtained by calculating the time interval.

[0079] In this embodiment, Figure 4 As shown in the figure, a vehicle labeling program is used to perform vehicle selection and calibration on a large number of images used for training and video analysis. The model is trained with a ratio of training set: validation set: test set = 0.7:0.2:0.1, and a total of 100 rounds of training are performed. The model with the best test results is used for vehicle detection. Figure 5 As shown, the vehicle detection effect can reach 100%, and the detection frame is stable and not affected by shadows.

[0080] In this embodiment, for the continuous tracking of the detected vehicle, based on the Deepsort algorithm, the calibration value of the pixel distance and the actual distance can be determined, and the speed and acceleration information can be collected according to the time interval. Figure 6 As shown in the figure, the vehicle trajectory is obtained after cascade matching and IOU matching of the detection target. By calculating the time interval, the vehicle displacement, speed, acceleration and other information are obtained, thereby obtaining vehicle tracking and speed extraction.

[0081] In this embodiment, in order to ensure the accuracy of the aerial video data extraction, the speed extracted by the program needs to be checked for accuracy. Randomly select 9 large and small cars, play them frame by frame, and record the time they pass the dotted line on the road. Because the actual length of the dotted line is known to be 6m, the formula The actual speed of the vehicle here is calculated. From the comparison of the accuracy of the actual speed value and the detection value data, it is found that there is data with a detection value error of 0. The analysis of the maximum error and the overall accuracy shows that the speed data extracted by machine vision technology has an accuracy of more than 95% for large vehicles and more than 98% for small vehicles, which meets the actual engineering requirements.

[0082] S4, calibrating the lane line, and extracting the cross-sectional data of each lane line number including the cross-sectional vehicle speed, acceleration, the lateral distance between the lane line and the vehicle, and the lateral speed according to the continuous speed and driving trajectory data of the vehicle;

[0083] The lane marking in step S4 includes the following steps:

[0084] A1. Extract an image from the registered driving video, import it into the CAD software with the upper left corner at the origin of the coordinates, and scale it so that the coordinate values ​​of the length and width of the image are the same as the pixel length of the image, that is, the CAD coordinates of any point in the image are the same as the pixel coordinates of the image in terms of X-axis coordinates, and the Y-axis coordinates are opposite;

[0085] A2. Design a design line that is the same as the lane line in the picture, and output a stake-by-stake coordinate table of the design line every 10 cm on the actual road;

[0086] A3. The Y-axis coordinate value of the pile-by-pile coordinate table is taken as a negative value, so that the lane line CAD coordinate table and the pixel coordinate of the lane line are the same;

[0087] A4. Obtain the calibration coordinate data of the lane line according to the coordinate calibration file of the lane line pixels, and complete the calibration of the lane line, wherein the calibration of the lane line is in the same coordinate system as the coordinate of the vehicle detection in step S3;

[0088] The step S4 of obtaining the cross-sectional vehicle speed includes the following steps:

[0089] B1. Traverse the numbers of all vehicles and obtain the continuous pixel coordinates and speed values ​​of each vehicle in the video;

[0090] B2. Traverse each extracted section pile number of the calibrated lane line, and use each section pile number and continuous pixel coordinates to calculate the distance to obtain the two coordinate points with the minimum distance between the vehicle and the section. The two coordinate numbers are the collection points before and after the vehicle in the section;

[0091] B3. Calculate the distance between the two collection points before and after the vehicle on the cross section according to the two smallest coordinate numbers;

[0092] B4. Take the minimum two values ​​of the distance between the two collection points before and after the vehicle in the section, where the minimum two values ​​are the two positions before and after the i-th vehicle in the k-th section;

[0093] B5. Based on the speed value, the speed of the i-th vehicle at the k-th section is calculated according to the speeds of the two positions before and after the section and the distance difference between the two points of the section;

[0094] B6, determine whether the speed of each vehicle in each section has been traversed. If so, calculate the speed of all vehicles in each section based on the speed of the i-th vehicle in the k-th section. Otherwise, return to step B1;

[0095] The step S4 of obtaining the lane line and the lateral distance of the vehicle comprises the following steps:

[0096] C1, forming a triangle based on the two collection points before and after the vehicle and the cross section k;

[0097] C2. The height of the triangular section k is used as the lateral distance between the lane line and the vehicle at the section k.

[0098] In this embodiment, based on the obtained vehicle data, the following processing is performed to obtain the calibrated lane line, cross-sectional vehicle speed, and lateral distance between the lane line and the vehicle for evaluating the high-risk section of the accident. The specific contents are as follows:

[0099] The method to obtain the calibrated lane lines is as follows: Using the Cartesian coordinate system to describe the position of an object is not the best choice to describe the position of a vehicle, such as Figure 7 As shown in the figure, the Cartesian coordinate system is converted to the Frenet coordinate system to better describe the state of the lane line and the vehicle on the road. In order to obtain the lateral distance between the vehicle and the lane line, in addition to converting the coordinate system, the lane line needs to be marked. For roads with constantly changing linear shapes, in addition to obtaining the vehicle position (x, y), the position relationship between the vehicle and the lane line is also required. Figure 7 As shown, the Frenet coordinate system is established, s represents the distance along the road as the ordinate, and d represents the distance from the longitudinal line as the abscissa ( Figure 7 There is no parameter d) in order to achieve a specific description of the position of the vehicle on the lane. Given the continuous coordinate position of the vehicle, the lane line needs to be marked to obtain the distance between the vehicle and the lane line. Take a picture from each fixed-point aerial video. Because the drone is a fixed-point aerial photography, it can be assumed that the background except the vehicle does not change between video frames. This picture represents the road position of this aerial video. Import the picture into the CAD software with the upper left corner located at the coordinate origin. By scaling, make the coordinate values ​​of the length and width of the picture equal to the pixel length, with the horizontal length being 1920 and the vertical length being 1080. Under this condition, the coordinates of any point in the figure in the CAD software correspond to the pixel coordinates of the image, which are the same as the extracted vehicle X coordinate, and the Y coordinate is the opposite number. If Figure 8 As shown, through the graphic design function of the Weidi software, a design line identical to the lane line in the picture is designed. In this project, the pile-by-pile coordinates of the design line on the actual road can be output each time at an interval of 10 cm. The Y-axis coordinate value of the pile-by-pile coordinate table is negative, which is the pixel coordinate calibration file of the lane line. Its coordinates are in the same coordinate system as the detected vehicle coordinates.

[0100] The method for obtaining the vehicle speed of each section of the lane line is as follows: According to the continuously collected coordinates and corresponding speed values ​​of each vehicle, as well as the calibration file of the lane line, the information of each vehicle is obtained, and the section speed is extracted using Python programming. A total of three loops are performed. First, the numbers of all vehicles are traversed to obtain the continuous pixel coordinates and speed values ​​of each vehicle in the video; then each section pile number to be extracted of the calibrated lane line is traversed. The total length of the ramp in this article is 2.5km. The section is collected every 1m. The distance is calculated using each section pile number and all the coordinates of the numbered vehicles extracted in the previous layer, and the two coordinate points with the smallest distance between the vehicle and the section are found, that is, the two collection points before and after the vehicle in this section, and l is taken. kThe minimum two values ​​are the two positions before and after the i-th vehicle in the k-th section. Because the vehicle collection time is very short, it can be considered that the vehicle is moving at a uniform speed between these two collection points. According to the speed of the two places and the distance difference between the two points of the section, the speed v of the i-th vehicle in the k-th section is obtained. ik :

[0101]

[0102]

[0103] Among them, v ik represents the speed of the i-th vehicle in the k-th section, v1 represents the speed of the collection point after the section, v2 represents the speed of the collection point before the section, I1 represents the horizontal distance between the section point and the collection point before the section, I'1 represents the vertical distance between the section point and the collection point before the section, I represents the horizontal distance between the collection points before and after the section, and I' represents the vertical distance between the collection points before and after the section.

[0104] In this embodiment, the horizontal distance is used for calculation. If the distance is not between the horizontal distances, the distance between the vertical distances is used for calculation.

[0105] By traversing each vehicle and each section in this way, the speed of each vehicle in each section can be obtained. Similarly, the speed of all vehicles in each section can also be obtained.

[0106] The method for obtaining the lateral distance between the lane line and the vehicle is as follows: in the triangle formed by two acquisition points and section k, the coordinates of the three points are known, and the lengths of the three sides of the triangle can be obtained. The height h of the triangle section k is the lateral distance between the lane line and the vehicle at section k. To obtain the vertical distance h of each vehicle from the lane line at each section, Heron's formula can be applied. By applying this method to each lane line section, the lateral distance between each lane line section and each vehicle can be obtained, which is the offset of the vehicle.

[0107] The expression for forming a triangle is as follows:

[0108]

[0109] Among them, l k represents the distance between the kth section and the jth collection point of the ith vehicle, represents the X coordinate of the kth section of the lane line, carx ij represents the X coordinate of the jth collection point of the i-th vehicle, represents the Y coordinate of the kth section of the lane line, cary ij Indicates the Y coordinate of the jth collection point of the i-th vehicle.

[0110] like Fig. 9As shown, in the process of extracting the lateral distance data, the coordinates of the three points are known, and the lengths of the three sides of the triangle can be obtained. The height h of the triangle section k is the lateral distance between the lane line and the vehicle at section k. To obtain the vertical distance h between each vehicle and the lane line at each section, Heron's formula can be applied. The height h of the known triangle established in this example can be obtained by the formula By applying this method to each lane line section, the lateral distance between each lane line section and the vehicle can be obtained, which is the offset of the vehicle.

[0111] S5, such as Fig.10 As shown, according to the calibrated lane line, cross-sectional vehicle speed, and lateral distance between the lane line and the vehicle, the continuous speed and lateral distance distribution of the vehicle relative to the lane line are obtained, so as to analyze and obtain the lateral distance distribution characteristics with the lane line, the continuous acceleration, the lateral acceleration, and the gradient of the cross-sectional vehicle speed;

[0112] S6. According to the distribution characteristics of the lateral distance from the lane line, continuous acceleration, lateral acceleration and the gradient of the cross-sectional vehicle speed, the high-risk road section is identified, and the high-risk road section identification based on the driving video is completed.

[0113] In this embodiment, the analysis obtained through processing obtains the distribution characteristics of the lateral distance to the lane line, the continuous acceleration, the lateral acceleration, and the gradient of the cross-sectional vehicle speed. According to the distribution characteristics of the lateral distance to the lane line, the continuous acceleration, the lateral acceleration, and the gradient of the cross-sectional vehicle speed, the high-risk road section is identified, and the high-risk road section identification based on the driving video is completed. The evaluation index and method are as follows:

[0114] 1. Transverse distance to the guardrail, that is, distribution characteristics of the transverse distance to the lane line

[0115] During braking, the vehicle sometimes veers to the sides, causing braking deviation, which can lead to the vehicle losing control and deviating from the original direction of travel. If the front wheels lock first, the vehicle loses its steering ability and experiences tailspin; if the rear wheels lock first, the vehicle deviates from the original lane and drives into the adjacent lane or collides with the guardrail. Especially when driving at a bend, there is longitudinal force on all four wheels, and braking intensifies and changes the vehicle load, thus affecting the turning stability.

[0116] According to the braking deviation and driving speed of the vehicle, when the speed is 50km / h, the lateral safety distance is 1m; when the speed is 120km / h, the lateral safety distance is 1.4m. Therefore, the lateral safety distance between the vehicle and the guardrail on the highway is determined to be 1.0~1.4m.

[0117] 2. Continuous vehicle acceleration

[0118] During normal driving on the highway, the vehicle speed and acceleration are constantly fluctuating. When the speed fluctuates around 110km / h, the acceleration fluctuation range is [-1.5, 1.5]. The vehicle deceleration changes dramatically during emergency braking. If an emergency stop is made, the deceleration can quickly reach -6m / s 2 When the vehicle deceleration is greater than the normal deceleration used by the driver during normal driving, it will make the driver and passengers feel uncomfortable. Therefore, it is very important to use the deceleration as a basis for judging whether the driver's driving safety status is abnormal.

[0119] 3. Vehicle lateral acceleration

[0120] like Fig.11 As shown in the figure, the vehicle lateral acceleration (a y ) often reflects the acceleration perpendicular to the vehicle's forward trajectory generated when the vehicle turns. If the turning time is too short, the vehicle will slip or roll over, and the vehicle stability will deteriorate. Therefore, a graded evaluation standard for lateral acceleration is given, as shown in Table 1. Table 1 is the graded evaluation standard for lateral acceleration.

[0121] Table 1

[0122] level definition Value range Normal Lower lateral acceleration <![CDATA[0≤a y <0.02g]]> Stronger Moderate lateral acceleration <![CDATA[0.02g≤a y <0.22g]]> Restricted Higher lateral acceleration <![CDATA[0.22g≤a y <0.5g]]> Maximum level Very high lateral acceleration <![CDATA[0.5g≤a y <0.64g]]>

[0123] In the table, a y is the lateral acceleration of the vehicle, g is the acceleration due to gravity, g = 9.8 m / s 2 , the lateral acceleration obtained by analysis is compared with the table to obtain the lateral acceleration classification. The restricted and maximum lateral accelerations need to be screened out separately, and the road sections corresponding to the lateral accelerations may be potential high-risk sections.

[0124] 4. Speed ​​difference, i.e. the gradient of the speed of the section

[0125] On the highway, small cars have higher speeds, while large cars have lower speeds. Due to their relatively large size and mass, large cars are far inferior to small cars in terms of power performance. Therefore, if large cars run too slowly, not only will the following vehicles have insufficient sight distance, but they will also affect the operating safety of overtaking small cars. The difference between the speeds of large and small cars is defined as the speed difference between large and small cars. Through the investigation of section speeds, the speed difference between large and small cars in section speeds is obtained, that is, the speed difference between large and small cars in section speeds = small car V85 - large car V85. In this way, the speed difference between large and small cars in each section of the road section is obtained. By extracting the maximum and average values ​​of the speed difference between large and small cars in each section of the road section, the correlation between them and the accident rate of the road section is analyzed. Among them, the calculation method of the extreme difference ratio of the speed difference between large and small cars in the road section is as follows:

[0126]

[0127] Among them, mb represents the speed difference ratio of large and small vehicles on the road section, Indicates the average speed difference of different types of vehicles in each section, km / h, m i Indicates the speed difference between large and small vehicle types at each section of the road, km / h.

[0128] As shown in Table 2, Table 2 is the evaluation standard for the speed difference range ratio of large and small vehicle models.

[0129] Table 2

[0130]

[0131] The speed dispersion of a road section describes the degree of dispersion of the vehicle speed within the road section relative to the average speed of the entire highway. Since this degree of dispersion is related to the various linear indicators of the road section, the speed dispersion is closely related to the division of the road section.

[0132] The speed gradient is defined as the difference ΔV between the characteristic speed of the section or the representative speed of the road section and the average speed of the highway. The calculation formula is as follows:

[0133]

[0134] Among them, V 85 represents the 85th percentile velocity, Indicates the average highway speed.

[0135] The speed dispersion of a road section is the area S enclosed by the speed of the first vehicle and the average speed within the mileage, as Fig.12 As shown, the calculation formula is as follows:

[0136]

[0137] Where, ΔV i Indicates the velocity gradient per unit length, m / s, l i+1 Indicates the starting pile number mileage corresponding to the i+1th geometric element, m, l i It represents the terminal pile number mileage corresponding to the i-th geometric element, m, L represents the mileage length, m, n represents the total number of geometric elements within the mileage length, and dl represents the integral sign.

[0138] Through the above design, the present invention can more effectively and accurately extract the continuous speed and trajectory data of multiple vehicles under the natural flow state; and can more scientifically identify high-risk road sections.

[0139] Example 2

[0140] like Fig.13 As shown, the present invention provides a high-risk road section identification system based on driving video, comprising:

[0141] Aerial video acquisition module, used to acquire driving video using drones;

[0142] An image registration module is used to perform registration processing on shaking frames according to the acquired driving video;

[0143] The vehicle data extraction module is used to mark and track the vehicle based on the registered driving video to obtain the continuous speed and driving trajectory data of the vehicle corresponding to time;

[0144] The cross-sectional driving video extraction module is used to calibrate the lane lines and extract the cross-sectional data of each lane line number including the cross-sectional vehicle speed, acceleration, the lateral distance between the lane line and the vehicle, and the lateral speed according to the continuous speed and driving trajectory data of the vehicle;

[0145] A data processing module, for obtaining vehicle driving characteristics including a lateral distance between the vehicle and the lane line, continuous acceleration, lateral acceleration and a sectional vehicle speed gradient according to the sectional data;

[0146] The analysis and identification module is used to identify high-risk road sections according to the vehicle driving characteristics and complete the high-risk road section identification based on the driving video.

[0147] like Fig.13 The point cloud registration system provided in the illustrated embodiment can execute the technical solution shown in the method for identifying high-risk road sections based on driving videos in the above-mentioned method embodiment, and its implementation principle and beneficial effects are similar and will not be repeated here.

[0148] In an embodiment of the present invention, the present application may divide the functional units according to the high-risk road section identification method based on driving video. For example, each function may be divided into each functional unit, or two or more functions may be integrated into one processing unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units. It should be noted that the division of units in the present invention is schematic and is only a logical division. There may be other division methods in actual implementation.

[0149] In the embodiment of the present invention, the high-risk road section identification system based on driving video includes hardware structures and / or software modules corresponding to the execution of various functions in order to realize the principles and beneficial effects of the high-risk road section identification method based on driving video. Those skilled in the art should easily realize that, in combination with the various schematic units and algorithm steps described in the embodiments disclosed in the present invention, the present invention can be implemented in the form of hardware and / or a combination of hardware and computer software. Whether a function is executed in a hardware or computer software driven manner depends on the specific application and design constraints of the technical solution. Different methods can be used for each specific application to implement the described function, but such implementation should not be considered to exceed the scope of this application.

[0150] In this embodiment, the present invention uses a drone to obtain driving video; uses image registration to register the shaking frame images; marks and tracks the vehicle to obtain the vehicle's driving trajectory data; extracts the cross-sectional driving video to obtain the calibrated lane line, cross-sectional vehicle speed, and the lateral distance between the lane line and the vehicle; analyzes the distribution characteristics of the lateral distance to the lane line, continuous acceleration, lateral acceleration, and the gradient of the cross-sectional vehicle speed, and identifies high-risk sections to complete the identification of high-risk sections based on driving video. The present invention solves the problem that it is difficult to achieve stable, continuous, and high-precision extraction of the distance between the vehicle and the intended lane line in the prior art.

[0151] The above-described embodiments are only specific implementations of the present invention, which are used to illustrate the technical solutions of the present invention, rather than to limit them. The protection scope of the present invention is not limited thereto. Although the present invention is described in detail with reference to the above-described embodiments, it should be understood by those skilled in the art that any person familiar with the technical field can still modify the technical solutions recorded in the above-described embodiments within the technical scope disclosed by the present invention, or can easily think of changes, or perform equivalent replacements on some of the technical features thereof; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention, and should all be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims

1. A method for identifying high-risk road sections based on driving videos. It is characterized in that The following steps are involved: S1. Use drones to obtain driving videos; S2, taking the first frame of the driving video as a reference, performing registration processing on the shaking frame images; S3. Mark and track the vehicle according to the registered driving video to obtain the continuous speed and driving trajectory data of the vehicle corresponding to time; S4, calibrating the lane line, and extracting the cross-sectional data of each lane line number including the cross-sectional vehicle speed, acceleration, the lateral distance between the lane line and the vehicle, and the lateral speed according to the continuous speed and driving trajectory data of the vehicle; S5. Obtaining vehicle driving characteristics including lateral distance between the vehicle and the lane line, continuous acceleration, lateral acceleration, and sectional vehicle speed gradient according to the cross-sectional data; S6. Identify high-risk road sections according to the vehicle driving characteristics, and complete the identification of high-risk road sections based on the driving video.

2. The high-risk road section identification method based on driving video according to claim 1, It is characterized in that The step S3 comprises the following steps: S301, marking the vehicle according to the registered driving video, and extracting a training set, a verification set and a test set from the registered driving video for training a vehicle detection model; S302, calibrating the pixel distance and the actual distance according to the marking line of known length on the actual road; S303, detecting the marked vehicle using a vehicle detection model; S304, continuously tracking the detected vehicle using a multi-target tracking algorithm Deepsort; S305: Based on the tracking result, the vehicle driving trajectory is obtained, and according to the calibration value of the actual distance and the pixel distance, the continuous speed of the vehicle corresponding to the time is obtained by calculating the time interval.

3. The high-risk road section identification method based on driving video according to claim 1, It is characterized in that The lane marking in step S4 includes the following steps: A1. Extract an image from the registered driving video, import it into the CAD software with the upper left corner at the origin of the coordinates, and scale it so that the coordinate values ​​of the length and width of the image are the same as the pixel length of the image, that is, the CAD coordinates of any point in the image are the same as the pixel coordinates of the image in terms of X-axis coordinates, and the Y-axis coordinates are opposite; A2. Design a design line that is the same as the lane line in the picture, and output a stake-by-stake coordinate table of the design line every 10 cm on the actual road; A3. The Y-axis coordinate value of the pile-by-pile coordinate table is taken as a negative value, so that the lane line CAD coordinate table and the pixel coordinate of the lane line are the same; A4. According to the coordinate calibration file of the lane line pixels, the calibration coordinate data of the lane line is obtained to complete the calibration of the lane line, wherein the calibration of the lane line is in the same coordinate system as the coordinate of the vehicle detection in step S3.

4. The high-risk road section identification method based on driving video according to claim 3, It is characterized in that The step S4 of obtaining the cross-sectional vehicle speed includes the following steps: B1. Traverse the numbers of all vehicles and obtain the continuous pixel coordinates and speed values ​​of each vehicle in the video; B2. Traverse each extracted section stake number of the calibrated lane line, and use each section stake number and continuous pixel coordinates to calculate the distance to obtain the two coordinate points with the minimum distance between the vehicle and the section, and the two coordinate points are the two collection points before and after the vehicle in the section; B3. Calculate the distance between the two collection points before and after the vehicle in the section according to the two coordinate points with the smallest distance; B4. Take the minimum two values ​​of the distance between the two collection points before and after the vehicle in the section, where the minimum two values ​​are the two positions before and after the i-th vehicle in the k-th section; B5. Based on the speed value, the speed of the i-th vehicle at the k-th section is calculated according to the speeds of the two positions before and after the section and the distance difference between the two points of the section; B6. Determine whether the speed of each vehicle in each section has been traversed. If so, calculate the speed of all vehicles in each section based on the speed of the i-th vehicle in the k-th section. Otherwise, return to step B1.

5. The high-risk road section identification method based on driving video according to claim 4, It is characterized in that The expression of the speed of the i-th vehicle at the k-th section in step B5 is as follows: in, represents the speed of the i-th vehicle at the k-th section, Indicates the vehicle speed at the point of collection after the section, Indicates the vehicle speed at the collection point before the section, Indicates the horizontal distance between the cross-section point and the sampling point before the cross-section. Indicates the vertical distance between the cross-section point and the acquisition point before the cross-section. Indicates the horizontal distance between the front and rear acquisition points of the section. Indicates the vertical distance between the front and rear collection points of the section.

6. The high-risk road section identification method based on driving video according to claim 1, It is characterized in that The step S4 of obtaining the lane line and the lateral distance of the vehicle comprises the following steps: C1, forming a triangle based on the two collection points before and after the vehicle and the cross section k; C2. The height of the triangular section k is used as the lateral distance between the lane line and the vehicle at the section k.

7. The high-risk road section identification method based on driving video according to claim 6, It is characterized in that The expression for forming a triangle in step C1 is as follows: in, represents the distance between the kth section and the jth collection point of the ith vehicle, represents the X coordinate of the kth section of the lane line, represents the X coordinate of the jth collection point of the i-th vehicle, represents the Y coordinate of the kth section of the lane line, Indicates the Y coordinate of the jth collection point of the i-th vehicle.

8. A high-risk road section identification system based on driving video, It is characterized in that include: Aerial video acquisition module, used to acquire driving video using drones; The image registration module is used to register the shaking frames based on the first frame of the driving video; The vehicle data extraction module is used to mark and track the vehicle based on the registered driving video to obtain the continuous speed and driving trajectory data of the vehicle corresponding to time; The cross-sectional driving video extraction module is used to calibrate the lane lines and extract the cross-sectional data of each lane line number including the cross-sectional vehicle speed, acceleration, the lateral distance between the lane line and the vehicle, and the lateral speed according to the continuous speed and driving trajectory data of the vehicle; A data processing module, for obtaining vehicle driving characteristics including a lateral distance between the vehicle and the lane line, continuous acceleration, lateral acceleration and a sectional vehicle speed gradient according to the sectional data; The analysis and identification module is used to identify high-risk road sections according to the vehicle driving characteristics and complete the high-risk road section identification based on the driving video.

Images