A method and system for all-weather detection of traffic incidents based on mobile sensing devices

Abstract

The application discloses a kind of traffic event all-weather detection method and system based on movable sensing device, it is related to highway traffic detection technical field, comprising the following steps: based on RTK technology and video lane recognition algorithm, millimeter wave radar equipment and camera equipment are automatically calibrated, so that the detection area of millimeter wave radar equipment and camera equipment is consistent;Radar data and video image data are obtained;Based on radar data and video image data, all-weather determination of highway traffic event is carried out.The application can realize the automatic calibration of equipment to highway lane line under different positions, improve the flexibility and all-weather intelligent detection capability of roadside point of highway traffic event detection.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of highway traffic detection, more particularly to a traffic event all-weather detection method and system based on a movable sensing device. BACKGROUND

[0002] Traffic events are the key monitoring objects of highway operation management. At present, the intelligent detection method for highway traffic events generally involves constructing fixed vertical poles on highways and installing monitoring cameras with intelligent analysis capabilities on the vertical poles to identify highway traffic events, including parking events, pedestrian events, congestion events, reverse events, and littering events.

[0003] The existing traffic event detection method simply relies on video detection, which is easily affected by insufficient light conditions such as night, rainy days, snowy days, and foggy days. Combining the all-weather detection characteristics of radar equipment, highway traffic events can be detected all-weather through the cooperation of radar equipment and camera equipment. However, whether it is a camera or a video intelligent detection, when the device position is fixed, it needs to be calibrated to the specific lane position. When facing temporary monitoring and detection and other types of movable sensing devices, repeated manual calibration will increase the workload.

[0004] Therefore, how to improve the flexibility of the roadside point of the highway traffic event detection and the all-weather intelligent detection capability is a problem that needs to be solved by those skilled in the art. SUMMARY

[0005] Therefore, the present application provides a traffic event all-weather detection method and system based on a movable sensing device, which overcomes the above-mentioned defects.

[0006] To achieve the above-mentioned purpose, the present application provides the following technical solutions:

[0007] A traffic event all-weather detection method based on a movable sensing device, comprising the following steps:

[0008] Based on RTK technology and video lane recognition algorithm, the millimeter wave radar equipment and the camera equipment are automatically calibrated to make the detection areas of the millimeter wave radar equipment and the camera equipment consistent;

[0009] Obtain radar data and video image data;

[0010] Based on the radar data and the video image data, the highway traffic events are all-weather determined.

[0011] Optionally, when the camera equipment is automatically calibrated, the lane lines are identified by training images through AI data sets in combination with local videos.

[0012] Optionally, when the millimeter wave radar device is automatically calibrated, the carrier phase difference technology is used for automatic calibration, specifically as follows:

[0013] When the movable sensing device is placed on the roadside for work, the RTK device is used to obtain the position of the movable sensing device; according to the positions of the two points, the angle of the horizontal rod of the support deviating from the lane is calculated, and the deviated angle is fed back to the millimeter wave radar device; the millimeter wave radar device adjusts the direction of the transmitting beam so that the beam direction is parallel to the lane direction.

[0014] Optionally, the radar data includes position information, speed information and acceleration information of each vehicle.

[0015] Optionally, after obtaining the radar data and the video image data, the video image data is used to calibrate the radar data, specifically as follows:

[0016] After the millimeter wave radar device detects the vehicle on the road, the corresponding lane on the host computer end is displayed; through the synchronous detection of the camera device and the millimeter wave radar device, auxiliary calibration is completed.

[0017] Optionally, the highway traffic event is a parking event, and the identification method is specifically as follows:

[0018] The target detection network processes visual information to find potential abnormal vehicle targets;

[0019] Through the millimeter wave radar device, the acceleration of the abnormal vehicle target is detected;

[0020] If the acceleration of the abnormal vehicle target exceeds the acceleration threshold, there is a traffic accident event, otherwise it is a illegal parking event or an emergency lane parking event;

[0021] If the acceleration of the abnormal vehicle target is lower than the acceleration threshold, the position of the abnormal vehicle target in the emergency lane area is determined through the video image data; if it is in the emergency lane area, it is an emergency lane parking event, otherwise it is an illegal parking event.

[0022] Optionally, the highway traffic event is a reverse event, a congestion event, and a speeding event, and the identification method is specifically as follows:

[0023] The millimeter wave radar device is used to collect vehicle speed information;

[0024] Determine whether the direction of the millimeter wave radar device detecting the vehicle speed information is consistent with the driving direction, if the vehicle speed information is consistent with the driving direction within a preset time, it is normal driving, otherwise it is a reverse event;

[0025] Determine whether there are more than a preset number of vehicle speed information below the speed threshold, if so, it is a congestion event, otherwise it is normal.

[0026] determining whether the vehicle speed information exceeds a speed limit value of the current road section, if yes, it is a speeding event, otherwise, it is normal.

[0027] Optionally, the highway traffic event is a pedestrian event or a litter event, and the identification method is specifically:

[0028] processing the visual information through a target detection network to find potential pedestrian targets and litter targets;

[0029] if no pedestrian target appears in the detection range, it is normal, otherwise, it is a pedestrian event;

[0030] if no litter target appears in the detection range, it is normal, otherwise, it is a litter event.

[0031] A traffic event all-weather detection system based on a movable sensing device, comprising:

[0032] a calibration module configured to calibrate the millimeter wave radar device and the camera device automatically through RTK technology and a video lane recognition algorithm, so that the detection ranges of the millimeter wave radar device and the camera device are consistent;

[0033] a data acquisition module configured to acquire radar data and video image data;

[0034] an event identification module configured to determine the highway traffic event based on the radar data and the video image data all-weather.

[0035] According to the above technical solution, compared with the prior art, the present disclosure provides a traffic event all-weather detection method and system based on a movable sensing device, which can realize automatic calibration of the movable sensing device, automatic calibration of the lane line in the detection range after position movement, and automatic locking of the detection range of the traffic event, thereby eliminating the huge workload of manual calibration; the radar data and the video image data can be effectively combined, and the all-weather detection of the highway traffic event can be realized. BRIEF DESCRIPTION OF DRAWINGS

[0036] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments or the prior art description will be briefly introduced as follows. Obviously, the drawings in the following description are only embodiments of the present application, and those skilled in the art can obtain other drawings according to the provided drawings without creative labor.

[0037] Figure 1 The principle diagram of the traffic event all-weather detection method based on the movable sensing device provided by the present application;

[0038] Figure 2A schematic diagram of automatic calibration of the millimeter wave radar device provided by the present application is shown in the figure.

[0039] Figure 3 A logic architecture diagram of automatic calibration of the millimeter wave radar device and the camera device provided by the present application is shown in the figure.

[0040] Figure 4(a) is a flow chart for determining a parking event provided by the present application.

[0041] Figure 4(b) is a flow chart for determining a parking violation event / shoulder lane parking event provided by the present application.

[0042] Figure 5 A flow chart for determining a reverse event provided by the present application is shown in the figure.

[0043] Figure 6 A flow chart for determining a congestion event provided by the present application is shown in the figure.

[0044] Figure 7 A flow chart for determining a pedestrian event provided by the present application is shown in the figure.

[0045] Figure 8 A flow chart for determining a littering event provided by the present application is shown in the figure.

[0046] Figure 9 A flow chart for determining a speeding event provided by the present application is shown in the figure. DETAILED DESCRIPTION

[0047] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0048] The present application aims to improve the flexibility and all-weather intelligent detection capability of the roadside point for highway traffic event detection. A mobile and telescopic stand is applied to install high-definition camera devices, millimeter wave radar devices and edge processing computing devices, and a mobile perception device is formed as a whole. Based on the perception device, an automatic calibration algorithm is constructed to realize automatic calibration of the highway lane line by the device in different positions, and a traffic event detection method combining radar and video data is constructed.

[0049] In detail, the embodiments of the present application disclose an all-weather traffic event detection method based on a mobile perception device, as shown in the figure. Figure 1 The specific steps are as follows:

[0050] Step 1, based on RTK technology and video lane recognition algorithm, automatically calibrate the millimeter wave radar device and the camera device, so that the detection area of the millimeter wave radar device and the camera device is consistent;

[0051] Step 2, obtain radar data and video image data;

[0052] Step 3, based on radar data and video image data, all-weather determine highway traffic events.

[0053] In step 1, when automatically calibrating the camera device, combine local video, train images through AI data set, and identify lane lines.

[0054] Further, when automatically calibrating the millimeter wave radar device, RTK technology, i.e. carrier phase difference technology, is used for automatic calibration, specifically:

[0055] When the movable perception device is placed on the roadside for work, due to human error, the front direction of the millimeter wave radar device may be deviated, as shown by the white horizontal bar in Figure 2 , use the RTK device to obtain its own position; according to the positions of the two points, calculate the angle of the bracket horizontal bar deviating from the lane, and feed back the deviated angle to the millimeter wave radar device; the millimeter wave radar device adjusts the direction of the beam to make the beam direction parallel to the lane direction. The RTK device calibration technology can enhance the mobility of the perception device, and automatically calibrate when starting without manual calibration.

[0056] In step 2, the main purpose is to collect data, radar data includes the position information (latitude and longitude coordinates) of each vehicle, speed information (vector) and acceleration information (vector).

[0057] Further, after obtaining radar data and video image data, use video image data to calibrate radar data, as shown in Figure 3 , specifically:

[0058] After the millimeter wave radar device detects the vehicle on the road, the vehicle that realizes stable tracking will be displayed on the corresponding lane on the host computer end; however, the actual situation may cause the lane displayed on the host computer and the actual lane not to match due to the deviation of the radar angle, therefore, through the synchronous detection of the camera device and the millimeter wave radar device, it is helpful to complete the auxiliary calibration.

[0059] In step 3, the video image data and radar data are combined and processed to realize highway traffic event detection, including: parking event, reverse event, congestion event, pedestrian event, littering event, overspeed event, etc., and the identification steps are specifically:

[0060] (1) Parking (Accident) Incident

[0061] Both parking incidents in the driving lane and parking incidents in the emergency lane can be considered abnormal parking incidents. Since radar cannot detect stationary vehicles, the detection of abnormal parking incidents mainly relies on visual information. The algorithm's input is the abnormal vehicle's position information and the acceleration information detected by radar; the output is whether the incident involved illegal parking or parking in the emergency lane.

[0062] As shown in Figures 4(a) and 4(b), the specific steps are as follows:

[0063] ① Visual information is processed through a target detection network to detect potential abnormal vehicle targets;

[0064] ②Use millimeter-wave radar equipment to detect the acceleration information of the same vehicle;

[0065] ③ Abnormal vehicles with accelerations exceeding 15 m / s² detected by millimeter-wave radar equipment. 2 If it is true, it indicates that a traffic accident has occurred; otherwise, it is an illegal parking incident or a parking incident in the emergency lane.

[0066] ④ Abnormal vehicles with acceleration detected by millimeter-wave radar equipment below 15 m / s² 2 The video determines whether the abnormal vehicle is located within the emergency lane area; if it is within the emergency lane area, it is considered an emergency lane parking incident; if it is not within the emergency lane area, it is considered illegal parking.

[0067] (2) Reverse movement event

[0068] Wrong-way driving event detection primarily relies on speed information, judging abnormal events based on the direction of the vehicle's real-time speed. While inferring vehicle direction from surveillance video is complex, radar can easily detect the speed and direction of each vehicle. Therefore, for simplicity and efficiency in wrong-way driving event detection, real-time speed information detected by radar is used.

[0069] like Figure 5 As shown, the specific steps are as follows:

[0070] ① Receive vehicle speed information detected by millimeter-wave radar equipment;

[0071] ② Determine whether the direction in which the millimeter-wave radar equipment detects vehicle speed information is consistent with the direction of travel;

[0072] ③ If the vehicle speed information is consistent with the driving direction within 3 seconds, then it is considered normal driving;

[0073] ④ If the vehicle speed information is opposite to the direction of travel within 3 seconds, it is considered a reverse driving event.

[0074] (3) Congestion events

[0075] Traffic congestion events are closely related to average speed, so the average speed information of radar is relied on for judgment. If the radar detects that there are more than 5 vehicles and the current average speed is less than 20 km / h, it is considered that there is a traffic congestion event.

[0076] As shown in Figure 6 , the specific steps are as follows:

[0077] ① Receive the vehicle speed information detected by the millimeter wave radar device;

[0078] ② Determine whether there are 5 or more vehicles with speed values less than 20 km / h;

[0079] ③ If there are no 5 or more vehicles with speed values less than 20 km / h, it is normal;

[0080] ④ If it is determined that there are 5 or more vehicles with speed values less than 20 km / h, a congestion event occurs.

[0081] (4) Pedestrian event

[0082] Since radar information can only identify vehicle size and cannot identify pedestrians, pedestrian intrusion events are mainly detected by video information. While training the target detection network, add the pedestrian data set and pedestrian category, and then perform pedestrian detection on the video screen.

[0083] As shown in Figure 7 , the specific steps are as follows:

[0084] ① Process visual information through the target detection network to find potential pedestrian targets;

[0085] ② If no pedestrian target appears within the detection range, it is normal;

[0086] ③ If a pedestrian target appears within the detection range, it is a pedestrian event.

[0087] (5) Spilled object event

[0088] Since radar information can only identify vehicle size and cannot identify spilled objects, spilled object events are mainly detected by video information. While training the target detection network, add the spilled object data set and pedestrian category, and then perform spilled object detection on the video screen.

[0089] As shown in Figure 8 , the specific steps are as follows:

[0090] ① Process visual information through the target detection network to find potential spilled object targets;

[0091] ② If no spilled object target appears within the detection range, it is normal;

[0092] ③If a target of the scattering object appears in the detection range, it is a scattering object event.

[0093] (6) Overspeed event

[0094] The overspeed event detection mainly relies on speed information, and the overspeed abnormal event is judged according to the comparison between the real-time speed of the vehicle and the speed limit value.

[0095] As shown in Figure 9 , the specific steps are as follows:

[0096] ① Receive the vehicle speed information detected by the millimeter wave radar device;

[0097] ② Determine whether the vehicle speed exceeds the speed limit requirement value 20km / h of the road section;

[0098] ③ If the vehicle speed does not exceed the speed limit requirement value of the current road section, it is normal;

[0099] ④ If the vehicle speed exceeds the speed limit requirement value of the road section, it is an overspeed event.

[0100] Corresponding to the method, the embodiment of the application also provides a traffic event all-weather detection system based on a movable sensing device, which is used for specific implementation of the method, and the traffic event all-weather detection system based on the movable sensing device provided by the embodiment of the application can be applied in a computer terminal or various mobile devices, and specifically includes:

[0101] A calibration module is configured to calibrate the millimeter wave radar device and the camera device automatically by using RTK technology and a video lane recognition algorithm, so that the detection regions of the millimeter wave radar device and the camera device are consistent.

[0102] A data acquisition module is configured to acquire radar data and video image data.

[0103] An event recognition module is configured to determine the highway traffic event according to the radar data and the video image data all-weather.

[0104] In specific implementation, the movable sensing device is placed on the roadside, the device vertical rod is raised to a height of 4 meters, the high-definition camera device and the millimeter wave radar device installed on the top of the movable vertical rod are directed to the road region to be detected. The movable sensing device will automatically calibrate the lane, the AI recognition of the lane line on the road surface is performed through the video data, and the specific range of the detection in the video image is determined; the radar mainly acquires the position and angle information of itself through the horizontal rod (RTK positioning device) installed on the top of the vertical rod, and when the vehicle passes through the detection region, the lane line coordinates can be roughly calibrated according to the trajectory information of the vehicle, the lane line position is corrected in combination with the lane line coordinate position given by the video recognition, and then the automatic calibration of the lane line is realized.

[0105] When the device automatic calibration is completed, it can be ensured that the detection area of the camera and the radar is the common area, and the video image data and the radar data (vehicle position, speed, acceleration) are combined to determine whether a traffic event occurs in the road detection range. When the traffic event information is generated, it can be stored in the local edge computing device, and can also be uploaded through the wireless network. Since the application adopts radar and camera for joint detection, it has the working characteristic of all-weather detection.

[0106] The various embodiments in the specification are described in a progressive manner, and each embodiment focuses on the difference from other embodiments. The same or similar parts between the various embodiments can be referred to each other. For the system disclosed in the embodiments, since it corresponds to the method disclosed in the embodiments, the description is relatively simple, and the relevant part can be referred to the method part.

[0107] The above description of the disclosed embodiments enables a person skilled in the art to implement or use the application. Various modifications to the embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the application. Therefore, the application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for all-weather detection of traffic incidents based on mobile sensing devices, characterized in that, The method comprises the following steps: Automatic calibration of the millimeter wave radar device and the camera device is performed based on RTK technology and a video lane recognition algorithm, so that the detection ranges of the millimeter wave radar device and the camera device are consistent. Radar data and video image data are acquired. Highway traffic events are determined based on the radar data and the video image data. When the movable perception device is placed on the roadside for work, the position of the device is acquired by using an RTK device; based on the positions of two points, the angle of deviation of the horizontal pole of the support from the lane is calculated, and the angle of deviation is fed back to the millimeter wave radar device; the millimeter wave radar device adjusts the direction of the transmission beam, so that the direction of the beam is parallel to the direction of the lane. After the radar data and the video image data are acquired, the video image data is used to calibrate the radar data, and specifically: After the millimeter wave radar device detects a vehicle on the road, the vehicle is displayed on the corresponding lane on the host computer; through synchronous detection of the camera device and the millimeter wave radar device, auxiliary calibration is completed. When the camera device is calibrated automatically, the lane lines are recognized by training images through AI data sets in combination with local videos. 2.The traffic incident all-weather detection method based on mobile sensing device according to claim 1, wherein, The radar data include position information, speed information and acceleration information of each vehicle. 3.The traffic incident detection method based on mobile sensing device according to claim 1, wherein, The highway traffic event is a parking event, and the identification method is specifically as follows: 4.The traffic incident detection method based on mobile sensing device according to claim 1, wherein, Potential abnormal vehicle targets are found by processing visual information through a target detection network; The acceleration of the abnormal vehicle target is detected by the millimeter wave radar device; If the acceleration of the abnormal vehicle target exceeds an acceleration threshold, a traffic accident event exists, otherwise, it is a parking violation event or an emergency lane parking event; If the acceleration of the abnormal vehicle target is lower than the acceleration threshold, whether the position of the abnormal vehicle target is in the emergency lane area is determined through the video image data; If yes, it is an emergency lane parking event, otherwise, it is a parking violation event. The highway traffic event is a reverse driving event, a congestion event or a speeding event, and the identification method is specifically as follows: 5.The all-weather traffic incident detection method based on mobile sensing device according to claim 1, wherein, Vehicle speed information is collected by using the millimeter wave radar device; It is determined whether the direction of the vehicle speed information detected by the millimeter wave radar device is consistent with the driving direction; if the vehicle speed information is consistent with the driving direction within a preset time, it is normal driving, otherwise, it is a reverse driving event; It is determined whether there are more than a preset number of vehicle speed information lower than a speed threshold; if yes, it is a congestion event, otherwise, it is normal; It is determined whether the vehicle speed information exceeds the speed limit value of the road section; if yes, it is a speeding event, otherwise, it is normal. The highway traffic event is a pedestrian event or a littering event, and the identification method is specifically as follows: 6.The all-weather traffic incident detection method based on mobile sensing device according to claim 1, wherein, Potential pedestrian targets and littering targets are found by processing visual information through a target detection network; If no pedestrian target appears in the detection range, it is normal, otherwise, it is a pedestrian event; If no littering target appears in the detection range, it is normal, otherwise, it is a littering event. The method comprises the following steps:

7. A mobile-sensing-device-based all-weather traffic incident detection system implementing the method of any one of claims 1-6, characterized in that, The calibration module is configured to perform automatic calibration of the millimeter wave radar device and the camera device based on RTK technology and a video lane recognition algorithm, so that the detection ranges of the millimeter wave radar device and the camera device are consistent. ​ A data acquisition module is configured to acquire radar data and video image data. An event identification module is configured to determine highway traffic events based on the radar data and the video image data.

Images