Traffic incident all-weather detection method and system based on movable sensing equipment

Abstract

The invention discloses a traffic incident all-weather detection method and system based on a movable sensing device, and relates to the technical field of road traffic detection, and the method comprises the following steps: carrying out the automatic calibration of a millimeter wave radar device and a camera device based on an RTK technology and a video lane recognition algorithm, the detection areas of the millimeter wave radar equipment and the camera equipment are consistent; acquiring radar data and video image data; based on radar data and video image data, the highway traffic incident is judged in an all-weather manner. According to the invention, the device can automatically calibrate the road lane line at different positions, and the flexibility and all-weather intelligent detection capability of the roadside point location for road traffic incident detection are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of highway traffic detection, and more particularly to an all-weather detection method and system for traffic events based on mobile sensing equipment. Background Art

[0002] Traffic incidents are the key monitoring targets of highway operation and management. At present, the intelligent detection method of highway traffic incidents generally involves building fixed poles on the highway and installing surveillance cameras with intelligent analysis capabilities on the poles. The cameras are used to identify highway traffic incidents, including parking incidents, pedestrian incidents, congestion incidents, wrong-way incidents, and spilled objects incidents.

[0003] Existing traffic incident detection methods that simply rely on video detection are easily affected by insufficient lighting conditions such as night, rainy days, snowy days, and foggy days. Combined with the all-weather detection characteristics of radar equipment, all-weather detection of highway traffic incidents can be achieved through the cooperation of radar equipment and camera equipment. However, whether it is a camera or video intelligent detection, when the device position is fixed, the specific lane position needs to be calibrated. When facing a type of mobile sensing equipment such as temporary monitoring detection, repeated manual calibration will increase the workload.

[0004] Therefore, how to improve the flexibility of roadside points for highway traffic incident detection and the all-weather intelligent detection capability is an issue that technical personnel in this field urgently need to solve. Summary of the invention

[0005] In view of this, the present invention provides a method and system for all-weather detection of traffic incidents based on mobile sensing equipment, which overcomes the above-mentioned defects.

[0006] In order to achieve the above object, the present invention provides the following technical solutions:

[0007] A method for detecting traffic incidents around the clock based on a mobile sensing device comprises the following steps:

[0008] Based on RTK technology and video lane recognition algorithm, the millimeter wave radar equipment and camera equipment are automatically calibrated to make the detection areas of the millimeter wave radar equipment and camera equipment consistent;

[0009] Acquire radar data and video image data;

[0010] Based on radar data and video image data, highway traffic events can be judged around the clock.

[0011] Optionally, when automatically calibrating the camera device, combine it with the local video and train the image through the AI ​​dataset to identify the lane lines.

[0012] Optionally, when automatically calibrating the millimeter wave radar device, the carrier phase difference technology is used for automatic calibration, specifically:

[0013] When the mobile sensing device is placed on the roadside to work, it uses the RTK device to obtain its own position; based on the positions of the two points, it calculates the angle of deviation of the bracket crossbar compared to the lane, and feeds the deviation angle back to the millimeter-wave radar device; the millimeter-wave radar device adjusts the direction of the transmitted beam so that the beam direction is parallel to the lane direction.

[0014] Optionally, the radar data includes position information, speed information and acceleration information of each vehicle.

[0015] Optionally, after acquiring the radar data and the video image data, the radar data is calibrated using the video image data, specifically:

[0016] After the millimeter-wave radar device detects a vehicle on the road, it displays it on the corresponding lane on the host computer; auxiliary calibration is completed through synchronous detection by the camera device and the millimeter-wave radar device.

[0017] Optionally, the highway traffic event is a parking event, and the identification method is specifically as follows:

[0018] Process visual information through the target detection network to detect potential abnormal vehicle targets;

[0019] Detect the acceleration of abnormal vehicle targets through millimeter wave radar equipment;

[0020] If the acceleration of the abnormal vehicle target exceeds the acceleration threshold, there is a traffic accident event, otherwise it is an illegal parking event or an emergency lane parking event;

[0021] If the acceleration of the abnormal vehicle target is lower than the acceleration threshold, the video image data is used to determine whether the position of the abnormal vehicle target is within the emergency lane area; if so, it is an emergency lane parking event, otherwise it is an illegal parking event.

[0022] Optionally, the highway traffic event is a wrong-way event, a congestion event, or a speeding event, and the identification method is specifically as follows:

[0023] Use millimeter wave radar equipment to collect vehicle speed information;

[0024] Determine whether the direction in which the vehicle speed information is detected by the millimeter-wave radar device is consistent with the driving direction. If the vehicle speed information is consistent with the driving direction within a preset time, it is normal driving, otherwise it is a reverse driving event;

[0025] Determine whether there are more than a preset number of vehicles whose speed information is lower than the speed threshold, if so, it is a congestion event, otherwise it is normal;

[0026] Determine whether the vehicle speed information exceeds the speed limit of the current road section. If so, it is a speeding event, otherwise it is normal.

[0027] Optionally, the highway traffic event is a pedestrian event or a spilled object event, and the identification method is as follows:

[0028] The target detection network processes visual information to detect potential pedestrians and scattered objects.

[0029] If no pedestrian target appears within the detection range, it is normal, otherwise it is a pedestrian event;

[0030] If no spilled object appears within the detection range, it is normal; otherwise, it is a spilled object event.

[0031] A traffic incident all-weather detection system based on mobile sensing equipment, comprising:

[0032] The calibration module is used to automatically calibrate the millimeter-wave radar device and the camera device through RTK technology and video lane recognition algorithm to make the detection areas of the millimeter-wave radar device and the camera device consistent;

[0033] Data acquisition module, used to obtain radar data and video image data;

[0034] The event recognition module is used to determine highway traffic events around the clock based on radar data and video image data.

[0035] It can be seen from the above technical solutions that, compared with the prior art, the present invention discloses a method and system for all-weather detection of traffic incidents based on mobile sensing devices, which can realize automatic calibration of mobile sensing devices, automatically calibrate lane lines within the detection range after the position is moved, and then automatically lock the detection range of traffic incidents, eliminating the huge workload of manual calibration; it can realize the effective combination of radar data and video image data, and thus realize all-weather detection of highway traffic incidents. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0037] Figure 1 A schematic diagram of the all-weather detection method for traffic incidents based on a mobile sensing device provided by the present invention;

[0038] Figure 2A schematic diagram of automatic calibration of a millimeter wave radar device provided by the present invention;

[0039] Figure 3 A logical architecture diagram of the automatic calibration of the millimeter wave radar device and the camera device provided by the present invention;

[0040] FIG4( a) is a flow chart of determining a parking event provided by the present invention;

[0041] FIG4( b) is a flow chart for determining illegal parking events / emergency lane parking events provided by the present invention;

[0042] Figure 5 A flow chart for determining a retrograde event provided by the present invention;

[0043] Figure 6 A flow chart for determining a congestion event provided by the present invention;

[0044] Figure 7 A flowchart of pedestrian event determination provided by the present invention;

[0045] Figure 8 A flow chart for determining a spilled object event provided by the present invention;

[0046] Fig. 9 This is a flow chart for determining an overspeed event provided by the present invention. DETAILED DESCRIPTION

[0047] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0048] The present invention aims to improve the flexibility of roadside points for highway traffic incident detection and the all-weather intelligent detection capability. By using movable and retractable poles, high-definition camera equipment, millimeter-wave radar equipment and edge processing computing equipment are installed to form a mobile sensing device. Based on this sensing device, an automatic device calibration algorithm is constructed to achieve automatic calibration of highway lane lines at different locations, and a traffic incident detection method combining radar and video data is constructed.

[0049] In detail, the embodiment of the present invention discloses a method for detecting traffic incidents around the clock based on a mobile sensing device, such as Figure 1 As shown, the specific steps are:

[0050] Step 1: Based on RTK technology and video lane recognition algorithm, the millimeter wave radar device and the camera device are automatically calibrated to make the detection areas of the millimeter wave radar device and the camera device consistent;

[0051] Step 2: Acquire radar data and video image data;

[0052] Step 3: Based on radar data and video image data, determine highway traffic events around the clock.

[0053] In step 1, the camera device is automatically calibrated, combined with local video, and the image is trained through the AI ​​dataset to identify lane lines.

[0054] Furthermore, when the millimeter wave radar equipment is automatically calibrated, RTK technology, that is, carrier phase difference technology, is used for automatic calibration, specifically:

[0055] When the mobile sensing device is placed on the roadside for operation, the front orientation of the millimeter wave radar device may deviate due to human error, such as Figure 2 As shown by the white bar in the figure, the RTK device is used to obtain its own position; based on the positions of the two points, the angle of deviation of the bracket bar compared to the lane is calculated, and the deviation angle is fed back to the millimeter wave radar device; the millimeter wave radar device adjusts the direction of the transmitted beam to make the beam direction parallel to the lane direction. The use of RTK equipment calibration technology can enhance the mobility of the perception device, and the calibration is automatically realized when it is turned on, without manual calibration.

[0056] In step 2, data is mainly collected, and the radar data includes the position information (latitude and longitude coordinates), speed information (vector) and acceleration information (vector) of each vehicle.

[0057] Furthermore, after acquiring the radar data and the video image data, the radar data is calibrated using the video image data, such as Figure 3 As shown, specifically:

[0058] After the millimeter-wave radar device detects a vehicle on the road, the vehicle that has achieved stable tracking will be displayed on the corresponding lane on the host computer side; however, due to the deviation of the radar angle, the actual situation may cause the lane displayed by the host computer to be inconsistent with the actual lane. Therefore, synchronous detection through camera equipment and millimeter-wave radar equipment can help complete auxiliary calibration.

[0059] In step 3, the video image data and radar data are combined and processed to realize highway traffic event detection, including: parking events, wrong-way events, congestion events, pedestrian events, spilled objects events, speeding events, etc. The specific identification steps are as follows:

[0060] (1) Parking (accident) incident

[0061] Both the driving lane parking incident and the emergency lane parking incident can be regarded as abnormal parking incidents. Since radar cannot detect stationary vehicles, the detection of abnormal parking incidents mainly relies on visual information. The input of the algorithm is the position information of the abnormal vehicle and the acceleration information detected by the radar, and the output is that the accident vehicle belongs to an illegal parking incident or an emergency lane parking incident.

[0062] As shown in Figure 4(a) and Figure 4(b), the specific steps are as follows:

[0063] ① Process visual information through the target detection network to detect potential abnormal vehicle targets;

[0064] ② Use millimeter wave radar equipment to detect the acceleration information of the same vehicle;

[0065] ③ Abnormal vehicle and the acceleration detected by the millimeter wave radar equipment exceeds 15m / s 2 , it indicates that there is a traffic accident, otherwise it is an illegal parking incident or an emergency lane parking incident;

[0066] ④ Abnormal vehicle and the acceleration detected by the millimeter wave radar equipment is less than 15m / s 2 The video determines whether the abnormal vehicle position is in the emergency lane area; if it is in the emergency lane area, it is an emergency lane parking incident; if it is not in the emergency lane area, it is an illegal parking.

[0067] (2) Retrograde events

[0068] The detection of reverse traffic events mainly relies on speed information, and abnormal events are judged based on the direction of the vehicle's real-time speed. It is complicated to infer the direction of the vehicle through monitoring video, but radar can easily detect the speed direction of each vehicle. Therefore, in order to make reverse traffic event detection simple and efficient, the real-time speed information detected by radar is used for detection.

[0069] like Figure 5 As shown, the specific steps are as follows:

[0070] ① Receive vehicle speed information detected by millimeter wave radar equipment;

[0071] ② Determine whether the direction in which the millimeter wave radar device detects vehicle speed information is consistent with the driving direction;

[0072] ③ Within 3 seconds, if the vehicle speed information is consistent with the driving direction, it is normal driving;

[0073] ④ If the vehicle speed information is opposite to the driving direction within 3 seconds, it is a wrong-way event.

[0074] (3) Congestion incidents

[0075] Traffic congestion events are closely related to average speed, so the average speed information of the radar is used for judgment. If the radar detects that there are more than 5 vehicles and the current average speed is less than 20km / h, it is considered that there is a traffic congestion event.

[0076] like Figure 6 As shown, the specific steps are as follows:

[0077] ① Receive vehicle speed information detected by millimeter wave radar equipment;

[0078] ② Determine whether there are 5 or more vehicles with speed values ​​below 20km / h;

[0079] ③ If there are no 5 or more vehicles with speed values ​​below 20km / h, it is normal;

[0080] ④ If it is determined that there are 5 or more vehicles with speed values ​​lower than 20km / h, a congestion event occurs.

[0081] (4) Pedestrian incidents

[0082] Since radar information can only identify the size of vehicles but not pedestrians, pedestrian intrusion events are mainly detected by video information. While training the target detection network, add pedestrian datasets and pedestrian categories, and then perform pedestrian detection on the video screen.

[0083] like Figure 7 As shown, the specific steps are as follows:

[0084] ① Process visual information through the target detection network to detect potential pedestrian targets;

[0085] ② If there is no pedestrian target within the detection range, it is normal;

[0086] ③If a pedestrian target appears within the detection range, it is a pedestrian event.

[0087] (5) Spillage incidents

[0088] Since radar information can only identify the size of vehicles but not the scattered objects, the scattered object events are mainly detected by video information. While training the target detection network, add the scattered object dataset and pedestrian category, and then perform scattered object detection on the video screen.

[0089] like Figure 8 As shown, the specific steps are as follows:

[0090] ① Processing visual information through the target detection network to detect potential targets of scattered objects;

[0091] ② If there is no spilled object within the detection range, it is normal;

[0092] ③If a spilled object appears within the detection range, it is a spilled object event.

[0093] (6) Speeding incidents

[0094] Speeding event detection mainly relies on speed information, and speeding abnormal events are judged based on the comparison between the real-time speed of the vehicle and the speed limit.

[0095] like Fig. 9 As shown, the specific steps are as follows:

[0096] ① Receive vehicle speed information detected by millimeter wave radar equipment;

[0097] ② Determine whether the vehicle speed exceeds the speed limit requirement of 20km / h on the road section;

[0098] ③ If the vehicle speed does not exceed the speed limit required by the current road section, it is normal;

[0099] ④If the vehicle speed exceeds the speed limit required for the road section, it is a speeding incident.

[0100] Corresponding to the method, an embodiment of the present invention further provides a traffic incident all-weather detection system based on a mobile sensing device, which is used to implement the method. The traffic incident all-weather detection system based on a mobile sensing device provided by the embodiment of the present invention can be applied to a computer terminal or various mobile devices, and specifically includes:

[0101] The calibration module is used to automatically calibrate the millimeter-wave radar device and the camera device through RTK technology and video lane recognition algorithm to make the detection areas of the millimeter-wave radar device and the camera device consistent;

[0102] Data acquisition module, used to obtain radar data and video image data;

[0103] The event recognition module is used to determine highway traffic events around the clock based on radar data and video image data.

[0104] During the specific implementation, the mobile sensing device is placed on the roadside, the device pole is raised to a height of 4 meters, and the high-definition camera device and millimeter-wave radar device installed on the top of the mobile pole are directed toward the road area that needs to be detected. The mobile sensing device will automatically calibrate the lane, perform AI recognition of the road lane line through video data, and clarify the specific detection range in the video image; the radar mainly obtains its own position and angle information through the crossbar (RTK positioning device) installed on the top of the pole. When the vehicle passes through the detection area, the lane line coordinates can be roughly calibrated according to the vehicle's trajectory information, and the lane line position can be corrected in combination with the given lane line coordinate position identified by the video, thereby realizing automatic calibration of the lane line.

[0105] After the automatic calibration of the equipment is completed, it can ensure that the detection areas of the camera and radar are the same area. The video image data and radar data (vehicle position, speed, acceleration) are combined to jointly determine whether a traffic incident occurs within the road detection range. When the traffic incident information is generated, it can be stored in the local edge computing device or uploaded via the wireless network. Since the present invention uses radar and camera for joint detection, it has the working characteristics of all-weather detection.

[0106] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the system disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0107] The above description of the disclosed embodiments enables one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for detecting traffic incidents around the clock based on mobile sensing equipment, characterized in that: The following steps are involved: Based on RTK technology and video lane recognition algorithm, the millimeter wave radar equipment and camera equipment are automatically calibrated to make the detection areas of the millimeter wave radar equipment and camera equipment consistent; Acquire radar data and video image data; Based on radar data and video image data, highway traffic events can be judged around the clock.

2. The all-weather detection method for traffic incidents based on mobile sensing equipment according to claim 1 is characterized in that: When automatically calibrating the camera equipment, it combines local video with the AI ​​dataset to train images and identify lane lines.

3. The all-weather detection method for traffic incidents based on mobile sensing equipment according to claim 1 is characterized in that: When automatically calibrating millimeter wave radar equipment, carrier phase difference technology is used for automatic calibration, specifically: When the mobile sensing device is placed on the roadside to work, it uses the RTK device to obtain its own position; based on the positions of the two points, it calculates the angle of deviation of the bracket crossbar compared to the lane, and feeds the deviation angle back to the millimeter-wave radar device; the millimeter-wave radar device adjusts the direction of the transmitted beam so that the beam direction is parallel to the lane direction.

4. The all-weather detection method for traffic incidents based on mobile sensing equipment according to claim 1 is characterized in that: Radar data includes the position, speed and acceleration information of each vehicle.

5. The all-weather detection method for traffic incidents based on mobile sensing equipment according to claim 1 is characterized in that: After acquiring radar data and video image data, the radar data is calibrated using the video image data, specifically: After the millimeter-wave radar device detects a vehicle on the road, it displays it on the corresponding lane on the host computer; auxiliary calibration is completed through synchronous detection by the camera device and the millimeter-wave radar device.

6. The all-weather detection method for traffic incidents based on mobile sensing equipment according to claim 1 is characterized in that: Highway traffic events are parking events, and the specific identification methods are as follows: Process visual information through the target detection network to detect potential abnormal vehicle targets; Detect the acceleration of abnormal vehicle targets through millimeter wave radar equipment; If the acceleration of the abnormal vehicle target exceeds the acceleration threshold, there is a traffic accident event, otherwise it is an illegal parking event or an emergency lane parking event; If the acceleration of the abnormal vehicle target is lower than the acceleration threshold, it is determined whether the position of the abnormal vehicle target is within the emergency lane area through the video image data; If yes, it is an emergency lane parking event, otherwise it is an illegal parking event.

7. The all-weather detection method for traffic incidents based on mobile sensing equipment according to claim 1 is characterized in that: Highway traffic incidents include wrong-way incidents, congestion incidents, and speeding incidents. The specific identification methods are as follows: Use millimeter wave radar equipment to collect vehicle speed information; Determine whether the direction in which the vehicle speed information is detected by the millimeter-wave radar device is consistent with the driving direction. If the vehicle speed information is consistent with the driving direction within a preset time, it is normal driving, otherwise it is a reverse driving event; Determine whether there are more than a preset number of vehicles whose speed information is lower than the speed threshold, if so, it is a congestion event, otherwise it is normal; Determine whether the vehicle speed information exceeds the speed limit of the current road section. If so, it is a speeding event, otherwise it is normal.

8. The all-weather detection method for traffic incidents based on mobile sensing equipment according to claim 1 is characterized in that: Highway traffic incidents include pedestrian incidents and spilled objects incidents. The specific identification methods are as follows: The target detection network processes visual information to detect potential pedestrians and scattered objects. If no pedestrian target appears within the detection range, it is normal, otherwise it is a pedestrian event; If no spilled object appears within the detection range, it is normal; otherwise, it is a spilled object event.

9. A traffic incident all-weather detection system based on a mobile sensing device for implementing the method described in any one of claims 1 to 8, characterized in that: include: The calibration module is used to automatically calibrate the millimeter-wave radar device and the camera device through RTK technology and video lane recognition algorithm to make the detection areas of the millimeter-wave radar device and the camera device consistent; Data acquisition module, used to obtain radar data and video image data; The event recognition module is used to determine highway traffic events around the clock based on radar data and video image data.

Images