Air release time calculation method and system based on radar and vision fusion

Abstract

The application discloses a kind of based on the space-time calculation method and system of fusion of radar vision.The video image data is fused with radar detection data, obtains the relative space-time information of each vehicle relative to intersection, obtains the absolute space-time information of all vehicles of intersection according to relative space-time information;According to the absolute space-time information of each vehicle, the time mark is carried out to each vehicle passing through stop line in stage time, obtains the marking time of each vehicle;According to the absolute space-time information of each vehicle, the saturation head time distance of queuing vehicle team is obtained;According to saturation head time distance, the time extension is carried out to marking time, obtains the extension time period of each vehicle;According to the absolute space-time information of each vehicle, the start-up sub-loss time period and the clearing sub-loss time period in stage time are obtained;The extension time period of each vehicle in stage time, start-up sub-loss time period and clearing sub-loss time period are removed, and the emptying time period of each lane is obtained.

Description

Technical Field

[0001] This application relates to the field of intelligent transportation technology, and in particular to a method and system for calculating idle time based on radar-visual fusion. Background Technology

[0002] Traffic congestion is a representative "urban disease" problem faced in my country's new urbanization process, causing huge losses to social and economic development. Optimizing traffic conditions at intersections is key to preventing and alleviating traffic congestion, and estimating traffic state parameters at intersections is an essential prerequisite for traffic optimization. Vehicle trajectories at intersections can comprehensively and completely characterize the traffic flow status, containing rich traffic flow information. These trajectories can reflect the changes in vehicle speed and acceleration over time and space, accurately reproducing the operational patterns of vehicles passing through intersections. This improves the accuracy of traffic state parameter estimation and prediction (such as travel speed, travel time, queue length, delay, etc.) and the efficiency of traffic signal control, possessing significant theoretical and practical value.

[0003] However, traditional methods for calculating idle time use observation to obtain idle time through sampling. This results in unstable sampling data due to environmental or human factors affecting different times and spaces, leading to large errors and low accuracy in traditional calculation methods. Summary of the Invention

[0004] The purpose of this application is to solve the technical problems of large errors and low accuracy in traditional idle discharge time calculation methods. To achieve the above objective, this application provides an idle discharge time calculation method and system based on radar-visual fusion.

[0005] This application provides a method for calculating idle discharge time based on radar-visual fusion, including:

[0006] Acquire video image data and radar detection data;

[0007] The video image data and the radar detection data are fused to obtain the relative spatiotemporal information of each vehicle relative to the intersection, and the absolute spatiotemporal information of all vehicles at the intersection is obtained based on the relative spatiotemporal information.

[0008] Obtain traffic timing information and map information of the intersection, and obtain the stage time for each lane based on the traffic timing information and map information;

[0009] Based on the absolute spatiotemporal information of each vehicle, time stamps are performed on each vehicle that passes the stop line within the phase time of each lane to obtain the stamp time of each vehicle;

[0010] Based on the traffic timing information and the map information, the saturation headway of the queuing platoon is obtained according to the absolute spatiotemporal information of each vehicle.

[0011] Based on the saturated headway, the marked time for each vehicle is extended to obtain the extended time period for each vehicle;

[0012] Based on the absolute spatiotemporal information of each vehicle, the start-up sub-loss time period and the clearing sub-loss time period within the stage time of each lane are obtained;

[0013] The idle time period for each lane is obtained by removing the extended time period, the starter loss time period, and the clearing loss time period for each vehicle within the phase time of each lane.

[0014] In one embodiment, the idle release time calculation method based on radar-visual fusion further includes:

[0015] In the time dimension, the intersection of the empty time periods of each lane in the same direction is calculated to obtain the direction-level empty time periods;

[0016] In the time dimension, the extended time period, the start-up sub-loss time period, and the clearing sub-loss time period of each vehicle in each lane in the same direction within the stage time are calculated by union to obtain the direction-level occupancy time period.

[0017] In one embodiment, the idle release time calculation method based on radar-visual fusion further includes:

[0018] In the time dimension, the intersection time periods of the directional-level empty parking periods for each direction of the intersection are calculated to obtain the intersection empty parking time periods.

[0019] In the time dimension, the intersection occupancy time period is obtained by performing a union calculation on the direction-level occupancy time period for each direction of the intersection; or

[0020] In the time dimension, the intersection of the empty time periods of each lane at the intersection is calculated to obtain the empty time periods of the intersection.

[0021] In the time dimension, the intersection occupancy time period is obtained by performing a union calculation on the extended time period, the start-up sub-loss time period, and the clearing sub-loss time period for each vehicle in each lane of the intersection within the stage time.

[0022] In one embodiment, the idle release time calculation method based on radar-visual fusion further includes:

[0023] In the time dimension, the intersection of the empty time periods of each lane in each stage within a cycle is calculated to obtain the stage empty time period.

[0024] In the time dimension, the extended time period, the start-up loss time period, and the clearing loss time period of each vehicle in each lane of each stage within a cycle are calculated by union to obtain the stage occupancy time period.

[0025] In one embodiment, the idle release time calculation method based on radar-visual fusion further includes:

[0026] In the time dimension, the idle time periods of each stage within a cycle are combined to obtain the periodic idle time period.

[0027] This application provides a system for calculating idle discharge time based on radar-visual fusion, including:

[0028] The data acquisition module is used to acquire video image data and radar detection data;

[0029] The absolute spatiotemporal information acquisition module is used to fuse the video image data with the radar detection data to obtain the relative spatiotemporal information of each vehicle relative to the intersection, and to obtain the absolute spatiotemporal information of all vehicles at the intersection based on the relative spatiotemporal information.

[0030] The stage time acquisition module is used to acquire traffic timing information and map information of the intersection, and obtain the stage time of each lane based on the traffic timing information and map information;

[0031] The marker time acquisition module is used to time-mark each vehicle that passes the stop line within the phase time of each lane based on the absolute spatiotemporal information of each vehicle, and obtain the marker time of each vehicle.

[0032] The saturated headway acquisition module is used to obtain the saturated headway of the queuing platoon based on the traffic timing information and the map information, according to the absolute spatiotemporal information of each vehicle.

[0033] The extended time period acquisition module is used to extend the marked time of each vehicle according to the saturated headway, so as to obtain the extended time period of each vehicle.

[0034] The loss time period acquisition module is used to obtain the start-up sub-loss time period and the clearing sub-loss time period within the stage time of each lane based on the absolute spatiotemporal information of each vehicle.

[0035] The idle time period acquisition module is used to remove the extended time period, the start-up loss time period, and the clearing loss time period of each vehicle within the phase time of each lane to obtain the idle time period of each lane.

[0036] In one embodiment, the idle release time calculation system based on radar-visual fusion further includes:

[0037] The directional empty time period acquisition module is used to perform intersection calculation on the empty time periods of each lane in the same direction in the time dimension to obtain the directional empty time period.

[0038] The directional occupancy time period acquisition module is used to perform a union calculation on the extended time period, the start-up sub-loss time period, and the clearing sub-loss time period of each vehicle in each lane in the same direction within the time of the stage, in the time dimension, to obtain the directional occupancy time period.

[0039] In one embodiment, the idle release time calculation system based on radar-visual fusion further includes:

[0040] The intersection vacancy period acquisition module is used to perform intersection calculation on the vacancy period of each direction of the intersection in the time dimension to obtain the intersection vacancy period.

[0041] The intersection occupancy time period acquisition module is used to perform a union calculation on the direction-level occupancy time periods for each direction of the intersection in the time dimension to obtain the intersection occupancy time period; or

[0042] The intersection vacancy period acquisition module is used to perform intersection calculation on the vacancy period of each lane of the intersection in the time dimension to obtain the intersection vacancy period.

[0043] The intersection occupancy time period acquisition module is used to perform a union calculation on the extended time period, the start-up sub-loss time period, and the clearing sub-loss time period of each vehicle in each lane of the intersection within the stage time in the time dimension to obtain the intersection occupancy time period.

[0044] In one embodiment, the idle release time calculation system based on radar-visual fusion further includes:

[0045] The phased empty time period acquisition module is used to calculate the intersection of the empty time periods of each lane in each phase within a cycle in the time dimension to obtain the phased empty time period.

[0046] The phase occupancy time period acquisition module is used to perform a union calculation on the extended time period, the start-up sub-loss time period, and the clearing sub-loss time period for each vehicle in each lane of each phase within a cycle in the time dimension, to obtain the phase occupancy time period.

[0047] In one embodiment, the idle release time calculation system based on radar-visual fusion further includes:

[0048] The periodic idle time period acquisition module is used to perform union calculation on the idle time periods of each stage within a period in the time dimension to obtain the periodic idle time period.

[0049] The aforementioned radar-visual fusion-based idle time calculation method and system acquires the absolute spatiotemporal information of each vehicle based on video image data and radar detection data. Combined with map information, it accurately reflects the vehicle-road relationship in real-world road conditions. Furthermore, it uses traffic timing information to obtain the extended time period, initiation loss time period, and clearance loss time period for each vehicle in each lane within the permitted time period. Based on these factors, the idle time period at the lane level is calculated. The radar-visual fusion-based idle time calculation method provided in this application achieves real-time monitoring and measurement, accurately and in real-time providing the idle time period for each lane. It can utilize multiple data parameters for calculation, fully considering the actual traffic conditions at intersections. Compared to traditional methods, it is more accurate and has smaller errors, which helps improve intersection utilization, reduce saturation, and alleviate traffic pressure. Attached Figure Description

[0050] Figure 1 This is a flowchart illustrating the steps of the idle release time calculation method based on radar-visual fusion provided in this application.

[0051] Figure 2 This is a schematic diagram of traffic timing information in one embodiment provided in this application.

[0052] Figure 3 This is a schematic diagram of the stage time, idle time, and lost time in one embodiment provided in this application.

[0053] Figure 4 This is a schematic diagram of the idle release time calculation system based on radar-visual fusion provided in this application. Detailed Implementation

[0054] The technical solution of this application will be further described in detail below with reference to the accompanying drawings and embodiments.

[0055] Please see Figure 1This application provides a method for calculating idle discharge time based on radar-visual fusion, including:

[0056] S10, acquire video image data and radar detection data;

[0057] S20 fuses video image data with radar detection data to obtain the relative spatiotemporal information of each vehicle relative to the intersection, and obtains the absolute spatiotemporal information of all vehicles at the intersection based on the relative spatiotemporal information.

[0058] S30: Obtain traffic timing information and map information at intersections, and obtain the stage time for each lane based on the traffic timing information and map information;

[0059] S40, based on the absolute spatiotemporal information of each vehicle, timestamps each vehicle that passes the stop line within the phase time of each lane, and obtains the marked time of each vehicle;

[0060] S50, based on traffic timing information and map information, obtains the saturation headway of the queuing platoon according to the absolute spatiotemporal information of each vehicle;

[0061] S60, based on the saturated headway, extends the marked time of each vehicle to obtain the extended time period for each vehicle;

[0062] S70 obtains the start-up sub-loss time period and the clearing sub-loss time period within the stage time of each lane based on the absolute spatiotemporal information of each vehicle.

[0063] S80 removes the extended time period, starter loss time period, and clearing loss time period for each vehicle within the phase time of each lane to obtain the idle time period for each lane.

[0064] In S10, video image data and radar detection data can be obtained through the integrated radar-visual system. In S20, video processing software extracts images frame by frame, and deep learning methods are used to detect and identify each frame, obtaining camera-based vehicle information such as vehicle trajectory, distance between vehicles, time distance between vehicles, vehicle license plate number, vehicle color, license plate color, and vehicle model. Radar detection can acquire radar-based vehicle information such as vehicle distance, vehicle orientation, vehicle speed, vehicle acceleration, and vehicle spatial position in harsh detection environments and adverse conditions.

[0065] Radar detection data can be used to obtain the relative distance of vehicles to the intersection. Video image data can be used to obtain vehicle attribute information (such as license plate number, license plate color, and vehicle color) and time information for each frame. By fusing the video image data and radar detection data, the relative spatiotemporal information of each vehicle relative to the intersection can be obtained. Relative spatiotemporal information is spatiotemporal information with the intersection as a reference. Spatiotemporal information can be understood as the spatial position of a vehicle at different times, specifically including multi-dimensional information such as time information, vehicle position information, vehicle speed information, vehicle direction information, and vehicle model information. The relative spatiotemporal information of each vehicle relative to the intersection includes the spatial position of each vehicle relative to the intersection at different times. Based on the transformation relationship between coordinate systems, the relative spatiotemporal information can be mapped to absolute spatiotemporal information. For example, in one embodiment, a high-precision map can provide the stop line outline position information, lane line outline position information, and latitude and longitude information of the intersection. Relative spatiotemporal information includes the relative spatial position of a vehicle relative to the intersection. Based on the distance of the vehicle relative to the intersection, this relative spatial position can be converted into an absolute spatial position in a geographic coordinate system, allowing us to obtain the vehicle's latitude and longitude information. Furthermore, based on this latitude and longitude information, vehicles can be marked on a map, providing relevant geographic location information. Therefore, by fusing video image data with radar detection data to obtain the relative spatiotemporal information of each vehicle, we can further obtain the relative spatiotemporal information of all vehicles passing through the intersection, and consequently, the absolute spatiotemporal information of all vehicles passing through the intersection.

[0066] Please see Figure 2 and Figure 3 In S30, traffic timing information can be understood as providing real-world twin information of signal control equipment in the physical world, including but not limited to phase, stage, cycle, scheme, and control method. It can also be understood as urban traffic signal control information at intersections, including the timing of traffic light colors such as red, green, and yellow. For example... Figure 2As shown, the phase can be east-west straight, east-west left, north-south straight, or north-south left. When the phase name is east-west straight, the corresponding green light time is 30 seconds, yellow light time is 4 seconds, red light time is 155 seconds, and all-red time is 2 seconds. The total phase time is the sum of the green light time, yellow light time, and all-red time, which is 36 seconds. The all-red time is the period when all four phases (east-west straight, east-west left, north-south straight, and north-south left) are red. When the phase name is east-west left, the corresponding green light time is 20 seconds, yellow light time is 4 seconds, and all-red time is 2 seconds. The total phase time is 26 seconds. When the phase name is north-south straight, the corresponding green light time is 80 seconds, yellow light time is 4 seconds, and all-red time is 2 seconds. The total phase time is 86 seconds. When the phase name is north-south left, the corresponding green light time is 35 seconds, yellow light time is 4 seconds, and all-red time is 2 seconds. The total phase time is 41 seconds. In this application, the phase time includes the green light time, yellow light time, and all-red time. Traffic timing information is used to divide green light time, yellow light time, and all-red time into a phase duration.

[0067] The map information is obtained from actual surveying and mapping, providing a true twin of the physical world environment, including reference lines, lane lines, center lines, curbs, corrugated guardrails, concrete guardrails, pedestrian overpasses, traffic signs, delineators, bridge piers, arrows, text, symbols, warning zones, traffic dividers, traffic lights, stop locations, pedestrian crossings, public transport stops, speed reduction zones, bicycle lanes within intersections, no-parking zones, manhole covers, parking spaces, traffic light poles, timing signs, traffic signals, traffic islands, overpasses, checkpoints, intersection center circles, tunnel walls, bus bays, central medians, intersection surfaces, tollbooth elements, etc. The map information is high-precision and includes, but is not limited to, lane areas, lane marking locations, channelization information, and ground markings.

[0068] The absolute spatiotemporal information of vehicles obtained at intersections includes both temporal and spatial information. Combined with traffic timing information and map data, the phase duration for each lane can be determined. Phase duration can be understood as the time interval from the onset of the green light in one traffic light phase to the onset of the green light in the next phase. (See reference...) Figure 2 The diagram shows the phase duration. The phase duration includes a combination of different light colors for a given phase, encompassing the green, yellow, and red times for that phase. For example, a phase duration for one phase might include 30 seconds of green light, 4 seconds of yellow light, and 2 seconds of all-red light.

[0069] A cycle can be understood as the time required for a traffic light color to change once according to a set signal phase sequence. A cycle contains n phases or n stages. A phase represents a state at a given moment, and a stage represents the duration of that state. The stages and phases within a cycle are obtained through timing. Traffic lights indicate the permitted flow direction, which is then matched with lanes to create permitted traffic lanes. For example, if the permitted phases are north-straight, north-right, south-straight, and south-right, then the right-turn lane and the straight-and-right-turn lane within the north-south straight lanes are also permitted, thus forming permitted traffic lanes. Each phase is assigned a corresponding time, forming a stage.

[0070] In S40, by combining time information and vehicle position information from absolute spatiotemporal information, vehicle trajectory tracking can be achieved, thereby determining the time when a vehicle reaches the stop line in each lane within the permitted time period. The moment when the front of the vehicle touches the stop line is recorded as the marked time.

[0071] In S50, the control times for red, yellow, and green traffic lights can be obtained based on traffic timing information. High-precision map information generated through surveying allows for the understanding of the real-world relationship between vehicles and roads, such as the specific lane a target vehicle is located in. Simultaneously, combining the absolute spatiotemporal information of each vehicle, it's possible to determine which vehicles are queuing in a particular lane. A queuing queue consists of consecutive vehicles with a stopping delay time greater than 0. The queuing queue includes vehicles that arrived at the intersection and stopped waiting during the red light and those that arrived at the intersection and stopped to proceed during the green light. Saturated headway can be understood as the headway of the nth vehicle and subsequent vehicles after a period of time following the green light, assuming uniform speed and no starting reaction or acceleration effects. Headway can be understood as the difference in time between the nth vehicle and the (n+1)th vehicle reaching the observation line (vehicle stop line). The headway, the first vehicle in the queue, and the last vehicle in the queue are obtained. Based on the first and last vehicles in the convoy, calculate the average headway of all vehicles from the nth vehicle (starting from the first vehicle) to the last vehicle in the convoy to obtain the saturation headway. Here, n is a positive integer greater than 4.

[0072] In S60, the marked time of a vehicle is a point in time. The time is extended by adding the saturated headway to the marked time to obtain the extended time period of the vehicle. The saturated headway is the time taken for a vehicle to pass continuously in a saturated state. That is, the time taken for a vehicle to pass continuously will not be less than this time. It is regarded as the time that the vehicle must use to pass, and is considered as effective utilization, which is the occupied time.

[0073] In S70, information such as the driving trajectory, headway, and headway of each vehicle can be obtained based on absolute spatiotemporal information. The initiation sub-phase represents the period from the start of vehicle startup onwards, including the time for startup and acceleration reactions. The additional headway can be understood as the extra green light time occupied by each vehicle due to the driver's reaction to the light color change, the start of the vehicle in front, and the vehicle's acceleration effect; it can be calculated from the difference between each vehicle's headway and the saturation headway. Summing the additional headway of each vehicle corresponding to the initiation sub-phase yields the extra green light time occupied by all vehicles in the initiation sub-phase due to startup reactions and acceleration effects, thus obtaining the initiation sub-phase loss time period.

[0074] The clearing sub-phase is the opposite of the initiation sub-phase. It can be understood as the period between the green light ending and the yellow and red lights, or as the time difference between the moment the front wheels of the last vehicle passing through during the yellow light cross the stop line and the moment the next phase's green light illuminates. It can also be understood as the period of lost time caused by vehicles behind the stop line during the yellow light being no longer permitted to cross. For example, in extreme cases where no vehicles pass through the entire phase, the green light period is considered a "no-vehicle" period, and the yellow and all-red light periods are considered clearing loss time.

[0075] The clearing phase includes two types of clearing losses. One scenario is that some vehicles, unable to pass through the intersection, voluntarily slow down and stop without crossing the stop line. They stop during the yellow light period before it turns red. In this case, the yellow light at the moment of stopping functions as stopping during a red light (some vehicles have already stopped and are waiting for the red light), resulting in a clearing loss. The other scenario is that when all four directions are red (which can be understood as all-red time), vehicles that have crossed the stop line but not yet passed through the intersection are cleared before the all-red time ends, eliminating the need for additional clearing time. This extra time is considered a loss, forming another type of clearing loss.

[0076] The effective green light extension time utilized by vehicles can be understood as the time during which vehicles effectively utilize the yellow light and the all-red time. For example, if a vehicle crosses the stop line at the last moment of the yellow light and passes through the intersection at the last moment of the all-red time, this period is the fully utilized time and is not lost.

[0077] The sum of the yellow light time and the all-red light time, minus the effective green light extension time, yields the clearing sub-loss time period. The start-up sub-loss time period and the clearing sub-loss time period are mostly considered lost time, which is unavoidable and can therefore be considered as being utilized by the clearing time demand, rather than being attributed to idle time.

[0078] In S80, the extended time period for each vehicle, the start-up loss time period, and the clearing loss time period are subtracted from the stage time to obtain the idle time period for each lane.

[0079] The radar-visual fusion-based idle time calculation method provided in this application obtains the absolute spatiotemporal information of each vehicle based on video image data and radar detection data. Combined with map information, it accurately reflects the vehicle-road relationship in real-world road conditions. Furthermore, it incorporates traffic timing information to obtain the extended time period, initiation loss time period, and clearance loss time period for each vehicle in each lane within the permitted time period. Based on these factors, the idle time period at the lane level is calculated. This radar-visual fusion-based idle time calculation method achieves real-time monitoring and measurement, accurately and in real-time providing the idle time period for each lane. It can utilize multiple data parameters for calculation, fully considering the actual traffic conditions at intersections. Compared to traditional methods, it is more accurate and has smaller errors, which helps improve intersection utilization, reduce saturation, and alleviate traffic pressure.

[0080] In one embodiment, S20, video image data and radar detection data are fused to obtain the relative spatiotemporal information of each vehicle relative to the intersection, including:

[0081] S210 obtains camera-based vehicle information based on video image data;

[0082] S220 obtains radar-based vehicle information based on radar detection data;

[0083] S230 uses a radar video fusion algorithm to perform spatiotemporal fusion of camera-based vehicle information and radar-based vehicle information to obtain the relative spatiotemporal information of each vehicle relative to the intersection.

[0084] In this embodiment, the radar video fusion algorithm includes temporal fusion and spatial fusion. During spatial fusion, a transformation between the radar coordinate system and the camera coordinate system is required. Since the radar and camera are relatively fixed, the transformation relationship between the radar and camera coordinate systems can be obtained. Based on the camera coordinate system and the image coordinate system, distortion correction can be performed to obtain the distorted image coordinates. Calibration is performed using camera calibration methods based on the distance and angle of the vehicle detected by the radar. This yields the camera's internal parameters such as focal length, principal point coordinates, and distortion parameters, allowing for the acquisition of the vehicle's projected coordinates in the image. This achieves the transformation from the radar coordinate system to the pixel coordinate system, completing the spatial fusion of video image data and radar detection data.

[0085] During temporal fusion, video image data and radar detection data are acquired synchronously in time to achieve temporal fusion. When the sampling rate of the camera and the sampling rate of the radar are different, the same sampling rate can be used as the basis for sampling. The camera acquires one frame of image data at a time, and the radar selects one frame of buffered data to achieve joint sampling of one frame of radar and camera fusion data, ensuring temporal synchronization between video image data and radar detection data.

[0086] By fusing camera-based and radar-based vehicle information in both time and space, the relative spatiotemporal information of each vehicle relative to the intersection can be obtained. Relative spatiotemporal information can be understood as the relative spatiotemporal information of a vehicle relative to the intersection or the radar-visual integrated device. Spatiotemporal information can be understood as time information and spatial information. It includes multi-dimensional information such as the time, position, speed, direction of travel, model, license plate, and color of a vehicle at a given moment. In this embodiment, real-time observation and measurement using radar-visual fusion technology overcomes the influence of noise and latency, achieving full-time, high-accuracy, and low-latency results.

[0087] In one embodiment, S50, based on traffic timing information and map information, and according to the absolute spatiotemporal information of each vehicle, the saturation headway of the queuing vehicle fleet is obtained. The timing of the red and green traffic lights can be determined from the traffic timing information. Vehicles arriving at the intersection and stopping to wait during a red light can be understood as vehicles arriving at a red light and coming to a complete stop to queue. Vehicles arriving at the intersection and stopping to queue during a green light can be understood as vehicles arriving at a green light but stopping to queue because the queue has not completely dispersed. The real-time data from the radar-visual fusion fully reflects the vehicle movement situation at each stage.

[0088] The queue includes two scenarios: vehicles that arrive at the intersection and stop to wait when the traffic light is red, and vehicles that arrive at the intersection and stop to queue to pass when the traffic light is green. Headway h n =t n+1 -t n Saturation headway Saturation headway h sThe formula represents the average headway of vehicles m+1-n, starting from the nth vehicle. A value greater than 4 for n avoids inaccuracies caused by the significant impact of starting response and acceleration effects on the first four vehicles in the queue. Under normal traffic conditions, the front and rear of the queue are identified and confirmed. The headway of vehicles from the nth vehicle onwards is calculated, and the average of these headway values ​​is used to calculate the saturated headway. The saturated headway calculation process involves multiple samplings and averaging, making it applicable to morning rush hours on rainy weekdays or off-peak hours on sunny non-working days.

[0089] In one embodiment, S70, based on the absolute spatiotemporal information of each vehicle, the initiation sub-loss time period and the clearing sub-loss time period within the phase time of each lane are obtained, including:

[0090] S710, based on the absolute spatiotemporal information of each vehicle, obtains the additional headway of each vehicle corresponding to the start sub-stage within the stage time of each lane;

[0091] S720 superimposes the additional headway of each vehicle to obtain the starter loss time period within the stage time of each lane;

[0092] S730, based on the absolute spatiotemporal information of each vehicle, obtains the effective green light extension time used by vehicles during the clearing sub-phase of each lane's phase time.

[0093] S740 calculates the clearing time loss for each lane within the phase duration based on the effective green light extension time.

[0094] In this embodiment, the driving trajectory, headway, and headway of each vehicle can be obtained based on absolute spatiotemporal information. The start-up sub-stage represents the period from the start of vehicle startup to the subsequent period, including the time intervals for startup response and acceleration response. The additional headway can be understood as the extra green light time occupied by each vehicle due to startup response and acceleration effects, and can be calculated from the difference between the headway of each vehicle and the saturated headway. Summing the additional headway of each vehicle yields the extra green light time occupied by all vehicles in the start-up sub-stage due to startup response and acceleration effects, thus obtaining the start-up sub-stage loss time. The difference between the headway of each vehicle and the saturated headway is the additional headway of each vehicle. By calculating the difference between the headway and the saturated headway, the additional headway of each vehicle can be obtained.

[0095] The effective green light extension time utilized by vehicles can be understood as the time during which vehicles effectively utilize the yellow light and the entire red light. For example, if a vehicle crosses the stop line at the last moment of the yellow light and passes through the intersection at the last moment of the entire red light, this time is the fully utilized time and is not lost. The sum of the yellow light time and the entire red light time, minus the effective green light extension time, yields the clearing time loss.

[0096] Based on the absolute spatiotemporal information of all vehicles, the effective extended passage time for each vehicle that crossed the stop line during the yellow light period and passed through the intersection during the red light period is obtained during the clearing sub-phase. The effective extended passage time of each vehicle is added together to obtain the effective green light extension time utilized by vehicles during the clearing sub-phase.

[0097] Based on absolute spatiotemporal information, vehicle trajectory tracking can be achieved, obtaining multi-dimensional information such as time, vehicle position, speed, direction of travel, and vehicle model. Simultaneously, by combining traffic timing information and map data, it is possible to capture every vehicle that crosses the stop line during the yellow light phase and then passes through the intersection during the full red light phase. The full red light phase can be understood as the duration during which vehicles from any approach lane are not allowed to cross the stop line into the intersection, or as the time interval between the end of the yellow light phase and the start of the green light phase.

[0098] Crossing the stop line during the yellow light and then passing through the intersection during the all-red light phase can be understood as some vehicles crossing the stop line when the green light turns yellow, and then passing through the intersection during the period between the yellow and red lights before the red light turns green. During the clearing phase, vehicles effectively utilize the yellow and all-red light periods to pass through the intersection, thus creating an effective extension of travel time. Adding up the effective green light extension time for each vehicle yields the total effective extension of travel time utilized by multiple vehicles during the clearing phase, forming the effective green light extension time.

[0099] Obtain the all-red time corresponding to the red traffic light and the yellow time corresponding to the yellow traffic light. Based on the all-red time, yellow time, and effective green light extension time, calculate the clearance sub-stage clearance loss time. The clearance sub-stage clearance loss time equals the sum of the all-red time and the yellow time, minus the effective green light extension time.

[0100] In one embodiment, the idle discharge time calculation method based on radar-visual fusion further includes:

[0101] S901, in the time dimension, calculates the intersection of the empty time periods of each lane in the same direction to obtain the direction-level empty time periods;

[0102] S902, in the time dimension, performs a union calculation on the extended time period, start-up sub-loss time period and clearing sub-loss time period of each vehicle in each lane in the same direction within the stage time to obtain the direction-level occupancy time period.

[0103] In this embodiment, the time dimension can be understood as taking the time axis as a reference and calculating the intersection of the idle time periods of multiple lanes traveling in the same direction to obtain the idle time period at the direction level.

[0104] For example: In the same direction, there are two lanes, lane a and lane b. The phase time for lane a and lane b is [0, 10). The start-up loss time, the extension time for each vehicle, and the clearing loss time in lane a constitute the occupied time of lane a, which are [0, 2), [4, 6), and [9, 10), respectively. The vacant time of lane a (which can also be understood as the non-occupied time) are [2, 4) and [6, 9), respectively. The start-up loss time, the extension time for each vehicle, and the clearing loss time in lane b constitute the occupied time of lane b, which are [0, 3), [4, 5), and [8, 10), respectively. The vacant time of lane b (which can also be understood as the non-occupied time) are [3, 4) and [5, 8), respectively.

[0105] The intersection of the idle time periods of lane a and lane b in the same direction is calculated to obtain the directional idle time periods, which are [3,4) and [6,8), respectively.

[0106] The extension time period, start-up loss time period, and clearing loss time period (which can also be understood as the occupancy time period) of each vehicle in lanes a and b in the same direction are combined to obtain the direction-level occupancy time periods, which are [0,3), [4,6), and [8,10), respectively.

[0107] The calculation method in this embodiment tracks and measures the time information of the idle time period, extended time period, starter loss time period and clearing loss time period of each lane in the same direction in real time. It provides the idle status and idle time period in the same direction in real time, and can provide the basis for timing scheme from both spatial and temporal dimensions. At the same time, it fully considers the actual vehicle operation data of multiple lanes in the same direction, so the calculation accuracy is much greater than the rough definition of traditional methods.

[0108] In one embodiment, the idle discharge time calculation method based on radar-visual fusion further includes:

[0109] S903, in the time dimension, calculates the intersection of the directional-level idle time periods of each direction of the intersection to obtain the idling time period of the intersection.

[0110] S904 calculates the intersection occupancy time by performing a union of the directional occupancy time periods for each direction at the intersection in the time dimension.

[0111] In this embodiment, based on the methods S901 and S902 in the above embodiments, the directional-level idle time period and directional-level occupied time period in the same direction can be obtained, thereby obtaining the directional-level idle time period and directional-level occupied time period in each direction of the intersection. Intersection and union calculations are performed on the directional-level idle time period and directional-level occupied time period in each direction to obtain the intersection's occupied time period. The methods for intersection and union calculations of each time period are based on the same principles as the calculation methods in S901 and S902 in the above embodiments, and can be referred to the relevant descriptions in the above embodiments.

[0112] The calculation method in this embodiment tracks and measures the time information of the directional idle time period and directional occupied time period in real time at each direction of the intersection, and provides the idle status and idle time period of the intersection in real time. It can provide the basis for the timing scheme of the intersection from both spatial and temporal dimensions. At the same time, it fully considers the actual vehicle operation data in multiple different directions at the intersection, so the calculation accuracy is much greater than the rough definition of traditional methods.

[0113] In one embodiment, the idle discharge time calculation method based on radar-visual fusion further includes:

[0114] S905 calculates the intersection of the empty time periods of each lane at the intersection in the time dimension to obtain the empty time periods of the intersection.

[0115] S906, in the time dimension, performs a union calculation on the extended time period, start-up sub-loss time period and clearing sub-loss time period of each vehicle in each lane of the intersection within the stage time to obtain the intersection occupancy time period.

[0116] In this embodiment, through steps S10 to S80 described above, the idle time period for each lane, the extended time period for each vehicle within the stage time of each lane, the start-up loss time period, and the clearing loss time period can be obtained, thereby obtaining the idle time period for each different lane of the intersection, the extended time period for each vehicle, the start-up loss time period, and the clearing loss time period. The intersection calculation of the idle time periods for each lane of the intersection can be performed, referring to the principle of the intersection calculation method in the above embodiment. The union calculation of the extended time period, start-up loss time period, and clearing loss time period for each vehicle within the stage time of each lane of the intersection can be performed, referring to the principle of the union calculation method in the above embodiment.

[0117] In one embodiment, the idle discharge time calculation method based on radar-visual fusion further includes:

[0118] S907, in the time dimension, calculates the intersection of the empty time periods of each lane in each stage within a cycle to obtain the stage empty time period;

[0119] S908, in the time dimension, performs a union calculation on the extended time period, start-up loss time period, and clearing loss time period of each vehicle in each lane of each stage within a cycle to obtain the stage occupancy time period.

[0120] In this embodiment, a cycle can be understood as the time required for the traffic light colors to change once according to the set signal phase sequence. A cycle contains n stages or phases. The stages and phases included in the cycle are obtained through timing. The traffic lights indicate the permitted flow direction, and the flow direction is then matched with lanes to obtain the permitted lanes. For example, the permitted phases are north-straight, north-right, south-straight, and south-right. Right-turn lanes and straight-and-right-turn lanes in the north-south direction are all permitted, thus forming permitted lanes. Each phase is assigned a corresponding time, forming a stage. A phase represents a momentary state, and a stage represents the duration of that state. One stage corresponds to at least one lane. By intersecting the idle time periods of each lane in each stage, the idle time period corresponding to a stage can be obtained, i.e., the stage idle time period. Similarly, by combining the extended time period, start-up loss time period, and clearing loss time period of each vehicle within the stage's duration for each lane in each stage, the occupied time period corresponding to a stage can be obtained, i.e., the stage occupied time period.

[0121] Through the steps S10 to S80 above, the idle time period of each lane, the extended time period of each vehicle within the stage time of each lane, the starter loss time period, and the clearing loss time period can be obtained. Furthermore, the intersection or union of the time periods of each lane corresponding to each stage within a cycle can be calculated to obtain the stage idle time period.

[0122] In one embodiment, the idle discharge time calculation method based on radar-visual fusion further includes:

[0123] S909, in the time dimension, performs union calculation on the idle time periods of each stage within a cycle to obtain the cycle idle time period.

[0124] In this embodiment, a cycle includes multiple stages. By performing intersection calculations on the idle time periods of each lane in each stage, the stage idle time period can be obtained. Similarly, the corresponding stage occupied time period can be further obtained. Based on the steps described in S907 and S908 in the above embodiment, the stage idle time periods corresponding to each stage within a cycle can be obtained. Then, by performing union calculations on the stage idle time periods corresponding to each stage, the cycle idle time period corresponding to a cycle can be obtained.

[0125] As can be seen from the above embodiments, the idle time calculation method based on radar-visual fusion provided in this application can achieve real-time and accurate calculation, accurately and in real-time provide the idle time period for each lane, intersection, direction, stage and cycle, and provide the idle time period in multiple dimensions, thus solving the problem of low accuracy of traditional methods.

[0126] Please see Figure 3 This application provides a radar-visual fusion-based idle time calculation system 100. The radar-visual fusion-based idle time calculation system 100 includes a data acquisition module 10, an absolute spatiotemporal information acquisition module 20, a stage time acquisition module 30, a marker time acquisition module 40, a saturation headway acquisition module 50, an extended time period acquisition module 60, a lost time period acquisition module 70, and an idle time period acquisition module 80. The data acquisition module 10 is used to acquire video image data and radar detection data. The absolute spatiotemporal information acquisition module 20 is used to fuse the video image data and radar detection data to obtain the relative spatiotemporal information of each vehicle relative to the intersection, and to obtain the absolute spatiotemporal information of all vehicles at the intersection based on the relative spatiotemporal information. The stage time acquisition module 30 is used to acquire traffic timing information and map information of the intersection, and to obtain the stage time of each lane based on the traffic timing information and map information. The marker time acquisition module 40 is used to time-mark each vehicle that crosses the stop line within the stage time of each lane based on the absolute spatiotemporal information of each vehicle, obtaining the marker time of each vehicle. The saturated headway acquisition module 50 is used to obtain the saturated headway of the queuing platoon based on traffic timing information and map information, according to the absolute spatiotemporal information of each vehicle.

[0127] The extended time period acquisition module 60 is used to extend the marked time of each vehicle according to the saturated headway, thereby obtaining the extended time period for each vehicle. The loss time period acquisition module 70 is used to obtain the start-up sub-loss time period and the clearing sub-loss time period within the stage time of each lane according to the absolute spatiotemporal information of each vehicle. The idle time period acquisition module 80 is used to remove the extended time period, start-up sub-loss time period, and clearing sub-loss time period of each vehicle within the stage time of each lane, thereby obtaining the idle time period for each lane.

[0128] In this embodiment, the description of the data acquisition module 10 can be referred to the description of S10 in the above embodiment. The description of the absolute spatiotemporal information acquisition module 20 can be referred to the description of S20 in the above embodiment. The description of the stage time acquisition module 30 can be referred to the description of S30 in the above embodiment. The description of the marker time acquisition module 40 can be referred to the description of S40 in the above embodiment. The description of the saturated headway acquisition module 50 can be referred to the description of S50 in the above embodiment. The description of the extended time period acquisition module 60 can be referred to the description of S60 in the above embodiment. The description of the lost time period acquisition module 70 can be referred to the description of S70 in the above embodiment. The description of the idle time period acquisition module 80 can be referred to the description of S80 in the above embodiment.

[0129] In one embodiment, the radar-visual fusion-based idle time calculation system 100 further includes a direction-level idle time period acquisition module and a direction-level occupancy time period acquisition module. The direction-level idle time period acquisition module is used to perform intersection calculations on the idle time periods of each lane in the same direction in the time dimension to obtain the direction-level idle time period. The direction-level occupancy time period acquisition module is used to perform union calculations on the extended time period, start-up sub-loss time period, and clearing sub-loss time period of each vehicle within the stage time for each lane in the same direction in the time dimension to obtain the direction-level occupancy time period.

[0130] In this embodiment, the description of the directional stage idle time period acquisition module can be found in the description of S901 in the above embodiment. The description of the directional stage occupied time period acquisition module can be found in the description of S902 in the above embodiment.

[0131] In one embodiment, the idle time calculation system 100 based on radar-visual fusion further includes an intersection idle time period acquisition module and an intersection occupancy time period acquisition module. The intersection idle time period acquisition module calculates the intersection idle time period by performing intersection calculations on the directional idle time periods for each direction of the intersection in the time dimension. The intersection occupancy time period acquisition module calculates the intersection occupancy time period by performing union calculations on the directional occupancy time periods for each direction of the intersection in the time dimension. Alternatively, the intersection idle time period acquisition module calculates the intersection idle time periods for each lane of the intersection in the time dimension. The intersection occupancy time period acquisition module calculates the intersection occupancy time period by performing union calculations on the extended time period, start-up sub-loss time period, and clearing sub-loss time period for each vehicle within the stage time of each lane of the intersection in the time dimension.

[0132] In this embodiment, the description of the intersection vacancy time period acquisition module can be found in S903 or S905 of the above embodiments. The description of the intersection occupancy time period acquisition module can be found in S904 or S906 of the above embodiments.

[0133] In one embodiment, the radar-visual fusion-based idle time calculation system 100 further includes a phase idle time period acquisition module and a phase occupied time period acquisition module. The phase idle time period acquisition module is used to calculate the intersection of the idle time periods of each lane in each phase within a cycle in the time dimension to obtain the phase idle time period. The phase occupied time period acquisition module is used to calculate the union of the extended time period, start-up sub-loss time period, and clearing sub-loss time period of each vehicle within the phase time of each lane in each phase within a cycle in the time dimension to obtain the phase occupied time period.

[0134] In this embodiment, the description of the phase idle time period acquisition module can be found in the description of S907 in the above embodiment. The description of the phase occupied time period acquisition module can be found in the description of S908 in the above embodiment.

[0135] In one embodiment, the idle time calculation system 100 based on radar-visual fusion further includes a periodic idle time period acquisition module. The periodic idle time period acquisition module is used to perform union calculation on the idle time periods of each stage within a period in the time dimension to obtain the periodic idle time period.

[0136] In this embodiment, the relevant description of the periodic idle time period acquisition module can be referred to the relevant description of S909 in the above embodiment.

[0137] In the various embodiments described above, the specific order or hierarchy of steps in the disclosed process is an example of an exemplary method. Based on design preferences, it should be understood that the specific order or hierarchy of steps in the process may be rearranged without departing from the scope of this disclosure. The appended method claims provide elements of various steps in an exemplary order and are not intended to limit the specific order or hierarchy described.

[0138] Those skilled in the art will also understand that the various illustrative logical blocks, modules, and steps listed in the embodiments of this application can be implemented by electronic hardware, computer software, or a combination of both. To clearly demonstrate the interchangeability of hardware and software, the functions of the various illustrative components, modules, and steps described above have been generally described. Whether such functionality is implemented through hardware or software depends on the specific application and the overall system design requirements. Those skilled in the art can implement the described functions using various methods for each specific application, but such implementation should not be construed as exceeding the scope of protection of the embodiments of this application.

[0139] The various illustrative logic blocks or modules described in the embodiments of this application can be implemented or operate the described functions using a general-purpose processor, digital signal processor, application-specific integrated circuit (ASIC), field-programmable gate array or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof. The general-purpose processor can be a microprocessor; alternatively, it can also be any conventional processor, controller, microcontroller, or state machine. The processor can also be implemented using a combination of computing devices, such as a digital signal processor and a microprocessor, multiple microprocessors, one or more microprocessors combined with a digital signal processor core, or any other similar configuration.

[0140] The steps of the methods or algorithms described in the embodiments of this application can be directly embedded in hardware, a software module executed by a processor, or a combination of both. The software module can be stored in RAM, flash memory, ROM, EPROM, EEPROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium in the art. Exemplarily, the storage medium can be connected to the processor so that the processor can read information from and write information to the storage medium. Optionally, the storage medium can also be integrated into the processor. The processor and storage medium can be housed in an ASIC, which can be housed in a user terminal. Optionally, the processor and storage medium can also be housed in different components of the user terminal.

[0141] The specific embodiments described above further illustrate the purpose, technical solution, and beneficial effects of this application. It should be understood that the above description is only a specific embodiment of this application and is not intended to limit the scope of protection of this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A method for calculating idle discharge time based on radar-visual fusion, characterized in that, include: Acquire video image data and radar detection data; The video image data and the radar detection data are fused to obtain the relative spatiotemporal information of each vehicle relative to the intersection, and the absolute spatiotemporal information of all vehicles at the intersection is obtained based on the relative spatiotemporal information. Obtain traffic timing information and map information of the intersection, and obtain the stage time for each lane based on the traffic timing information and map information; Based on the absolute spatiotemporal information of each vehicle, time stamps are performed on each vehicle that passes the stop line within the phase time of each lane to obtain the stamp time of each vehicle; Based on the traffic timing information and the map information, the saturation headway of the queuing platoon is obtained according to the absolute spatiotemporal information of each vehicle. Based on the saturated headway, the marked time for each vehicle is extended to obtain the extended time period for each vehicle; Based on the absolute spatiotemporal information of each vehicle, the start-up sub-loss time period and the clearing sub-loss time period within the stage time of each lane are obtained; The idle time period for each lane is obtained by removing the extended time period, the starter loss time period, and the clearing loss time period for each vehicle within the phase time of each lane.

2. The method for calculating idle release time based on radar-visual fusion according to claim 1, characterized in that, The idle time calculation method based on radar-visual fusion also includes: In the time dimension, the intersection of the empty time periods of each lane in the same direction is calculated to obtain the direction-level empty time periods; In the time dimension, the extended time period, the start-up sub-loss time period, and the clearing sub-loss time period of each vehicle in each lane in the same direction within the stage time are calculated by union to obtain the direction-level occupancy time period.

3. The method for calculating idle release time based on radar-visual fusion according to claim 2, characterized in that, The idle time calculation method based on radar-visual fusion also includes: In the time dimension, the intersection time periods of the directional-level empty parking periods for each direction of the intersection are calculated to obtain the intersection empty parking time periods. In the time dimension, the intersection occupancy time period is obtained by performing a union calculation on the direction-level occupancy time period for each direction of the intersection; or In the time dimension, the intersection of the empty time periods of each lane at the intersection is calculated to obtain the empty time periods of the intersection. In the time dimension, the intersection occupancy time period is obtained by performing a union calculation on the extended time period, the start-up sub-loss time period, and the clearing sub-loss time period for each vehicle in each lane of the intersection within the stage time.

4. The method for calculating idle release time based on radar-visual fusion according to claim 1, characterized in that, The idle time calculation method based on radar-visual fusion also includes: In the time dimension, the intersection of the empty time periods of each lane in each stage within a cycle is calculated to obtain the stage empty time period. In the time dimension, the extended time period, the start-up loss time period, and the clearing loss time period of each vehicle in each lane of each stage within a cycle are calculated by union to obtain the stage occupancy time period.

5. The method for calculating idle release time based on radar-visual fusion according to claim 4, characterized in that, The idle time calculation method based on radar-visual fusion also includes: In the time dimension, the idle time periods of each stage within a cycle are combined to obtain the periodic idle time period.

6. A system for calculating idle discharge time based on radar-visual fusion, characterized in that, include: The data acquisition module is used to acquire video image data and radar detection data; The absolute spatiotemporal information acquisition module is used to fuse the video image data with the radar detection data to obtain the relative spatiotemporal information of each vehicle relative to the intersection, and to obtain the absolute spatiotemporal information of all vehicles at the intersection based on the relative spatiotemporal information. The stage time acquisition module is used to acquire traffic timing information and map information of the intersection, and obtain the stage time of each lane based on the traffic timing information and map information; The marker time acquisition module is used to time-mark each vehicle that passes the stop line within the phase time of each lane based on the absolute spatiotemporal information of each vehicle, and obtain the marker time of each vehicle. The saturated headway acquisition module is used to obtain the saturated headway of the queuing platoon based on the traffic timing information and the map information, according to the absolute spatiotemporal information of each vehicle. The extended time period acquisition module is used to extend the marked time of each vehicle according to the saturated headway, so as to obtain the extended time period of each vehicle. The loss time period acquisition module is used to obtain the start-up sub-loss time period and the clearing sub-loss time period within the stage time of each lane based on the absolute spatiotemporal information of each vehicle. The idle time period acquisition module is used to remove the extended time period, the start-up loss time period, and the clearing loss time period of each vehicle within the phase time of each lane to obtain the idle time period of each lane.

7. The idle release time calculation system based on radar-visual fusion according to claim 6, characterized in that, The idle time calculation system based on radar-visual fusion also includes: The directional empty time period acquisition module is used to perform intersection calculation on the empty time periods of each lane in the same direction in the time dimension to obtain the directional empty time period. The directional occupancy time period acquisition module is used to perform a union calculation on the extended time period, the start-up sub-loss time period, and the clearing sub-loss time period of each vehicle in each lane in the same direction within the time of the stage, in the time dimension, to obtain the directional occupancy time period.

8. The idle release time calculation system based on radar-visual fusion according to claim 7, characterized in that, The idle time calculation system based on radar-visual fusion also includes: The intersection vacancy period acquisition module is used to perform intersection calculation on the vacancy period of each direction of the intersection in the time dimension to obtain the intersection vacancy period. The intersection occupancy time period acquisition module is used to perform a union calculation on the direction-level occupancy time periods for each direction of the intersection in the time dimension to obtain the intersection occupancy time period; or The intersection vacancy period acquisition module is used to perform intersection calculation on the vacancy period of each lane of the intersection in the time dimension to obtain the intersection vacancy period. The intersection occupancy time period acquisition module is used to perform a union calculation on the extended time period, the start-up sub-loss time period, and the clearing sub-loss time period of each vehicle in each lane of the intersection within the stage time in the time dimension to obtain the intersection occupancy time period.

9. The idle release time calculation system based on radar-visual fusion according to claim 6, characterized in that, The idle time calculation system based on radar-visual fusion also includes: The phased empty time period acquisition module is used to calculate the intersection of the empty time periods of each lane in each phase within a cycle in the time dimension to obtain the phased empty time period. The phase occupancy time period acquisition module is used to perform a union calculation on the extended time period, the start-up sub-loss time period, and the clearing sub-loss time period for each vehicle in each lane of each phase within a cycle in the time dimension, to obtain the phase occupancy time period.

10. The idle release time calculation system based on radar-visual fusion according to claim 9, characterized in that, The idle time calculation system based on radar-visual fusion also includes: The periodic idle time period acquisition module is used to perform union calculation on the idle time periods of each stage within a period in the time dimension to obtain the periodic idle time period.

Images