A vehicle and pedestrian recognition and warning system based on intersection imagery

Abstract

The application discloses a kind of vehicle and pedestrian identification early warning systems based on intersection image, and the application relates to intelligent transportation technology field, and identification early warning system includes image acquisition module, image recognition module, data processing module and alarm module, image acquisition module includes remote camera and intersection camera, remote camera is used to shoot road far place data, intersection camera is photographed to both sides of intersection, the advantage of the application is that vehicle data captured by remote camera is handled in advance by data processing module, then data cache unit is used to cache the vehicle data after processing, data processing module is used to analyze which vehicle driving data is affected when pedestrian passes through intersection, then driving data is matched with the data in data cache unit, and then when data matching is successful, alarm module is alarmed by buzzer, to improve the response efficiency of identification early warning system.

Description

Technical Field

[0001] This invention relates to the field of intelligent transportation technology, specifically to a vehicle and pedestrian recognition and early warning system based on intersection images. Background Technology

[0002] Traditional intersection safety measures mainly rely on traffic lights and road markings. However, these measures often have limitations in complex traffic environments. Traffic lights can only provide fixed signal indications and cannot adapt to dynamically changing traffic conditions in real time. Road markings may also be ignored due to obstructed visibility or driver negligence. To improve intersection safety, vehicle and pedestrian recognition and early warning systems utilize advanced imaging technology and intelligent algorithms to monitor and identify vehicles and pedestrians at intersections in real time. Through cameras installed at intersections, the system can acquire real-time images of the intersection and use image processing and pattern recognition technology to quickly and accurately detect the position and movement status of vehicles and pedestrians. When the system detects potential dangers, it can quickly issue early warning signals to remind drivers and pedestrians to take appropriate measures to avoid traffic accidents.

[0003] However, the system's reaction speed is crucial for effective early warning. If the system's reaction speed is too slow, it may not be able to issue early warning signals in time, resulting in dangerous situations not being avoided in time. To address this, we propose a vehicle and pedestrian recognition early warning system based on intersection images. Summary of the Invention

[0004] The purpose of this invention is to provide an intelligent early warning system for intersections based on multimodal behavior prediction and dynamic risk grid, so as to solve the problems of high response delay and lack of predictive ability mentioned in the background art.

[0005] Technical solution

[0006] To achieve the above objectives, the present invention provides the following technical solution:

[0007] An intelligent early warning system for intersections based on multimodal behavior prediction and dynamic risk grid is characterized by comprising: a roadside holographic perception base station, an edge computing fusion unit, a behavior prediction engine, a dynamic risk grid generator, and a vehicle-road cooperative broadcasting unit;

[0008] Roadside holographic sensing base station:

[0009] Includes high-definition optical cameras, millimeter-wave radar, and traffic light signal analyzers;

[0010] The high-definition optical camera is used to collect visual image data of intersections and distant roads;

[0011] The millimeter-wave radar is used to collect precise point cloud data of moving targets (vehicles, pedestrians), including instantaneous speed, acceleration, and precise distance relative to roadside equipment.

[0012] The traffic light signal parser is used to obtain the traffic light status and remaining duration for the current and next cycle in real time.

[0013] Edge computing fusion unit:

[0014] It is connected to the camera and radar respectively, and is used to perform spatiotemporal alignment and fusion of visual image data and radar point cloud data to generate structured trajectory data containing target unique ID, type, latitude and longitude coordinates, speed and heading angle.

[0015] Behavioral prediction engine:

[0016] Built-in behavior prediction model based on temporal convolutional networks or long short-term memory networks;

[0017] Receive the structured trajectory data and output the probability of a pedestrian's intention to cross the street in the next 3-5 seconds and the probability of a vehicle's trajectory deviation at the intersection.

[0018] Dynamic Risk Grid Generator:

[0019] The intersection area is divided into continuous two-dimensional risk grid units;

[0020] Based on the trajectory data, predicted intent probability, and real-time traffic light data, the risk coefficient R of each grid cell is calculated using a risk field model.

[0021] The formula for calculating the risk coefficient R is as follows:

[0022]

[0023] in, The instantaneous speed of vehicles within the grid. This represents the relative distance between pedestrians and vehicles within the grid. The probability of a pedestrian's intention to cross the street. The field-of-view occlusion coefficient calculated based on radar point cloud density. The danger level of a traffic light countdown.

[0024] Vehicle-Road Cooperative Broadcasting Unit:

[0025] When the risk coefficient of a certain grid When the preset dynamic threshold is exceeded, the system generates an early warning message;

[0026] The warning message is broadcast to connected vehicles within a preset range via cellular vehicle-to-everything (V2X) or dedicated short-range communication technology, while simultaneously triggering roadside high-intensity warning lights and directional buzzers.

[0027] As a preferred embodiment of the present invention, the dynamic risk grid generator further includes an adaptive threshold adjustment module, used to dynamically adjust the weight coefficients in the risk coefficient R based on historical accident data and traffic flow through reinforcement learning. To adapt to traffic safety needs at different times and at different intersections.

[0028] As a preferred embodiment of the present invention, the behavior prediction engine further includes an adversarial generative network data enhancement module for generating rare "ghost peek" scene data to optimize the prediction accuracy of the model in long-tailed distribution scenarios.

[0029] As a preferred embodiment of the present invention, the edge computing fusion unit is deployed on the roadside computing node. When a spatiotemporal intersection point is detected between the vehicle trajectory and the pedestrian trajectory and the predicted intention points to the intersection point, the warning data is directly sent through the broadcast unit, with a response latency of less than 50 milliseconds.

[0030] Compared with the prior art, the beneficial effects of the present invention by adopting the above technical solution are as follows:

[0031] 1. Pioneering Dynamic Risk Grid Technology: This invention upgrades traditional point-based target monitoring to area-based risk field monitoring. Through grid-based risk assessment, it can more intuitively and comprehensively reflect the real-time safety situation of complex intersection areas.

[0032] 2. Possesses behavioral prediction capabilities: Introduces deep learning models to predict the future trajectories and intentions of pedestrians and vehicles, upgrading the early warning mechanism from "post-event response" to "pre-event prediction," effectively addressing sudden dangers such as "ghost pedestrians";

[0033] 3. Multimodal fusion perception: Combining vision and millimeter-wave radar, it overcomes the shortcomings of single vision in insufficient light or bad weather, and achieves all-weather, high-precision target tracking;

[0034] 4. Low-latency architecture with edge-edge collaboration: Data fusion and risk calculation are completed by using edge computing, and C-V2X direct communication is combined to ensure that early warning information can reach traffic participants at a very fast speed, which significantly improves the practicality and reliability of the system. Attached Figure Description

[0035] Figure 1 This is a flowchart of the identification and early warning system in an embodiment of the present invention. Detailed Implementation

[0036] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0037] Example 1

[0038] See Figure 1 This embodiment provides an intelligent early warning system for intersections based on multimodal behavior prediction and dynamic risk grid.

[0039] The system first collects data through roadside holographic sensing base stations. High-definition cameras capture image sequences containing pedestrians and vehicles, while millimeter-wave radar simultaneously acquires distance and speed information of the targets. A traffic light signal analyzer connects to the traffic signal controller via an interface to obtain real-time light status.

[0040] The acquired data is transmitted to the edge computing fusion unit. This unit first maps the target point cloud in the radar coordinate system to the image pixel coordinate system, achieving spatiotemporal synchronization. Subsequently, a smooth trajectory sequence is generated using a Kalman filter tracking algorithm.

[0041] The behavior prediction engine receives trajectory sequences. For example, when a pedestrian is detected at the starting point of a zebra crossing and has a tendency to move towards the center of the road, the temporal convolutional network model will combine historical trajectories and accelerations to output an 85% probability that the pedestrian will "cross the road" within the next 3 seconds.

[0042] The dynamic risk grid generator establishes a 10cm × 10cm grid with the center of the intersection as the origin. For each grid, its risk coefficient is calculated. Assuming a pedestrian is predicted to cross in grid A, and there is a vehicle traveling at 50km / h in the adjacent lane, with a certain blind spot for the vehicle, the risk coefficient R is calculated to be 0.95 (high risk).

[0043] When the risk factor exceeds the preset threshold of 0.8, the vehicle-to-infrastructure (V2I) broadcast unit immediately generates a TS stream and broadcasts a warning message "Pedestrian ahead, please slow down" via the RSU (Roadside Unit) PC5 interface. Simultaneously, roadside spotlights project a red warning area under the pedestrian's feet to alert both the driver and pedestrian.

[0044] Example 2

[0045] Based on Example 1, this example is optimized for complex intersection scenarios. An adaptive threshold adjustment module is introduced into the dynamic risk grid generator.

[0046] Specifically, at an intersection near a school, pedestrian traffic is high during morning and evening rush hours. The system, through a reinforcement learning model, discovered that the original fixed threshold (0.8) would cause warnings to be issued too frequently, resulting in "warning fatigue." Therefore, the system automatically adjusted the weighting coefficients, increasing the probability of pedestrian intent. weight This also reduced the vehicle speed. weight This makes the calculation of the risk coefficient R focus more on the actual behavior of pedestrians, thereby reducing invalid warnings while ensuring safety.

[0047] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely illustrative of the principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of the present invention is defined by the appended claims and their equivalents.

Claims

1. A vehicle and pedestrian recognition and early warning system based on intersection images, characterized in that: include: Roadside holographic sensing base stations are used to collect visual images, radar point clouds, and traffic light status data at intersections. An edge computing fusion unit is used to perform spatiotemporal alignment and fusion of the visual image and radar point cloud to generate structured trajectory data; The behavior prediction engine, with a built-in time series model, is used to predict the probability of pedestrians' crossing intentions and the probability of vehicle trajectory deviations based on the structured trajectory data. A dynamic risk grid generator is used to divide the intersection into a two-dimensional grid and calculate the dynamic risk coefficient of each grid cell by combining the trajectory data, predicted probability and traffic light data. The vehicle-road cooperative broadcasting unit is used to issue early warning information to surrounding vehicles and pedestrians through vehicle-to-everything (V2X) communication when the dynamic risk coefficient exceeds the threshold.

2. The vehicle and pedestrian recognition and early warning system based on intersection images according to claim 1, characterized in that: The formula for calculating the risk coefficient in the dynamic risk grid generator is as follows: in, For vehicle speed, For driving distance, For the probability of intent, The occlusion coefficient is... The danger level of traffic lights, to These are preset or adaptive weights.

3. The vehicle and pedestrian recognition and early warning system based on intersection images according to claim 2, characterized in that: The behavior prediction engine is built on a temporal convolutional network or a long short-term memory network and is used to output the probability of motion intention in the next 3 to 5 seconds.

4. The vehicle and pedestrian recognition and early warning system based on intersection images according to claim 2, characterized in that: The edge computing fusion unit is deployed at roadside edge nodes to achieve millisecond-level data processing and risk calculation.

5. A vehicle and pedestrian recognition and early warning system based on intersection images according to claim 4, characterized in that: The dynamic risk grid generator also includes an adaptive threshold adjustment module, used to dynamically adjust the weight coefficients based on historical data using a reinforcement learning model. to .

6. A vehicle and pedestrian recognition and early warning system based on intersection images according to claim 4, characterized in that: The vehicle-road cooperative broadcasting unit supports cellular vehicle networking or dedicated short-range communication technology and is linked to roadside warning lights and buzzers.

Images