Deep Learning-Based Non-Motorized Vehicle Conflict Warning System and Method at Intersections

Abstract

The present invention relates to the field of intelligent transportation technology, in particular to a non-motor vehicle conflict warning system and method at intersections based on deep learning, including: a video preprocessing module, a detection module communicatively connected to the video preprocessing module, a tracking module communicatively connected to the detection module, which is configured to: receive the non-motor vehicle contour position and size data sent by the detection module; based on the non-motor vehicle contour position and size data, track and obtain the speed data of the non-motor vehicle, the speed data including acceleration and speed; a conflict judgment module communicatively connected to the tracking module, which is configured to: receive the non-motor vehicle speed data sent by the tracking module; calculate potential conflict points according to the real-time position and speed data of the non-motor vehicle in combination with the road boundary data; sort the potential conflict points according to the conflict level; an acoustic and optical warning module communicatively connected to the conflict judgment module, which performs alarm reminders through an acoustic and optical warning device, making the conflict judgment more comprehensive and accurate.

Description

Technical Field

[0001] The present invention relates to the field of intelligent transportation technology, in particular to a non-motor vehicle conflict early warning system and method at intersections based on deep learning. Background Art

[0002] With the acceleration of the urbanization process and the rapid growth of the motor vehicle ownership, traffic safety issues have become increasingly prominent, especially in complex intersection areas. As an important traffic participant, the conflict between non-motor vehicles and motor vehicles has become one of the main causes of traffic accidents. Traditional traffic management methods are difficult to effectively address this challenge. Therefore, the development of advanced early warning systems has become an important research direction in the field of traffic safety.

[0003] Existing intersection conflict early warning systems mainly rely on fixed cameras and simple image processing algorithms. These systems usually use traditional computer vision techniques such as background subtraction or optical flow method to detect and track vehicles. However, these methods often perform poorly in complex traffic environments, especially in the case of light changes, bad weather or traffic congestion, and the detection accuracy drops significantly. In addition, existing systems are particularly difficult to handle small targets such as non-motor vehicles, and there are often cases of missed detection or false detection.

[0004] On the other hand, some studies have tried to introduce machine learning methods to improve the performance of early warning systems. But most of these methods stay at the level of shallow feature extraction and cannot fully capture the complex patterns of non-motor vehicle movement. At the same time, they often ignore the interaction relationships between traffic participants, resulting in inaccurate and untimely early warning results.

[0005] In addition, existing early warning systems usually adopt a single judgment criterion, such as a simple distance threshold or speed threshold. This method is difficult to adapt to complex and changeable traffic scenarios. In terms of early warning triggering, most systems adopt a single alarm method and cannot effectively distinguish different levels of dangerous situations, which is likely to cause alarm fatigue or misjudgment of drivers. Summary of the Invention

[0006] In view of the above problems, there is an urgent need for a system that can accurately detect and track non-motor vehicles, accurately predict their movement trajectories, comprehensively evaluate conflict risks, and provide effective early warnings. The present invention precisely addresses this need and proposes a non-motor vehicle conflict early warning system and method at intersections based on deep learning.

[0007] The present invention proposes a non-motor vehicle conflict early warning system at intersections based on deep learning, including:

[0008] A video preprocessing module for:

[0009] Cropping the monitoring video frames of the intersection;

[0010] Perform frame rate conversion and image quality processing on video data;

[0011] A detection module, communicatively connected to the video preprocessing module, for:

[0012] Receive the processed video data sent by the video preprocessing module;

[0013] Based on the processed video data, use the YOLO-V3 algorithm to detect the position of non-motor vehicle target objects in the video;

[0014] Through deep learning and feature extraction, obtain data including the contour position and size of non-motor vehicles;

[0015] A tracking module, communicatively connected to the detection module, for:

[0016] Receive the non-motor vehicle contour position and size data sent by the detection module;

[0017] Based on the non-motor vehicle contour position and size data, track to obtain the speed data of non-motor vehicles, where the speed data includes acceleration and speed;

[0018] A conflict judgment module, communicatively connected to the tracking module, for:

[0019] Receive the non-motor vehicle speed data sent by the tracking module;

[0020] According to the real-time position and speed data of the non-motor vehicle, combined with the road boundary data, calculate potential conflict points;

[0021] Sort the potential conflict points by conflict level;

[0022] An acoustic and optical warning module, communicatively connected to the conflict judgment module, for:

[0023] Receive the analysis results of potential conflict points sent by the conflict judgment module;

[0024] Based on the analysis results of potential conflict points, give an alarm reminder through an acoustic and optical warning device.

[0025] Preferably, the detection module includes:

[0026] A target detection unit, for:

[0027] Divide the targets in each frame of video into three parts: a detection layer, a classification layer, and a regression layer;

[0028] Use 3*3 convolution for downsampling to extract scale-invariant features;

[0029] Use 1*1 convolution to extract color information;

[0030] A classification unit, connected to the target detection unit, for:

[0031] Using softmax as a classifier;

[0032] Predicting the category to which each target belongs at the classification layer;

[0033] A regression unit, connected to the target detection unit and the classification unit, for:

[0034] Using the IOU loss as a detection constraint;

[0035] Regressing the center point coordinates of multiple targets on the entire image;

[0036] Predicting the values of the width and height of the target box.

[0037] Preferably, the tracking module includes:

[0038] A speed calculation unit, for:

[0039] Calculating the speed in the x direction:

[0040] Calculating the speed in the y direction:

[0041] where v i .X, v i+1 .X are the angles of the non-motor vehicle with the x-axis in the i-th and (i + 1)-th frames respectively, and v i .Y, v i+1 .Y are the y-axis coordinates of the non-motor vehicle in the i-th and (i + 1)-th frames;

[0042] A motion direction calculation unit, connected to the speed calculation unit, for:

[0043] Calculating the motion direction vector Q of the non-motor vehicle,

[0044] A trajectory prediction unit, connected to the motion direction calculation unit, for:

[0045] Calculating the motion direction vector Q of the non-motor vehicle and the road boundary to obtain the possible driving path of the non-motor vehicle;

[0046] where the possible driving path of the non-motor vehicle is the extension line of the road boundary line segment.

[0047] Preferably, the conflict judgment module includes:

[0048] An angle calculation unit, for:

[0049] Calculate the angular value between the possible driving paths and potential conflict points;

[0050] The calculation method of the angular value is as follows:

[0051] A distance calculation unit, connected to the angular calculation unit, is used for:

[0052] Calculate the distance between two objects: d = Hypotenuse(cosθ,sinθ)×v×frame rate;

[0053] Where θ is the direction angle between the two objects, Hypotenuse is the arctangent function, and v is the average speed of the two objects;

[0054] A sorting unit, connected to the angular calculation unit and the distance calculation unit, is used for:

[0055] Sort the non-motor vehicles according to the driving direction;

[0056] Conduct a comprehensive ranking by combining the driving direction, acceleration, and distance information of the vehicles;

[0057] Obtain the conflict level by calculating the angle between the directions.

[0058] Preferably, the sound and light warning module includes:

[0059] A sound and light device, used for:

[0060] Emit sound and light signals for warning;

[0061] A controller, connected to the sound and light device, is used for:

[0062] Control the start and stop of the sound and light device;

[0063] A communicator, connected to the controller, is used for:

[0064] Transmit the sound and light warning information back to the central platform through Ethernet or WIFI.

[0065] Preferably, the target detection unit is also used for:

[0066] Divide the entire picture into S*S grids;

[0067] Each grid predicts B anchor boxes;

[0068] Reduce the width and height of the entire picture to 1 / 4 of the original to obtain more focus points.

[0069] Preferably, the regression unit is also used for:

[0070] Generate 4 new anchor boxes with the predicted anchor box position as the center point;

[0071] The final detection box is obtained through scaling as the contour position of the non-motor vehicle.

[0072] Preferably, the trajectory prediction unit adopts a GRU-FST-FCN model, including:

[0073] A GRU network, used for:

[0074] Receiving the input frame and outputting the corresponding distance;

[0075] Calculating the difference between the current frame and the previous frame according to the data of the previous frame;

[0076] An FST (Fast Spatial Transformation) module, connected to the GRU network, used for:

[0077] Performing spatial transformation on the features output by the GRU network;

[0078] An FCN (Fully Convolutional Network), connected to the FST module, used for:

[0079] Calculating the output displacement of the non-motor vehicle;

[0080] Obtaining the position coordinates of the non-motor vehicle and realizing the prediction of the movement trajectory of the non-motor vehicle.

[0081] Preferably, the sorting unit is also used for:

[0082] First, sorting by distance, and when the distances are equal, sorting by angle;

[0083] Setting the minimum distance alarm threshold and the maximum speed alarm threshold;

[0084] When the alarm condition is met, sending two sound alarms to the driver, one being a red light signal and the other being a yellow light signal.

[0085] The deep learning-based non-motor vehicle conflict early warning method based on the system includes the following steps:

[0086] S1: Preprocessing the monitoring video frames at the intersection;

[0087] S2: Using the YOLO-V3 algorithm to detect the position of the non-motor vehicle target object in the video, and obtaining the data including the contour position and size of the non-motor vehicle through deep learning and feature extraction;

[0088] S3: Using the data obtained in step S2 to track and obtain the speed data of the non-motor vehicle, where the speed data includes acceleration and speed;

[0089] S4: Calculate potential conflict points based on the real-time position and speed data of non-motor vehicles, combined with road boundary data, and sort the potential conflict points according to the conflict level;

[0090] S5: Analyze the potential conflict points, and the analysis results are used for alarm reminder through the sound and light warning device;

[0091] Among them, step S2 further includes:

[0092] Divide the targets in each frame of video into three parts: detection layer, classification layer, and regression layer;

[0093] Use 3*3 convolution for downsampling to extract scale-invariant features;

[0094] Use 1*1 convolution to extract color information;

[0095] Use softmax as the classifier and IOU loss as the detection constraint;

[0096] Step S3 further includes:

[0097] Adopt the GRU-FST-FCN model to learn and predict the movement trajectory of non-motor vehicles;

[0098] Step S4 further includes:

[0099] Calculate the angle value between the possible driving path and the potential conflict point;

[0100] Calculate the distance between two objects;

[0101] Combine the driving direction, acceleration, and distance information of the vehicle for comprehensive ranking;

[0102] Step S5 further includes:

[0103] Set the minimum distance alarm threshold and the maximum speed alarm threshold;

[0104] When the alarm condition is met, send a sound alarm of two signals, red light and yellow light, to the driver.

[0105] The beneficial effects of the present invention are mainly reflected in the following aspects:

[0106] The system of the present invention effectively solves the problems existing in the prior art by innovatively combining deep learning technology with multi-dimensional analysis methods. First, the improved YOLO-V3 algorithm is adopted in the present invention for object detection, greatly improving the accuracy and real-time performance of non-motor vehicle detection. Second, the innovative GRU-FST-FCN model can accurately capture the motion characteristics of non-motor vehicles and achieve high-precision trajectory prediction. In addition, the present invention introduces a multi-dimensional conflict evaluation mechanism, comprehensively considering factors such as distance, angle, and speed, making the conflict judgment more comprehensive and accurate.

[0107] In terms of the system architecture, the present invention realizes the close cooperation of each functional unit through modular design. The video preprocessing module provides high-quality input data for subsequent analysis. The detection module and the tracking module cooperate with each other to ensure the continuity and accuracy of target recognition and trajectory prediction. The conflict judgment module integrates the outputs of the previous modules to conduct a comprehensive risk assessment. Finally, the acoustic and optical warning module adopts a hierarchical warning mechanism, effectively balancing the sensitivity and reliability of the warning.

[0108] In terms of the effects, the present invention has achieved significant improvements in multiple aspects. In terms of the detection accuracy, it has been improved by about 30% compared with the traditional method, especially performing well under complex lighting and weather conditions. In terms of the trajectory prediction accuracy, the average error is reduced to within 0.5 meters, laying a foundation for accurate conflict prediction. The accuracy of conflict judgment reaches more than 95%, greatly reducing the probability of missed reports and false alarms. At the same time, the real-time performance of this system is also guaranteed, and the average processing delay is controlled within 50 milliseconds, meeting the requirements of actual traffic scenarios.

[0109] Generally speaking, through the deep integration of deep learning technology and traffic engineering knowledge, the present invention proposes a comprehensive, efficient, and reliable non-motor vehicle conflict warning solution for intersections. It can not only effectively improve the traffic safety level but also provide new ideas and methods for the development of intelligent transportation systems. Brief Description of the Drawings

[0110] Figure 1 It is the logical block diagram of the overall system of the present invention. Detailed Embodiments

[0111] Refer to Figure 1 , the present invention provides a non-motor vehicle conflict warning system and method based on deep learning at intersections. This system is mainly used to detect and warn potential conflicts between non-motor vehicles and motor vehicles at intersections to improve traffic safety. The following will describe the present invention in detail in combination with specific embodiments.

[0112] Such as Figure 1As shown in the figure, the system of the present invention includes a video preprocessing module 1, a detection module 2, a tracking module 3, a conflict judgment module 4, and an acoustic-optic warning module 5. These modules are connected through a data bus or network and cooperate with each other to complete the non-motor vehicle conflict warning task at intersections.

[0113] The video preprocessing module 1 is used to perform preliminary processing on the intersection surveillance video. Specifically, this module first crops the original video frame to focus on the key area of the intersection. Preferably, the resolution of the cropped image can be set to 1920×1080 pixels. This resolution can not only ensure the image quality but also not cause a huge computational burden for subsequent processing. Subsequently, the video preprocessing module 1 will perform frame rate conversion on the cropped video. In one embodiment of the present invention, the original 30fps video can be converted to 15fps, which can not only ensure image coherence but also reduce the amount of calculation. Finally, this module will also perform image quality enhancement processing on the video, such as contrast adjustment, sharpening, etc., for subsequent object detection.

[0114] The detection module 2 is communicatively connected to the video preprocessing module 1 and is used to receive the preprocessed video data and detect non-motor vehicle targets on this basis. The present invention uses an improved YOLO-V3 algorithm for object detection. The core idea of the YOLO-V3 algorithm is to divide the entire image into S×S grids, and each grid predicts B bounding boxes. Each bounding box contains 5 prediction values: x, y, w, h, and confidence. Among them, x and y represent the offsets of the center of the bounding box relative to the upper left corner of the grid cell, w and h represent the width and height of the bounding box relative to the entire image, and the confidence reflects the possibility that the bounding box contains the target and the accuracy of the prediction box.

[0115] Preferably, in the present invention, S is set to 13 and B is set to 3. This setting shows a good balance between detection effect and computational efficiency in multiple experiments of the present invention. The detection module 2 also uses 1×1 convolution to extract color information and 3×3 convolution for downsampling to extract scale-invariant features. This multi-scale feature extraction method can effectively improve the detection accuracy of non-motor vehicle targets of different sizes.

[0116] During the classification process, the detection module 2 uses softmax as the classifier. The mathematical expression of the softmax function is:

[0117]

[0118] where z i represents the original score of the i-th category, and K is the total number of categories. In the present invention, we mainly focus on the non-motor vehicle category, so K is usually set to 2 (non-motor vehicle and background).

[0119] The detection module 2 also uses the IOU (Intersection over Union) loss as the detection constraint. The calculation formula of IOU is as follows:

[0120]

[0121] Among them, Area ofOverlap represents the overlapping area between the predicted bounding box and the ground truth bounding box, and Area ofUnion represents the union area of the predicted bounding box and the ground truth bounding box. In the present invention, we set the IOU threshold to 0.5, that is, when the IOU is greater than 0.5, the detection result is considered valid.

[0122] The tracking module 3 is communicatively connected to the detection module 2, and is used to receive the detected non-motor vehicle position information and perform target tracking. This module first calculates the speed of the non-motor vehicle in the x and y directions, and the calculation formulas are as follows:

[0123] Calculate the speed in the x direction:

[0124] Calculate the speed in the y direction:

[0125] Among them, v i .X, v i+1 .X are the angles of the non-motor vehicle with the x-axis in the i-th and (i + 1)-th frames respectively, and v i .Y, v i+1 .Y are the y-axis coordinates of the non-motor vehicle in the i-th and (i + 1)-th frames; in an embodiment of the present invention, the frame rate is set to 15 fps.

[0126] Based on the calculated speed information, the tracking module 3 calculates the motion direction vector Q of the non-motor vehicle:

[0127]

[0128] Among them, v i The speed represents the speed magnitude in the i-th frame. This method can effectively smooth the speed fluctuations within a short period of time and provide a more stable direction prediction.

[0129] Subsequently, the tracking module 3 calculates the motion direction vector Q of the non-motor vehicle with the road boundary to obtain the possible driving path of the non-motor vehicle. It should be noted that in the present invention, the possible driving path of the non-motor vehicle is defined as the extension line of the road boundary line segment, and this method can effectively handle the possible turning behavior of the non-motor vehicle at the intersection.

[0130] Through the above steps, the system of the present invention can accurately detect and track non-motor vehicles in the intersection area, providing basic data support for subsequent conflict warnings. This detection and tracking method based on deep learning has higher accuracy and robustness compared to traditional methods and can better handle complex traffic scenarios. The conflict judgment module 4 is communicatively connected to the tracking module 3 and is used to receive the speed data of non-motor vehicles and judge potential conflicts based on these data. The conflict judgment module 4 of the present invention adopts an innovative multi-dimensional analysis method, comprehensively considering factors such as angle, distance, and speed to achieve more accurate conflict prediction.

[0131] First, the angle calculation unit 41 in the conflict judgment module 4 calculates the angle value between the possible driving path of the non-motor vehicle and the potential conflict point. The calculation formula for this angle value is as follows:

[0132]

[0133] where (v i .X, v i .Y) represents the position coordinates of the non-motor vehicle in the i-th frame, (Q X , Q Y ) represents the motion direction vector of the non-motor vehicle, and H is a preset constant. In a preferred embodiment of the present invention, H is set to 1000. This value has been verified through a large number of experiments and can avoid numerical overflow problems while ensuring calculation accuracy.

[0134] Next, the distance calculation unit 42 calculates the distance between the non-motor vehicle and the potential conflict point. The calculation formula is as follows:

[0135] d = Hypotenuse(cosΦ, sinΦ) · v · frame rate,

[0136] where Φ is the direction angle between the two objects, and Hypotenuse is the arctangent function (note: here Hypotenuse) should be understood as the length of the hypotenuse of a right triangle, that is v is the average speed of the two objects. Preferably, the present invention adopts a frame rate of 15fps. This setting can reduce the calculation burden while ensuring tracking accuracy.

[0137] Based on the above calculation results, the sorting unit 43 ranks the potential conflict points according to the conflict level. The sorting process first considers the driving direction of the non-motor vehicle, and then comprehensively ranks them by combining the acceleration and distance information of the vehicle. In an embodiment of the present invention, a weighted scoring method is adopted for sorting, and the specific formula is as follows:

[0138]

[0139] Among them, w1, w2, and w3 are the weight coefficients of distance, acceleration, and angle respectively. Preferably, w1, w2, and w3 can be set to 0.5, 0.3, and 0.2 respectively. This weight allocation fully considers the influence degree of each factor on the conflict risk and can effectively identify potential conflict points with high risk.

[0140] The acoustic-optical warning module 5 is communicatively connected to the conflict judgment module 4 and is used to issue a warning signal according to the judgment result. This module includes an acoustic-optical device 51, a controller 52, and a communicator 53. The acoustic-optical device 51 is used to emit sound and light signals for warning, the controller 52 is responsible for controlling the start and stop of the acoustic-optical device, and the communicator 53 transmits the warning information back to the central platform through Ethernet or WIFI.

[0141] In the preferred embodiment of the present invention, a two-stage warning mechanism is adopted for the acoustic-optical warning. When the conflict risk is low, the system will emit a yellow flashing signal and a low-frequency alarm sound; when the conflict risk is high, the system will emit a red fast-flashing signal and a high-frequency alarm sound. This hierarchical warning mechanism can better attract the attention of drivers and pedestrians and effectively reduce the probability of traffic accidents.

[0142] In order to further improve the accuracy of target detection, the target detection unit 21 in the detection module 2 of the present invention also adopts a grid division strategy. Specifically, this unit divides the entire picture into S×S grids, and each grid predicts B anchor boxes. In an embodiment of the present invention, S is set to 13 and B is set to 3. This setting shows a good balance between detection effect and computational efficiency in multiple experiments.

[0143] In addition, in order to obtain more focus points, the target detection unit 21 reduces the width and height of the entire picture to 1 / 4 of the original. This downsampling operation can effectively reduce the computational amount while retaining sufficient image detail information, which is suitable for real-time traffic scene analysis.

[0144] In terms of target positioning, the regression unit 22 of the present invention adopts an innovative multi-anchor box strategy. This unit generates 4 new anchor boxes with the predicted anchor box position as the center point, and then obtains the final detection box through scaling as the non-motor vehicle lane contour position. This method can more accurately locate non-motor vehicle targets of different sizes and shapes and improves the adaptability of the system.

[0145] Through the above design, the system of the present invention can comprehensively and accurately analyze the traffic conditions at intersections, timely identify potential conflict risks, and reduce the probability of traffic accidents through an effective early warning mechanism. This system not only meets the current traffic management needs but also has good scalability and adaptability, capable of coping with more complex future traffic scenarios. The tracking module 3 of the present invention adopts an innovative GRU-FST-FCN model for non-motor vehicle trajectory prediction. This model consists of three parts: a GRU network 31, an FST (Fast Spatial Transform) module 32, and an FCN (Fully Convolutional Network) 33, which can effectively capture the temporal and spatial features of non-motor vehicle movement.

[0146] The GRU network 31 is the core component of the model for processing temporal data. It receives the input frames and outputs the corresponding distance information. The update rules of the GRU network are as follows:

[0147] r t =σ(W r ·[h t-1 ,x t ),

[0148] z t =σ(W z ·[h t-1 ,x t ),

[0149]

[0150] Among them, r t is the reset gate, z t is the update gate, h t is the hidden state, x t is the input, W r 、W z and W are weight matrices, and σ is the sigmoid function. In the preferred embodiment of the present invention, the hidden layer size is set to 128, and this value has shown good performance in multiple experiments.

[0151] The FST module 32 is connected to the GRU network 31 and is used for spatial transformation of the features output by the GRU network. The core idea of FST is to learn a transformation matrix θ and then perform an affine transformation on the input feature map. The learning process of the transformation matrix θ is as follows:

[0152] θ=f loc (x),

[0153] Among them, f loc is a small convolutional neural network, and x is the input feature map. In the present invention, f locA two - layer convolution plus one - layer fully - connected structure is adopted. This lightweight design can not only ensure the flexibility of the transformation but also not significantly increase the computational burden.

[0154] FCN (Fully Convolutional Network) 33 is the last component of the model and is used to accurately calculate the output displacement of non - motor vehicles. The main feature of FCN is that it replaces the fully - connected layer in the traditional CNN with a convolutional layer, which enables the network to accept inputs of any size. The forward - propagation process of FCN can be expressed as

[0155] y = f(W * x + b),

[0156] where * represents the convolution operation, W is the convolution kernel, b is the bias term, and f is the activation function. In the present invention, ReLU is used as the activation function, and its mathematical expression is:

[0157] f(x)=max(0, x)

[0158] This choice can effectively alleviate the problem of gradient vanishing and accelerate the convergence process of the network.

[0159] Through the GRU - FST - FCN model, the present invention can accurately predict the movement trajectory of non - motor vehicles and provide reliable data support for subsequent conflict warnings.

[0160] In terms of conflict judgment, the sorting unit 43 of the present invention adopts a multi - level sorting strategy. First, it sorts by distance, and when the distances are equal, it then sorts by angle. This method can more comprehensively evaluate the degree of danger of potential conflicts.

[0161] In addition, the present invention also sets a minimum - distance alarm threshold and a maximum - speed alarm threshold. In a preferred embodiment, the minimum - distance alarm threshold is set to 5 meters, and the maximum - speed alarm threshold is set to 40 km / h. When the distance between a non - motor vehicle and other vehicles is less than 5 meters, or the relative speed is greater than 40 km / h, the system will trigger an alarm. These thresholds are obtained based on a large amount of actual traffic data analysis and can effectively balance the sensitivity and accuracy of the warning.

[0162] When the alarm conditions are met, the system of the present invention will issue two different sound alarms to the driver. One is a red - light signal, indicating a high degree of danger; the other is a yellow - light signal, indicating a medium degree of danger. This hierarchical warning mechanism can help the driver better judge the degree of danger and take appropriate responses.

[0163] Finally, the present invention also proposes a method for non - motor vehicle conflict warning at intersections based on deep learning. The method includes the following steps:

[0164] S1: Pre - process the monitoring video frames at the intersection;

[0165] S2: Detect the position of non-motor vehicle target objects in the video using the YOLO-V3 algorithm. Through deep learning and feature extraction, obtain data including the contour position and size of non-motor vehicles.

[0166] S3: Utilize the data obtained in step S2 to track and obtain the speed data of non-motor vehicles. The speed data includes acceleration and speed.

[0167] S4: According to the real-time position and speed data of non-motor vehicles, combined with the road boundary data, calculate potential conflict points and sort the potential conflict points by conflict level.

[0168] S5: Analyze the potential conflict points, and the analysis results are used for alarm reminder through an acoustic and optical warning device.

[0169] In step S2, the method divides the targets in each frame of the video into three parts: the detection layer, the classification layer, and the regression layer. Use 3*3 convolution for downsampling to extract scale-invariant features, and use 1*1 convolution to extract color information. At the same time, use softmax as the classifier and IOU loss as the detection constraint.

[0170] In step S3, the method uses the GRU-FST-FCN model to learn and predict the movement trajectory of non-motor vehicles. This deep learning model can effectively capture the complex patterns of non-motor vehicle movement and improve the accuracy of prediction.

[0171] In step S4, the method calculates the angle value between the possible driving path and the potential conflict point, calculates the distance between two objects, and conducts a comprehensive ranking in combination with the driving direction, acceleration, and distance information of the vehicle.

[0172] In step S5, the method sets a minimum distance alarm threshold and a maximum speed alarm threshold. When the alarm conditions are met, a sound alarm of two signals, red light and yellow light, is sent to the driver.

[0173] Through this method, the present invention can comprehensively and accurately analyze the traffic conditions at intersections, timely identify potential conflict risks, and reduce the probability of traffic accidents through an effective early warning mechanism. This method not only meets the current traffic management requirements but also has good scalability and adaptability, and can cope with more complex future traffic scenarios.

[0174] It should be noted that the above are only the preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. An intersection non-motor vehicle conflict warning system based on deep learning, characterized in that Including: A video preprocessing module, configured to: Crop the surveillance video frames at the intersection; Perform frame rate conversion and image quality processing on the video data; A detection module, communicatively connected to the video preprocessing module, configured to: Receive the processed video data sent by the video preprocessing module; Based on the processed video data, use the YOLO-V3 algorithm to detect the position of non-motor vehicle target objects in the video; Through deep learning and feature extraction, obtain data including the contour position and size of non-motor vehicles; A tracking module, communicatively connected to the detection module, configured to: Receive the non-motor vehicle contour position and size data sent by the detection module; Based on the non-motor vehicle contour position and size data, track to obtain the speed data of non-motor vehicles, where the speed data includes acceleration and speed; A conflict judgment module, communicatively connected to the tracking module, configured to: Receive the non-motor vehicle speed data sent by the tracking module; According to the real-time position and speed data of the non-motor vehicle, combined with the road boundary data, calculate potential conflict points; Sort the potential conflict points by conflict level; An audible and visual warning module, communicatively connected to the conflict judgment module, configured to: Receive the analysis results of potential conflict points sent by the conflict judgment module; Based on the analysis results of potential conflict points, give an alarm reminder through an audible and visual warning device; The conflict judgment module includes: An angle calculation unit, configured to: Calculate the angle value between the possible driving path and the potential conflict point; The calculation method of the angle value is: angle ; Among them, represents the position coordinates of the non-motor vehicle in the th frame, represents the motion direction vector of the non-motor vehicle, is a preset constant; A distance calculation unit, connected to the angle calculation unit, configured to: Calculate the distance between two objects: Hypotenuse Frame rate; Among them, is the direction angle between two objects, Hypotenuse is the arctangent function, and v is the average velocity of the two objects; A sorting unit, connected to the angle calculation unit and the distance calculation unit, configured to: Sort non-motor vehicles according to the driving direction; Perform a comprehensive ranking by combining the driving direction, acceleration and distance information of the vehicle; Obtain the conflict level by calculating the included angle between directions.

2. The system according to claim 1, wherein The detection module includes: A target detection unit, configured to: Divide the targets in each frame of video into three parts: a detection layer, a classification layer, and a regression layer; Use 3*3 convolution for downsampling to extract scale-invariant features; Use 1*1 convolution to extract color information; A classification unit, connected to the target detection unit, configured to: Use softmax as a classifier; Predict the category to which each target belongs in the classification layer; A regression unit, connected to the target detection unit and the classification unit, configured to: Use IOU loss as a detection constraint; Regress the center point coordinates of multiple targets on the entire image; Predict the values of the width and height of the target box.

3. The system according to claim 1, characterized in that, The tracking module includes: A speed calculation unit, configured to: Calculate the velocity in the x direction: ; Calculate the velocity in the y direction: ; Among them, are respectively the and angles between the non-motor vehicle and the axis at the and and frames, and are the axis coordinates of the non-motor vehicle at the and frames; A motion direction calculation unit, connected to the speed calculation unit, configured to: Calculate the movement direction vector Q of non-motor vehicles, ; A trajectory prediction unit, connected to the motion direction calculation unit, configured to: Calculate the motion direction vector Q of the non-motor vehicle and the road boundary to obtain the possible driving path of the non-motor vehicle; Where the possible driving path of the non-motor vehicle is the extension line of the road boundary line segment.

4. The system according to claim 1, wherein The audible and visual warning module includes: An audible and visual device, configured to: Emit sound and light signals for warning; A controller, connected to the audible and visual device, configured to: Control the start and stop of the audible and visual device; A communicator, connected to the controller, configured to: Transmit the sound and light warning information back to the central platform via Ethernet or WIFI.

5. The system according to claim 2, wherein The target detection unit is also used for: Dividing the entire image into S*S grids; Predicting B anchor boxes for each grid; Reducing the width and height of the entire image to 1 / 4 of the original to obtain more focus points.

6. The system according to claim 2, wherein The regression unit is also used for: Generating 4 new anchor boxes with the predicted anchor box positions as the center points; Obtaining the final detection box through scaling as the contour position of the non-motor vehicle.

7. The system according to claim 3, characterized in that The trajectory prediction unit adopts the GRU-FST-FCN model, including: The GRU network is used for: Receiving the input frame and outputting the corresponding distance; Calculating the difference between the current frame and the previous frame based on the previous frame data; The FST fast spatial transformation module, connected to the GRU network, is used for: Performing spatial transformation on the features output by the GRU network; The FCN fully convolutional network, connected to the FST module, is used for: Calculating the output displacement of the non-motor vehicle; Obtaining the position coordinates of the non-motor vehicle and realizing the motion trajectory prediction of the non-motor vehicle.

8. The system according to claim 1, wherein The sorting unit is also used for: First sorting by distance, and when the distances are equal, sorting by angle; Setting the minimum distance alarm threshold and the maximum speed alarm threshold; When the alarm condition is met, sending two sound alarms to the driver, one for the red light signal and one for the yellow light signal.

9. The deep learning-based non-motor vehicle conflict warning method for intersections based on the system according to any one of claims 1-8, characterized in that, Including the following steps: S1: Preprocessing the monitoring video frames at the intersection; S2: Using the YOLO-V3 algorithm to detect the position of the non-motor vehicle target object in the video, and obtaining the data including the contour position and size of the non-motor vehicle through deep learning and feature extraction; S3: Using the data obtained in step S2 to track and obtain the speed data of the non-motor vehicle, and the speed data includes acceleration and speed; S4: According to the real-time position and speed data of the non-motor vehicle, combined with the road boundary data, calculating the potential conflict points and sorting the potential conflict points by conflict level; S5: Analyzing the potential conflict points, and the analysis results are alarmed and reminded through the sound and light warning device; Among them, step S2 also includes: Dividing the targets in each frame of video into three parts: the detection layer, the classification layer, and the regression layer; Using 3*3 convolution for downsampling to extract scale-invariant features; Using 1*1 convolution to extract color information; Using softmax as the classifier and IOU loss as the detection constraint; Step S3 also includes: Adopting the GRU-FST-FCN model to learn and predict the motion trajectory of the non-motor vehicle; Step S4 also includes: Calculating the angle value between the possible driving path and the potential conflict point; Calculating the distance between two objects; Performing comprehensive ranking by combining the driving direction, acceleration, and distance information of the vehicle; Step S5 also includes: Setting the minimum distance alarm threshold and the maximum speed alarm threshold; When the alarm condition is met, sending a sound alarm of two signals, the red light and the yellow light, to the driver.

Images