Signal-free control road section pedestrian crossing early warning system based on video AI identification and method thereof

Abstract

The invention discloses a non-signal control road section pedestrian crossing early warning system based on video AI identification and a method thereof, and relates to the technical field of traffic management. The system comprises a monitoring subsystem, an early warning subsystem, a power supply subsystem, a control subsystem and a processing subsystem, the method comprises the following steps: acquiring image data under a non-signal control road section in real time by using a pedestrian crossing early warning system, and inputting an image into a processing subsystem for preprocessing; inputting the processed image data into a YOLO model for identification and positioning, and inputting the processed image data into a convolutional neural network for feature extraction; and then judgment is carried out through a random forest model, and pedestrians are identified and pre-warned, so that the pedestrian crossing pre-warning system for the road section without signal control is completed. The method can adapt to different weather and illumination conditions, ensures that pedestrians can still be accurately recognized under the condition of strong light, low light or shielding, effectively improves the pedestrian crossing safety, and has a good self-adaptive early warning mechanism.

Description

Technical Field

[0001] The present invention relates to the technical field of traffic management, and in particular to a pedestrian crossing warning system for a signal-free controlled road section based on video AI recognition and a method thereof. Background Art

[0002] Pedestrian crossing safety is an important issue in urban traffic management. Traditional pedestrian crossing warning systems mainly rely on fixed traffic lights and simple sensors. These systems have many limitations: First, traffic light signals are usually set based on fixed time settings and cannot be adjusted according to real-time traffic conditions, resulting in pedestrians waiting too long during peak hours or failing to cross the street in time when traffic is sparse. Second, although pressure sensors can detect pedestrians passing by, their response mechanism is relatively passive and cannot actively judge pedestrians' intentions and behaviors, resulting in limited warning effects. In addition, traditional visual monitoring technologies are mostly passive monitoring, lack intelligent analysis capabilities, and cannot identify pedestrians' movement patterns and potential dangers in real time. Most systems also rely on manual monitoring, which can easily cause slow responses in complex traffic environments.

[0003] Therefore, traditional pedestrian crossing warning systems mostly rely on manual patrols or simple sensors. These systems have problems such as delayed warning response and low accuracy. With the rapid development of computer vision and artificial intelligence technology, intelligent recognition systems based on video surveillance have become an effective means to improve traffic safety. To this end, the present invention proposes to detect and identify pedestrians crossing the zebra crossing by means of video AI recognition, and control the lighting of the studs on the zebra crossing and the display of the LED screen to achieve the effect of early warning for the driver. At the same time, the warning voice alarm device, flashing lights and pedestrian warning screens are used in parallel to remind and warn pedestrians crossing the street, thereby reducing the possibility of traffic accidents, improving safety, and protecting the lives and property of pedestrians. Summary of the invention

[0004] In order to solve the above problems, the present invention provides a pedestrian crossing warning system and method for non-signal controlled road sections based on video AI recognition;

[0005] To implement the above technical solution, the details are as follows:

[0006] A pedestrian crossing warning system for non-signal controlled road sections based on video AI recognition, comprising: a monitoring subsystem, a warning subsystem, a power supply subsystem, a control subsystem and a processing subsystem;

[0007] The monitoring subsystem includes: a video AI camera installed on the warning column (with the processing subsystem installed inside - integrated image processing chip and AI analysis module);

[0008] The warning subsystem includes: pedestrian warning columns installed on both sides of the crosswalk (with flashing lights and pedestrian warning screens installed on them), warning columns, and luminous warning LED display screens;

[0009] An electronic power module is installed inside the warning column;

[0010] A voice alarm device is installed inside the electronic power module;

[0011] The luminous warning LED display screen and pedestrian warning screen use day and night dual-opening mode signs;

[0012] The power supply subsystem includes: solar panels installed on the pedestrian warning column and electronic power modules installed on the warning column;

[0013] A method for warning pedestrians crossing a street on a non-signal controlled road section based on video AI recognition, comprising the following steps:

[0014] S1, using the pedestrian crossing warning system to collect image data of the non-signal controlled road section in real time and input the image into the processing subsystem;

[0015] The pedestrian crossing warning system includes: monitoring subsystem, warning subsystem, power supply subsystem, control subsystem and processing subsystem;

[0016] The monitoring subsystem includes: an AI analysis module, a video AI camera installed on the warning column, and the processing subsystem installed inside;

[0017] The warning subsystem includes: warning columns, flashing lights, pedestrian warning screens and luminous warning LED display screens;

[0018] The control subsystem is used to control the operation of underground luminous road studs, early warning voice alarm devices, flashing lights, pedestrian warning screens and luminous warning LED display screens;

[0019] The control subsystem is connected to the signal receiving module;

[0020] S2, preprocessing the image data using the integrated image processing chip in the processing subsystem;

[0021] Specifically, the preprocessing includes: image denoising and histogram equalization;

[0022] In addition, the integrated image processing chip in the processing subsystem can also store and process the weight update or model optimization information of the YOLO model. Over time, the warning system can adjust the parameters of the YOLO model based on the actual environmental data to make it more adaptable to the traffic pattern of a specific section of road, thereby improving the accuracy and efficiency of the system. The specific process is as follows:

[0023] S2.1. Use an integrated image processing chip to record pedestrian behavior and surrounding environment information in the video, store and process the data, and provide it to the model for subsequent optimization;

[0024] The data includes: pedestrian flow information, weather change information and time period information;

[0025] S2.2. Evaluate the performance of the early warning system monthly by calculating the detection precision, recall and F1 score, focusing on the performance under different conditions to identify false positives and false negatives;

[0026] Different conditions include: time conditions, weather conditions, environmental conditions, and pedestrian characteristics;

[0027] Furthermore, the time conditions include: peak hours, off-peak hours, and day-night changes;

[0028] Peak hours, when traffic and pedestrians are dense, may increase false alarms and missed alarms;

[0029] During non-peak hours, the traffic volume is low and the detection environment is simpler;

[0030] Diurnal changes, i.e. differences in illumination, affect detection accuracy, and additional enhancement techniques may be required at night;

[0031] Furthermore, weather conditions include: sunny, rainy / snowy, foggy;

[0032] On sunny days, when there is sufficient sunlight, the system usually performs better, but it is necessary to prevent overexposure and shadow effects;

[0033] Rainy or snowy days will affect image clarity and increase the risk of missed reports;

[0034] Haze means low visibility and blurred images, which affects recognition effects;

[0035] Furthermore, environmental conditions include: occlusion, crowded environment, and complex background;

[0036] Occlusion, when a pedestrian is blocked by an object, may result in missed reports;

[0037] Crowded environments, where multiple people are active at the same time, may lead to misidentification or missed detection;

[0038] Complex background means that the background is complicated and prone to false alarms;

[0039] Furthermore, pedestrian features include: posture, occlusion, clothing, and size;

[0040] Posture, that is, when pedestrians move quickly or have their backs to the camera, they are prone to missed reports;

[0041] Occlusion, i.e. partial or complete obstruction, increases the risk of false negatives;

[0042] Clothing and size, i.e., pedestrians at a distance or clothing similar to the background, are easily missed;

[0043] For example, accuracy issues during rush hour or in adverse weather conditions;

[0044] S2.3. Use data analysis tools (Pandas and Seaborn in Python) to visualize video data, analyze pedestrian behavior patterns, and identify key factors that affect pedestrian crossing safety; for example, pedestrian crossing rate and vehicle speed changes during specific time periods;

[0045] Further, key factors include:

[0046] Pedestrian passing rate: the number of pedestrians passing per unit time (such as the number of pedestrians per hour), which increases significantly during peak hours;

[0047] Vehicle speed: 0-30km / h (low speed), 30-60km / h (medium speed), above 60km / h (high speed);

[0048] Weather conditions: sunny, rainy, snowy, foggy, etc., which affect pedestrian behavior and traffic conditions;

[0049] Specific time periods include: Peak hours: 7:00-9:00AM, 5:00-7:00PM and off-peak hours: 9:00AM-4:00PM, 7:00PM-7:00AM;

[0050] S2.4. According to the analysis results, adjust the non-maximum suppression (NMS) threshold of the YOLO model to reduce missed detections, increase the probability of pedestrians being accurately identified, and reduce false positives;

[0051] Specifically, the threshold of the YOLO model was reduced from 0.5 to 0.4;

[0052] S2.5. Using a deep learning framework (PyTorch), we collected newly collected video data to retrain the YOLO model and used gradient descent to update weights, with a special focus on pedestrian behavior in bad weather and at night.

[0053] The gradient descent formula is as follows:

[0054]

[0055] In the formula, w new is the updated parameter value (e.g. the weight of a layer of the YOLO model); w old is the current parameter value; η is the learning rate (taken as 0.001); the learning rate controls the amplitude of parameter adjustment at each update; too large a learning rate will lead to oscillation, and too small a learning rate will converge too slowly; is the gradient, that is, the partial derivative of the loss function with respect to the parameters, which indicates the impact of the current parameters on the loss and the direction and magnitude of the parameter adjustment;

[0056] Furthermore, for regression problems, the commonly used loss function is the mean square error (MSE), which is expressed as follows:

[0057]

[0058] In the formula, y pred is the predicted value, y true is the actual value, N is the number of samples;

[0059] Furthermore, the loss function is derived and the gradient of each parameter is calculated, that is, the gradient calculation is performed; the expression is as follows:

[0060]

[0061]

[0062] In the formula, x i is the i-th parameter;

[0063] Finally, the expression for weight update is as follows:

[0064]

[0065]

[0066] S2.6. Deploy the retrained model, record real-time detection results and performance indicators, analyze new data to improve the model, and adapt the system to changes in traffic patterns through feedback mechanisms;

[0067] Specifically, the deployment process is as follows:

[0068] 1. Model export and optimization: Export the retrained YOLO model to a format suitable for edge devices (using PyTorch training model, export format is ONNX); use quantization and pruning technology to optimize the model and reduce the computational burden;

[0069] 2. Model loading and inference: Deploy the retrained YOLO model to the processing subsystem in the video AI camera for local video stream processing and real-time object detection;

[0070] 3. Data transmission and feedback: The detection results are transmitted to the back-end system (early warning subsystem) through the network, and the warning is triggered or the traffic lights are controlled according to the detection output;

[0071] 4. Continuous optimization and updating: monitor the model’s reasoning performance (accuracy, latency) to ensure stable system operation;

[0072] Specifically, the process of analyzing new data is as follows:

[0073] Evaluation Metrics:

[0074] Precision: The proportion of samples predicted by the model as positive examples (pedestrians) that are actually positive examples; the expression is as follows:

[0075]

[0076] In the formula, TP is the correctly identified pedestrian, and FP is the misidentified pedestrian;

[0077] Recall: The proportion of all actual pedestrians that the model can correctly identify; the expression is as follows:

[0078]

[0079] Where FN is the pedestrian that was missed;

[0080] F1 score: The harmonic mean of precision and recall. It takes into account both accuracy and completeness; the expression is as follows:

[0081]

[0082] The target F1 score is usually set to ≥ 0.9 to ensure detection quality;

[0083] Furthermore, the standard thresholds for the evaluation indicators are as follows:

[0084] Accuracy: ≥95% (the goal is to minimize false positives in most cases)

[0085] Recall rate: ≥90% (the goal is to minimize missed calls and ensure pedestrian safety)

[0086] F1 score: ≥0.9 (taking into account accuracy and completeness)

[0087] Furthermore, the detection delay of the early warning system: ≤100ms (real-time requirement, especially in the scene of pedestrians crossing the street);

[0088] Performance of the early warning system under specific conditions: Under adverse weather conditions (such as rain and fog), if the accuracy drops by more than 10%, optimization is required;

[0089] Nighttime conditions of the early warning system: If the accuracy at night is more than 5% lower than that during the day, it is necessary to collect nighttime data and retrain;

[0090] Early warning system during peak hours: During peak traffic hours, the false alarm rate increases by more than 5%, and the model recognition performance should be optimized;

[0091] Specific methods to improve the model include:

[0092] 1. Data enhancement; that is, increasing the diversity of training data: Increase data diversity and improve the generalization ability of the model through the following methods:

[0093] Rotate, zoom, and crop: simulate different shooting angles and distances;

[0094] Weather simulation: synthesize pedestrian images under different lighting conditions and in rainy and foggy weather conditions to improve the model’s adaptability in different weather conditions;

[0095] Low-light enhancement: simulate nighttime or low-light environments to improve the model's nighttime recognition capabilities;

[0096] Synthetic data: When insufficient data can be collected in certain scenarios (e.g., at night, in bad weather), training samples are generated through image synthesis technology;

[0097] 2. Improve the model structure; the details are as follows:

[0098] Lightweight model, that is, reducing the model's computational workload and memory usage through pruning and quantization methods, thereby improving inference speed;

[0099] Pruning is to reduce the complexity of the model by removing connections with small weights or unimportant connections;

[0100] Quantization is to convert the floating-point parameters in the model into lower precision (such as 8-bit integers) to reduce the amount of calculation;

[0101] S2.7. Regularly conduct a comprehensive check of the system and update model weights and parameter settings;

[0102] Specifically, the system will be comprehensively checked regularly (every quarter), and model weights and parameter settings will be updated to keep the system running efficiently under different conditions to cope with changing traffic conditions;

[0103] S3, input the processed image data into the YOLO model for recognition and positioning;

[0104] Specifically, the YOLO model detects target pedestrians by dividing the input image into n×n grids. If the center of a target pedestrian falls in a certain grid, then this grid is responsible for detecting the target pedestrian; each grid can predict B range boxes and the confidence of a certain category represented by these range boxes; the confidence represents which categories are included in this range box and the accuracy of the prediction of these categories respectively;

[0105] Furthermore, the confidence is defined as the product of the probability of detecting the target and the intersection over union (IOU), and the expression is as follows:

[0106]

[0107] In the formula, P(Object) represents whether there is a pedestrian crossing the street in a certain grid. If so, the value is 1, otherwise it is 0;

[0108] Intersection Over Union (IOU) is the overlap ratio between the range box generated by the model and the marked box, that is, the intersection of the detection result (DetectionResult) and the marked ground truth (GroundTruth) over their union. The expression is as follows:

[0109]

[0110] If there is no pedestrian to be detected in a grid, its confidence score is 0, otherwise it is the IOU between the detection result and the true range;

[0111] Each scope box consists of five parameters, namely (x, y, w, h, confidence), where x and y represent the horizontal and vertical coordinates of the scope box relative to the normalized center point of the grid; w and h represent the width and height of the scope box respectively, and confidence represents the confidence;

[0112] Each grid also predicts the probability of C target categories, that is, the probability of whether each grid contains the target pedestrian; the target pedestrian category probability is multiplied by the confidence level, and the expression is as follows:

[0113]

[0114] The product is taken as the score for each confidence level;

[0115] Since the image is divided into n×n grids, and B range boxes are predicted in each grid, each range box has 5 parameters. Assuming that the dataset has a total of C target categories, the final prediction result must contain S×S (B×5+C) parameters;

[0116] In YOLO, each grid predicts multiple range boxes, but in the training of the network model, each detection target pedestrian is best predicted by only one range box; therefore, when selecting a range box, determine which prediction result has the largest IOU with the real target pedestrian range, and then use this range box to predict this target; this will make each object have a specific range box to detect; as training progresses, each range box will get better and better at predicting the height, BMI, and category of a specific pedestrian;

[0117] Furthermore, when the pedestrian in the image is at the grid boundary, multiple grids may be located. In this case, non-maximal suppression (Nms) is used to remove the repeated detection of target pedestrians.

[0118] In order to avoid repeated detection of the same target pedestrian by multiple boxes, non-maximum suppression is needed to select the best target box;

[0119] The operation process of non-maximum suppression is as follows: sort all generated candidate boxes according to confidence, and take the box with the highest confidence as the starting box; start from the box with the highest confidence and traverse the remaining boxes one by one; for the currently traversed box, if the overlap with the starting box is greater than the preset non-maximum suppression threshold (the default value of 0.5 is taken at the beginning, which can strike a balance between detection accuracy and redundant results), the box is discarded, otherwise it is retained; continue to traverse the next box until all boxes are processed; through this process, non-maximum suppression will filter out the target box with the highest confidence and non-overlapping with the starting box, thereby generating the final detection result; the threshold of non-maximum suppression is an important parameter, which is used to control the degree to which overlapping boxes are discarded. A higher threshold will more strictly remove overlapping boxes, which may result in missed detection (i.e., ignoring some real targets); a lower threshold will retain more overlapping boxes, which may result in redundant detection results; and by suppressing redundant detection results, non-maximum suppression can improve the accuracy and efficiency of the target pedestrian detection algorithm;

[0120] Adjustment range: 0.4-0.6: After preliminary testing, if the missed detection rate is found to be high, you can try to reduce it to 0.4; if there are many redundant frames in the detection results, you can increase it to 0.6.

[0121] Final value: 0.5

[0122] This value has shown good detection results in many experiments, especially in scenes with dense pedestrians, which helps reduce false alarms;

[0123] Specific value basis:

[0124] In actual tests, it was observed that a non-maximum suppression threshold of 0.5 can maintain high precision and recall under various weather and traffic conditions. If missed detection occurs under specific circumstances (such as peak hours or bad weather), fine-tuning to 0.4 can be considered, but the overall performance changes need to be monitored.

[0125] S4, inputting the recognized and located image data into a convolutional neural network for feature extraction;

[0126] Specifically, a convolutional neural network (CNN) based on darknet-53 is used to obtain multi-scale image features through convolution upsampling and combine multi-scale features to achieve pedestrian recognition and monitoring pedestrian data information tasks;

[0127] The convolutional neural network (CNN) based on darknet-53 has a processing speed of 78 images / s, which is slower than Darknet-19, but 1 times faster than ResNet-152 with the same accuracy. Therefore, Darknet-53 is a feature extraction network architecture that takes into account both speed and accuracy.

[0128] Feature extraction includes: pedestrian’s position, speed, direction of travel, and distance from the vehicle;

[0129] S5, input the extracted features into the random forest model for judgment, and identify pedestrians based on the judgment results to complete the pedestrian crossing warning system for the unsignalized road section;

[0130] Specifically, the extracted features are used to train the random forest model, and the judgment results of multiple features and multiple decision trees are combined to make a more accurate quantitative prediction of the pedestrian's next intention;

[0131] Improve the reliability of the system through voting mechanism (in classification tasks) or averaging (in regression tasks);

[0132] When it is identified that the pedestrian has no intention to cross the street, no action is taken to ensure normal traffic flow;

[0133] When a pedestrian is identified to cross the street, the pedestrian crossing signal is transmitted to the control subsystem through the signal receiving module to activate the early warning subsystem;

[0134] The warning subsystem is activated by sending warning signals to drivers and pedestrians. Drivers take measures to slow down or stop after receiving the warning signals, and pedestrians pay attention and pass after receiving the warning signals.

[0135] Sending warning signals to drivers includes: raising the warning column and emitting warning LED display;

[0136] Sending warning signals to pedestrians includes: warning voice broadcast device and pedestrian warning screen lighting up, strobe lights flashing;

[0137] The distance D before the luminous warning LED display screen should be arranged on the crosswalk should meet the formula:

[0138] D≥R+J+L-S

[0139] Where D is the distance before the warning sign before the lane stop line of the pedestrian crossing, R is the reading distance of the warning sign; J is the reaction distance; L is the action distance of the vehicle changing lanes; S is the visual distance of the warning sign (m);

[0140] When the driver is driving, the calculation formula for the reading distance R of the warning sign is as follows:

[0141]

[0142] Where V0 is the vehicle speed when reading the warning sign (km / h), which is 100km / h; t1 is the driver's visual recognition time (s);

[0143] The reaction distance J includes: the distance J1 that the vehicle on the inner lane waits for to enter the gap and the driving distance J2 that the driver determines the time to enter the gap, as shown in the following formula:

[0144]

[0145] Where V is the design speed (km / h), which is 120km / h; t w is the average waiting time for entering the gap; v h is the minimum lane-changing speed, which is 0.76 times V (km / h); △v is the relative speed of the adjacent lanes (km / h); t is the critical time (s), which is 4s; λ is the average arrival rate (vehicles / s), which is 0.411 vehicles / s; τ is the minimum headway (s), which is 1.2s; t2 is the vehicle lateral movement time, which is 3.75 (s);

[0146] The lane change adjustment speed distance L1, the lane change lateral distance L2 and the deceleration distance L3 constitute the vehicle lane change action distance L, as shown in the following formula:

[0147]

[0148] Where, t1 is the time required to adjust the speed during lane change, which is 2.5s; t2 is the time required for lane change and lateral movement, which is 3.75s; V0 is 100km / h; V t is the speed limit within the road section, which is 80km / h; φ is the road adhesion coefficient, which is 0.4 for asphalt road surface;

[0149] The visual recognition distance S of the luminous warning LED display screen is calculated using a common model of the relationship between character height and symbol recognition distance. The expression is as follows:

[0150] S=-272.33h 2 +408.31h-25.72+ε

[0151] In the formula, S is the visual distance of the warning sign (m); h is the height of the Chinese character (m); ε is the correction coefficient of the complexity and simplicity of the Chinese character strokes and font structure, which is 20;

[0152] Therefore, the layout location of the luminous warning LED display screen should be no less than 424~710m away from the lane stop line of the pedestrian crossing;

[0153] Beneficial effects of the present invention

[0154] The present invention combines the video AI camera with the deep learning model based on YOLO, namely the convolutional neural network (CNN), and monitors pedestrian behavior information based on the darknet-53 convolutional network. With the help of the hardware acceleration capability of the embedded integrated image processing chip, the system can complete target detection and recognition in milliseconds, ensuring real-time response under complex road conditions, meeting the needs of actual road traffic, ensuring pedestrian safety, and alleviating traffic safety problems caused by conflicts between people and vehicles.

[0155] The video AI camera designed by the present invention is combined with the machine learning model - random forest (RF) model, which can efficiently make predictions and decisions, and is particularly suitable for traffic safety systems that require rapid response. Through parallel processing and effective feature selection, rapid early warning and response can be achieved while ensuring high accuracy, thereby improving the real-time performance of the entire system. Compared with other complex deep learning models, it has lower training costs and deployment complexity. They can usually run with lower computing resources and do not require frequent parameter adjustments. This makes the deployment and operating costs of the early warning system relatively low, and easy to maintain and update in practical applications.

[0156] The video AI camera designed by the present invention is based on the YOLO model and is equipped with an embedded integrated image processing chip. The system can adapt to different weather and lighting conditions. Through the image enhancement function of the image processing chip and the robustness of the YOLO model, it ensures that pedestrians can still be accurately identified in strong light, low light or obstructed conditions, effectively improving the safety of pedestrians crossing the street and having a good adaptive early warning mechanism.

[0157] The design of the present invention focuses on low power consumption, especially in embedded devices. The integrated image processing chip can efficiently run the deep learning model and maintain low energy consumption, which enables the system to operate stably and for a long time outdoors, reducing maintenance costs. The system can also be optimized through model updates and data feedback. As time goes by and the environment changes, the system can gradually adapt to the characteristics of different road sections, improve detection accuracy and response speed, and make traffic management more intelligent. BRIEF DESCRIPTION OF THE DRAWINGS

[0158] Figure 1 It is a system schematic diagram of the present invention;

[0159] Figure 2 This is a schematic diagram of the structure of the luminous warning LED screen of the present invention;

[0160] Figure 3 It is a schematic diagram of the structure of the underground luminous road stud of the present invention;

[0161] Figure 4 is a flow chart of the method of the present invention;

[0162] Figure 5 Schematic diagram of the YOLO detection process of the present invention;

[0163] Figure 6 It is a workflow diagram of the non-maximum suppression technology of the present invention;

[0164] Figure 7 Schematic diagram of the driver's visual recognition process. DETAILED DESCRIPTION

[0165] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0166] like Figure 1 As shown, a pedestrian crossing warning system for signal-free controlled sections based on video AI recognition includes: a monitoring subsystem, a warning subsystem, a power supply subsystem, a control subsystem and a processing subsystem;

[0167] The monitoring subsystem includes: a video AI camera installed on the warning column (with the processing subsystem installed inside - integrated image processing chip and AI analysis module);

[0168] The warning subsystem includes: pedestrian warning columns installed on both sides of the crosswalk (with flashing lights and pedestrian warning screens installed on them), warning columns, such as Figure 3As shown, and the luminous warning LED display, such as Figure 2 As shown;

[0169] An electronic power module is installed inside the warning column;

[0170] A voice alarm device is installed inside the electronic power module;

[0171] The luminous warning LED display screen and pedestrian warning screen use day and night dual-opening mode signs;

[0172] The power supply subsystem includes: solar panels installed on the pedestrian warning column and electronic power modules installed on the warning column;

[0173] like Figure 4 As shown, a pedestrian crossing warning method for a non-signal controlled road section based on video AI recognition includes the following steps:

[0174] S1, using the pedestrian crossing warning system to collect image data of the non-signal controlled road section in real time and input the image into the processing subsystem;

[0175] The pedestrian crossing warning system includes: monitoring subsystem, warning subsystem, power supply subsystem, control subsystem and processing subsystem;

[0176] The monitoring subsystem includes: an AI analysis module, a video AI camera installed on the warning column, and the processing subsystem installed inside;

[0177] The warning subsystem includes: warning columns, flashing lights, pedestrian warning screens and luminous warning LED display screens;

[0178] The control subsystem is used to control the operation of the underground luminous road studs, warning voice alarm device, flashing lights, pedestrian warning screen and luminous warning LED display screen;

[0179] The control subsystem is connected to the signal receiving module;

[0180] S2, preprocessing the image data using the integrated image processing chip in the processing subsystem;

[0181] Specifically, the preprocessing includes: image denoising and histogram equalization;

[0182] In addition, the integrated image processing chip in the processing subsystem can also store and process the weight update or model optimization information of the YOLO model. Over time, the warning system can adjust the parameters of the YOLO model based on the actual environmental data to make it more adaptable to the traffic pattern of a specific section of road, thereby improving the accuracy and efficiency of the system. The specific process is as follows:

[0183] S2.1. Use an integrated image processing chip to record pedestrian behavior and surrounding environment information in the video, store and process the data, and provide it to the model for subsequent optimization;

[0184] The data includes: pedestrian flow information, weather change information and time period information;

[0185] S2.2. Evaluate the performance of the early warning system monthly by calculating the detection precision, recall and F1 score, focusing on the performance under different conditions to identify false positives and false negatives;

[0186] Different conditions include: time conditions, weather conditions, environmental conditions, and pedestrian characteristics;

[0187] Furthermore, the time conditions include: peak hours, off-peak hours, and day-night changes;

[0188] Peak hours, when traffic and pedestrians are dense, may increase false alarms and missed alarms;

[0189] During non-peak hours, the traffic volume is low and the detection environment is simpler;

[0190] Diurnal changes, i.e. differences in illumination, affect detection accuracy, and additional enhancement techniques may be required at night;

[0191] Furthermore, weather conditions include: sunny, rainy / snowy, foggy;

[0192] On sunny days, when there is sufficient sunlight, the system usually performs better, but it is necessary to prevent overexposure and shadow effects;

[0193] Rainy or snowy days will affect image clarity and increase the risk of missed reports;

[0194] Haze means low visibility and blurred images, which affects recognition effects;

[0195] Furthermore, environmental conditions include: occlusion, crowded environment, and complex background;

[0196] Occlusion, when a pedestrian is blocked by an object, may result in missed reports;

[0197] Crowded environments, where multiple people are active at the same time, may lead to misidentification or missed detection;

[0198] Complex background means that the background is complicated and prone to false alarms;

[0199] Furthermore, pedestrian features include: posture, occlusion, clothing, and size;

[0200] Posture, that is, when pedestrians move quickly or have their backs to the camera, they are prone to being missed.

[0201] Occlusion, i.e. partial or complete obstruction, increases the risk of false negatives;

[0202] Clothing and size, i.e., pedestrians at a distance or clothing similar to the background, are easily missed;

[0203] For example, accuracy issues during rush hour or in adverse weather conditions;

[0204] S2.3. Use data analysis tools (Pandas and Seaborn in Python) to visualize video data, analyze pedestrian behavior patterns, and identify key factors that affect pedestrian crossing safety; for example, pedestrian crossing rate and vehicle speed changes during specific time periods;

[0205] Further, key factors include:

[0206] Pedestrian passing rate: the number of pedestrians passing per unit time (such as the number of pedestrians per hour), which increases significantly during peak hours;

[0207] Vehicle speed: 0-30km / h (low speed), 30-60km / h (medium speed), above 60km / h (high speed);

[0208] Weather conditions: sunny, rainy, snowy, foggy, etc., which affect pedestrian behavior and traffic conditions;

[0209] Specific time periods include: Peak hours: 7:00-9:00AM, 5:00-7:00PM and off-peak hours: 9:00AM-4:00PM, 7:00PM-7:00AM;

[0210] S2.4. According to the analysis results, adjust the non-maximum suppression (NMS) threshold of the YOLO model to reduce missed detections, increase the probability of pedestrians being accurately identified, and reduce false positives;

[0211] Specifically, the threshold of the YOLO model was reduced from 0.5 to 0.4;

[0212] S2.5. Using a deep learning framework (PyTorch), we collected newly collected video data to retrain the YOLO model and used gradient descent to update weights, with a special focus on pedestrian behavior in bad weather and at night.

[0213] The gradient descent formula is as follows:

[0214]

[0215] In the formula, w new is the updated parameter value (e.g. the weight of a layer of the YOLO model); w old is the current parameter value; η is the learning rate (taken as 0.001); the learning rate controls the amplitude of parameter adjustment at each update; too large a learning rate will lead to oscillation, and too small a learning rate will converge too slowly; is the gradient, that is, the partial derivative of the loss function with respect to the parameters, which indicates the impact of the current parameters on the loss and the direction and magnitude of the parameter adjustment;

[0216] Furthermore, for regression problems, the commonly used loss function is the mean square error (MSE), which is expressed as follows:

[0217]

[0218] In the formula, y pred is the predicted value, y true is the actual value, N is the number of samples;

[0219] Furthermore, the loss function is derived and the gradient of each parameter is calculated, that is, the gradient calculation is performed; the expression is as follows:

[0220]

[0221]

[0222] In the formula, x i is the i-th parameter;

[0223] Finally, the expression for weight update is as follows:

[0224]

[0225]

[0226] S2.6. Deploy the retrained model, record real-time detection results and performance indicators, analyze new data to improve the model, and adapt the system to changes in traffic patterns through feedback mechanisms;

[0227] Specifically, the deployment process is as follows:

[0228] 1. Model export and optimization: Export the retrained YOLO model to a format suitable for edge devices (using PyTorch training model, export format is ONNX); use quantization and pruning technology to optimize the model and reduce the computational burden;

[0229] 2. Model loading and inference: Deploy the retrained YOLO model to the processing subsystem in the video AI camera for local video stream processing and real-time object detection;

[0230] 3. Data transmission and feedback: The detection results are transmitted to the back-end system (early warning subsystem) through the network, and the warning is triggered or the traffic lights are controlled according to the detection output;

[0231] 4. Continuous optimization and updating: monitor the model’s reasoning performance (accuracy, latency) to ensure stable system operation;

[0232] Specifically, the process of analyzing new data is as follows:

[0233] Evaluation Metrics:

[0234] Precision: The proportion of samples predicted by the model as positive examples (pedestrians) that are actually positive examples; the expression is as follows:

[0235]

[0236] In the formula, TP is the correctly identified pedestrian, and FP is the misidentified pedestrian;

[0237] Recall: The proportion of all actual pedestrians that the model can correctly identify; the expression is as follows:

[0238]

[0239] Where FN is the pedestrian that was missed;

[0240] F1 score: The harmonic mean of precision and recall. It takes into account both accuracy and completeness; the expression is as follows:

[0241]

[0242] The target F1 score is usually set to ≥ 0.9 to ensure detection quality;

[0243] Furthermore, the standard thresholds for the evaluation indicators are as follows:

[0244] Accuracy: ≥95% (the goal is to minimize false positives in most cases)

[0245] Recall rate: ≥90% (the goal is to minimize missed calls and ensure pedestrian safety)

[0246] F1 score: ≥0.9 (taking into account accuracy and completeness)

[0247] Furthermore, the detection delay of the early warning system: ≤100ms (real-time requirement, especially in the scene of pedestrians crossing the street);

[0248] Performance of the early warning system under specific conditions: Under adverse weather conditions (such as rain and fog), if the accuracy drops by more than 10%, optimization is required;

[0249] Nighttime conditions of the early warning system: If the accuracy at night is more than 5% lower than that during the day, it is necessary to collect nighttime data and retrain;

[0250] Early warning system during peak hours: During peak traffic hours, the false alarm rate increases by more than 5%, and the model recognition performance should be optimized;

[0251] Specific methods to improve the model include:

[0252] 1. Data enhancement; that is, increasing the diversity of training data: Increase data diversity and improve the generalization ability of the model through the following methods:

[0253] Rotate, zoom, and crop: simulate different shooting angles and distances;

[0254] Weather simulation: synthesize pedestrian images under different lighting conditions and in rainy and foggy weather conditions to improve the model’s adaptability in different weather conditions;

[0255] Low-light enhancement: simulate nighttime or low-light environments to improve the model's nighttime recognition capabilities;

[0256] Synthetic data: When insufficient data can be collected in certain scenarios (e.g., at night, in bad weather), training samples are generated through image synthesis technology;

[0257] 2. Improve the model structure; the details are as follows:

[0258] Lightweight model, that is, reducing the model's computational workload and memory usage through pruning and quantization methods, thereby improving inference speed;

[0259] Pruning is to reduce the complexity of the model by removing connections with small weights or unimportant connections;

[0260] Quantization is to convert the floating-point parameters in the model into lower precision (such as 8-bit integers) to reduce the amount of calculation;

[0261] S2.7. Regularly conduct a comprehensive check of the system and update model weights and parameter settings;

[0262] Specifically, the system will be comprehensively checked regularly (every quarter), and model weights and parameter settings will be updated to keep the system running efficiently under different conditions to cope with changing traffic conditions;

[0263] S3, input the processed image data into the YOLO model for recognition and positioning;

[0264] Specifically, Figure 5 , Figure 6 As shown in the figure, the YOLO model detects target pedestrians by dividing the input image into n×n grids. If the center of a target pedestrian falls in a certain grid, then this grid is responsible for detecting the target pedestrian; each grid can predict B range boxes and the confidence of a certain category represented by these range boxes; the confidence represents which categories are included in this range box and the accuracy of the prediction of these categories respectively;

[0265] Furthermore, the confidence is defined as the product of the probability of detecting the target and the intersection over union (IOU), and the expression is as follows:

[0266]

[0267] In the formula, P(Object) represents whether there is a pedestrian crossing the street in a certain grid. If so, the value is 1, otherwise it is 0;

[0268] Intersection Over Union (IOU) is the overlap ratio between the range box generated by the model and the marked box, that is, the intersection of the detection result (DetectionResult) and the marked ground truth (GroundTruth) over their union. The expression is as follows:

[0269]

[0270] If there is no pedestrian to be detected in a grid, its confidence score is 0, otherwise it is the IOU between the detection result and the true range;

[0271] Each scope box consists of five parameters, namely (x, y, w, h, confidence), where x and y represent the horizontal and vertical coordinates of the scope box relative to the normalized center point of the grid; w and h represent the width and height of the scope box respectively, and confidence represents the confidence;

[0272] Each grid also predicts the probability of C target categories, that is, the probability of whether each grid contains the target pedestrian; the target pedestrian category probability is multiplied by the confidence level, and the expression is as follows:

[0273]

[0274] The product is taken as the score for each confidence level;

[0275] Since the image is divided into n×n grids, and B range boxes are predicted in each grid, each range box has 5 parameters. Assuming that the dataset has a total of C target categories, the final prediction result must contain S×S (B×5+C) parameters;

[0276] In YOLO, each grid predicts multiple range boxes, but in the training of the network model, each detection target pedestrian is best predicted by only one range box; therefore, when selecting a range box, determine which prediction result has the largest IOU with the real target pedestrian range, and then use this range box to predict this target; this will make each object have a specific range box to detect; as training progresses, each range box will get better and better at predicting the height, BMI, and category of a specific pedestrian;

[0277] Furthermore, when the pedestrian in the image is at the grid boundary, multiple grids may be located. In this case, non-maximal suppression (Nms) is used to remove the repeated detection of target pedestrians.

[0278] In order to avoid repeated detection of the same target pedestrian by multiple boxes, non-maximum suppression is needed to select the best target box;

[0279] The operation process of non-maximum suppression is as follows: sort all generated candidate boxes according to confidence, and take the box with the highest confidence as the starting box; start from the box with the highest confidence and traverse the remaining boxes one by one; for the currently traversed box, if the overlap with the starting box is greater than the preset non-maximum suppression threshold (the default value of 0.5 is taken at the beginning, which can strike a balance between detection accuracy and redundant results), the box is discarded, otherwise it is retained; continue to traverse the next box until all boxes are processed; through this process, non-maximum suppression will filter out the target box with the highest confidence and non-overlapping with the starting box, thereby generating the final detection result; the threshold of non-maximum suppression is an important parameter, which is used to control the degree to which overlapping boxes are discarded. A higher threshold will more strictly remove overlapping boxes, which may result in missed detection (i.e., ignoring some real targets); a lower threshold will retain more overlapping boxes, which may result in redundant detection results; and by suppressing redundant detection results, non-maximum suppression can improve the accuracy and efficiency of the target pedestrian detection algorithm;

[0280] Adjustment range: 0.4-0.6: After preliminary testing, if the missed detection rate is found to be high, you can try to reduce it to 0.4; if there are many redundant frames in the detection results, you can increase it to 0.6.

[0281] Final value: 0.5

[0282] This value has shown good detection results in many experiments, especially in scenes with dense pedestrians, which helps reduce false alarms;

[0283] Specific value basis:

[0284] In actual tests, it was observed that a non-maximum suppression threshold of 0.5 can maintain high precision and recall under various weather and traffic conditions. If missed detection occurs under specific circumstances (such as peak hours or bad weather), fine-tuning to 0.4 can be considered, but the overall performance changes need to be monitored.

[0285] S4, inputting the recognized and located image data into a convolutional neural network for feature extraction;

[0286] Specifically, a convolutional neural network (CNN) based on darknet-53 is used to obtain multi-scale image features through convolution upsampling and combine multi-scale features to achieve pedestrian recognition and monitoring pedestrian data information tasks;

[0287] The convolutional neural network (CNN) based on darknet-53 has a processing speed of 78 images / s, which is slower than Darknet-19, but 1 times faster than ResNet-152 with the same accuracy. Therefore, Darknet-53 is a feature extraction network architecture that takes into account both speed and accuracy.

[0288] Feature extraction includes: pedestrian’s position, speed, direction of travel, and distance from the vehicle;

[0289] S5, input the extracted features into the random forest model for judgment, and identify pedestrians based on the judgment results to complete the pedestrian crossing warning system for the unsignalized road section;

[0290] Specifically, the extracted features are used to train the random forest model, and the judgment results of multiple features and multiple decision trees are combined to make a more accurate quantitative prediction of the pedestrian's next intention;

[0291] Furthermore, each decision tree outputs a judgment result based on the input features (the pedestrian's position, speed, direction of travel, and distance from the vehicle); the judgment result is a classification of the pedestrian's behavior, such as "prepare to cross the street", "continue to stay", and "move quickly";

[0292] The random forest model consists of multiple decision trees, each of which is trained independently to make predictions about pedestrian intentions; specific methods include: voting mechanism and weighted voting;

[0293] The voting mechanism means that each decision tree independently predicts the pedestrian's next move, and the final prediction result is based on the majority decision. For example, if 70 out of 100 trees predict that the pedestrian is "ready to cross the street", the final output of the random forest is "ready to cross the street". This method reduces the bias of a single tree and improves the robustness of the overall model.

[0294] Weighted voting is to assign different weights to each tree according to its accuracy and comprehensively calculate the prediction results. For example, trees with high accuracy are given higher weights when voting, which can further improve the prediction accuracy of the model.

[0295] The random forest algorithm is used to evaluate feature importance and prevent overfitting, simplifying the model and improving efficiency;

[0296] Specifically, random forests evaluate the importance of each feature by calculating its contribution in the decision tree split. Importance can be expressed by indicators such as Gini impurity or information gain; random forests use randomly selected samples and features for training, and the training data for each tree is a subset randomly selected from the overall data set;

[0297] Improve the reliability of the system through voting mechanism (in classification tasks) or averaging (in regression tasks);

[0298] When it is identified that the pedestrian has no intention to cross the street, no action is taken to ensure normal traffic flow;

[0299] When a pedestrian is identified to cross the street, the pedestrian crossing signal is transmitted to the control subsystem through the signal receiving module to activate the early warning subsystem;

[0300] The warning subsystem is activated by sending warning signals to drivers and pedestrians. Drivers take measures to slow down or stop after receiving the warning signals, and pedestrians pay attention and pass after receiving the warning signals.

[0301] Sending warning signals to drivers includes: raising the warning column and emitting warning LED display;

[0302] like Figure 7 As shown, sending warning signals to pedestrians includes: the warning voice broadcast device and the pedestrian warning screen light up, and the strobe light flashes;

[0303] The distance D before the luminous warning LED display screen should be arranged on the crosswalk should meet the formula:

[0304] D≥R+J+L-S

[0305] Where D is the distance before the warning sign before the lane stop line of the pedestrian crossing, R is the reading distance of the warning sign; J is the reaction distance; L is the action distance of the vehicle changing lanes; S is the visual distance of the warning sign (m);

[0306] When the driver is driving, the calculation formula for the reading distance R of the warning sign is as follows:

[0307]

[0308] Where V0 is the vehicle speed when reading the warning sign (km / h), which is 100km / h; t1 is the driver's visual recognition time (s);

[0309] The reaction distance J includes: the distance J1 that the vehicle on the inner lane waits for to enter the gap and the driving distance J2 that the driver determines the time to enter the gap, as shown in the following formula:

[0310]

[0311] Where V is the design speed (km / h), which is 120km / h; t w is the average waiting time for entering the gap; v h is the minimum lane-changing speed, which is 0.76 times V (km / h); △v is the relative speed of the adjacent lanes (km / h); t is the critical time (s), which is 4s; λ is the average arrival rate (vehicles / s), which is 0.411 vehicles / s; τ is the minimum headway (s), which is 1.2s; t2 is the vehicle lateral movement time, which is 3.75 (s);

[0312] The lane change adjustment speed distance L1, the lane change lateral distance L2 and the deceleration distance L3 constitute the vehicle lane change action distance L, as shown in the following formula:

[0313]

[0314] Where, t1 is the time required to adjust the speed during lane change, which is 2.5s; t2 is the time required for lane change and lateral movement, which is 3.75s; V0 is 100km / h; V t is the speed limit within the road section, which is 80km / h; φ is the road adhesion coefficient, which is 0.4 for asphalt road surface;

[0315] The visual recognition distance S of the luminous warning LED display screen is calculated using a common model of the relationship between character height and symbol recognition distance. The expression is as follows:

[0316] S=-272.33h 2 +408.31h-25.72+ε

[0317] In the formula, S is the visual distance of the warning sign (m); h is the height of the Chinese character (m); ε is the correction coefficient of the complexity and simplicity of the Chinese character strokes and font structure, which is 20;

[0318] Therefore, the layout location of the luminous warning LED display screen should be no less than 424~710m from the lane stop line of the pedestrian crossing.

Claims

1. A pedestrian crossing warning system for non-signal controlled sections based on video AI recognition, characterized in that: include: Monitoring subsystem, early warning subsystem, power supply subsystem, control subsystem and processing subsystem; The monitoring subsystem includes: a video AI camera installed on the warning column, which has the processing subsystem installed inside; The warning subsystem includes: pedestrian warning columns installed on both sides of the crosswalk, on which flashing lights and pedestrian warning screens, warning columns, and luminous warning LED display screens are installed; An electronic power supply module is installed inside the warning column; A voice alarm device is installed inside the electronic power module; The luminous warning LED display screen and the pedestrian warning screen use day and night dual-opening mode signs; The power supply subsystem includes: a solar panel installed on the pedestrian warning column and an electronic power supply module installed on the warning column; The control subsystem is used to control the operation of the buried luminous road studs, the early warning voice alarm device, the flashing lights, the pedestrian early warning screen and the luminous early warning LED display screen; The control subsystem is connected to the signal receiving module.

2. A pedestrian crossing warning method for a non-signal controlled road section based on video AI recognition, the method is applicable to a pedestrian crossing warning system for a non-signal controlled road section based on video AI recognition, which includes a monitoring subsystem, an early warning subsystem, a power supply subsystem, a control subsystem and a processing subsystem, and is characterized in that The method comprises the following steps: S1. Use the video AI camera in the monitoring subsystem to collect image data in real time on the non-signal controlled road section and input the image data into the processing subsystem; S2, preprocessing the image data using the integrated image processing chip in the processing subsystem; Preprocessing includes: image denoising and histogram equalization; S3, input the processed image data into the YOLO model for pedestrian recognition and positioning; The YOLO model is installed in the video AI camera; S4, inputting the recognized and located image data into a convolutional neural network for feature extraction; The feature extraction includes: the pedestrian's position, speed, direction of travel, and distance from the vehicle; S5. Input the extracted features into the random forest model for judgment, and identify and warn pedestrian behavior based on the judgment results; The random forest model runs on the AI ​​analysis module; The AI ​​analysis module and the integrated image processing chip are located in the processing subsystem; The random forest model uses a voting mechanism and a weighted voting method to obtain a judgment result, and the judgment results include: preparing to cross the street, continuing to stay, and moving quickly.

3. According to claim 2, a method for warning pedestrians crossing a street in a non-signal controlled section based on video AI recognition is characterized in that: The identification and warning method includes: When it is identified that the pedestrian has no intention to cross the street, no action is taken to ensure normal traffic flow; When it is recognized that a pedestrian intends to cross the street, the pedestrian crossing signal is transmitted to the control subsystem through the signal receiving module to activate the early warning subsystem.

4. According to claim 3, a method for warning pedestrians crossing a street in a non-signal controlled section based on video AI recognition is characterized in that: The distance expression of the luminous warning LED display screen in the warning subsystem arranged in front of the pedestrian crossing is as follows: D≥R+J+L-S Where D is the leading distance of the warning sign before the lane stop line of the pedestrian crossing, R is the reading distance of the warning sign; J is the reaction distance; L is the action distance of the vehicle changing lanes; S is the visual distance of the warning sign (m).

5. According to claim 4, a method for warning pedestrians crossing a street in a non-signal controlled section based on video AI recognition is characterized in that: The expression of the reading distance of the warning sign is as follows: Where V0 is the vehicle speed when reading the warning sign (km / h), which is 100km / h; t1 is the driver's visual recognition time (s); The reaction distance includes: the distance J1 that the vehicle on the inner lane waits for to enter the gap and the driving distance J2 that the driver determines the time to enter the gap; the expression is as follows: Where V is the design speed (km / h), which is 120km / h; t w is the average waiting time for entering the gap; v h is the minimum lane-changing speed, which is 0.76 times V (km / h); △v is the relative speed of the adjacent lanes (km / h); t is the critical time (s), which is 4s; λ is the average arrival rate (vehicles / s), which is 0.411 vehicles / s; τ is the minimum headway (s), which is 1.2s; t2 is the vehicle lateral movement time, which is 3.75 (s); The lane-changing distance of the vehicle includes: the lane-changing speed adjustment distance L1, the lane-changing lateral movement distance L2 and the deceleration distance L3; the expression is as follows: Where, t1 is the time required to adjust the speed during lane change, which is 2.5s; t2 is the time required for lane change and lateral movement, which is 3.75s; V0 is 100km / h; V t is the speed limit within the road section, which is 80km / h; φ is the road adhesion coefficient, which is 0.4 for asphalt road surface; The expression of the visual recognition distance of the warning sign is as follows: S=-272.33h 2 +408.31h-25.72+e Where S is the visual distance of the warning sign (m); h is the height of the Chinese character (m); ε is the correction coefficient for the complexity and simplicity of the Chinese character strokes and font structure, which is taken as 20.

Images