A method and system for monitoring pedestrian yielding at a crosswalk

Abstract

The application discloses a zebra crossing pedestrian courtesy monitoring method and system, and relates to the technical field of traffic engineering, comprising: a pedestrian detection device and a crossing prompting device connected to a target zebra crossing; a first waiting area is detected for overline behavior through a millimeter wave radar, and when overline behavior is detected, pedestrian coordinate information and pedestrian motion information of a first pedestrian are generated; pedestrian trajectory prediction is performed to obtain a predicted trajectory; pedestrian motion capture is performed on the first pedestrian through a high-speed camera on the predicted trajectory in a second waiting area to obtain a pedestrian motion image set; pedestrian intention analysis is performed, and when it is determined that the first pedestrian wants to cross the zebra crossing, the crossing prompting device is activated to give courtesy prompts to vehicles. The application solves the technical problems of inaccurate pedestrian behavior prediction and inaccurate intention judgment in the prior art, achieves accurate detection and prediction of the behavior of pedestrians, and further improves the technical effect of safety at the zebra crossing.

Description

Technical Field

[0001] This invention relates to the field of traffic engineering technology, specifically to a monitoring method and system for monitoring pedestrian yielding at zebra crossings. Background Technology

[0002] With increasingly congested urban traffic, pedestrian safety at crosswalks has become a growing concern. While waiting to cross, pedestrians may engage in behaviors that cross the line prematurely, such as entering the crosswalk prematurely or walking back and forth on it. These actions can lead to traffic accidents, and the unclear intentions of pedestrians can also cause traffic chaos at crosswalks. Against this backdrop, existing technologies primarily focus on detecting pedestrian behavior through cameras and image processing techniques. However, these technologies suffer from inaccurate predictions of pedestrian behavior and inaccurate judgments of intent. Summary of the Invention

[0003] This application provides a monitoring method and system for pedestrian yielding at zebra crossings, which effectively solves the technical problems of inaccurate prediction of pedestrian behavior and inaccurate judgment of intent in the prior art, and achieves the technical effect of accurately detecting and predicting pedestrian behavior, thereby improving the safety of zebra crossings.

[0004] This application provides a method and system for monitoring pedestrian yielding at zebra crossings, the technical solution of which is as follows:

[0005] In a first aspect, embodiments of this application provide a method for monitoring pedestrian yielding at zebra crossings, the method comprising:

[0006] A pedestrian detection device and a crossing warning device are connected to a target zebra crossing. The pedestrian detection device includes a high-speed camera and a millimeter-wave radar.

[0007] The first waiting area is detected by millimeter-wave radar. When the crossing behavior is detected, the pedestrian coordinates and movement information of the first pedestrian are generated.

[0008] Based on the pedestrian coordinate information and the pedestrian movement information, perform pedestrian trajectory prediction to obtain the predicted trajectory;

[0009] On the predicted trajectory of the second waiting area, the high-speed camera captures the first pedestrian's motion to obtain a set of pedestrian motion images.

[0010] Based on the pedestrian motion image set, pedestrian intentions are analyzed. When it is determined that the first pedestrian is about to cross the zebra crossing, the crossing prompt device is activated to prompt oncoming vehicles to give way.

[0011] Secondly, embodiments of this application provide a monitoring system for pedestrian yielding at zebra crossings, the system comprising:

[0012] A connection module is used to connect the pedestrian detection device and the crossing warning device of the target zebra crossing. The pedestrian detection device includes a high-speed camera and a millimeter-wave radar.

[0013] The pedestrian information generation module is used to detect crossing behavior in the first waiting area using millimeter-wave radar. When crossing behavior is detected, the module generates pedestrian coordinate information and pedestrian movement information of the first pedestrian.

[0014] A predicted trajectory acquisition module is used to predict the pedestrian trajectory based on the pedestrian coordinate information and the pedestrian movement information, and to acquire the predicted trajectory.

[0015] A pedestrian motion image set acquisition module is used to capture the motion of the first pedestrian on the predicted trajectory of the second waiting area using the high-speed camera, and acquire a pedestrian motion image set.

[0016] The pedestrian crossing warning device activation module is used to analyze pedestrian intentions based on the pedestrian action image set. When it is determined that the first pedestrian is about to cross the zebra crossing, the pedestrian crossing warning device is activated to remind oncoming vehicles to give way.

[0017] One or more technical solutions provided in the embodiments of this application have at least the following technical effects or advantages:

[0018] This application first connects a pedestrian detection device and a crossing warning device at a target zebra crossing. The pedestrian detection device includes a high-speed camera and a millimeter-wave radar. Then, the millimeter-wave radar detects crossing behavior in a first waiting area. When crossing behavior is detected, the device generates the pedestrian's coordinates and movement information. Based on these coordinates and movement information, the device predicts the pedestrian's trajectory. Then, in a second waiting area, the high-speed camera captures the pedestrian's motion on the predicted trajectory, obtaining a set of pedestrian motion images. Based on these images, the device analyzes the pedestrian's intent. When it is determined that the pedestrian intends to cross the zebra crossing, the crossing warning device is activated to prompt oncoming vehicles to yield. This effectively solves the technical problems of inaccurate pedestrian behavior prediction and intent judgment in existing technologies, achieving accurate detection and prediction of pedestrian behavior, thereby improving safety at zebra crossings. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 A schematic flowchart of a method for monitoring pedestrian yielding at zebra crossings provided in this application embodiment;

[0021] Figure 2 This is a schematic diagram of a monitoring system for pedestrian yielding at zebra crossings, provided as an embodiment of this application.

[0022] Figure labeling: Module 1 is the connection module, Module 2 is the pedestrian information generation module, Module 3 is the predicted trajectory acquisition module, Module 4 is the pedestrian action image set acquisition module, and Module 5 is the traverse prompt device activation module. Detailed Implementation

[0023] This application provides a monitoring method and system for pedestrian yielding at zebra crossings, which addresses the technical problems of inaccurate prediction of pedestrian behavior and inaccurate judgment of intent in existing technologies.

[0024] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0025] It should be noted that the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or server that includes a series of steps or units is not necessarily limited to those steps or units that are explicitly listed, but may include other steps or modules that are not explicitly listed or that are inherent to such process, method, product, or device.

[0026] Example 1

[0027] like Figure 1 As shown, this invention provides a method for monitoring pedestrian yielding at crosswalks, used to detect pedestrian behavior in real time and predict their future actions, thereby providing advance warnings to vehicles, avoiding potential traffic conflicts, and improving traffic safety at crosswalks. The method includes:

[0028] The pedestrian detection device and the crossing warning device at the target zebra crossing are connected via communication. The target zebra crossing refers to the zebra crossing to be monitored, on which the pedestrian detection device and the crossing warning device are installed. The pedestrian detection device includes a high-speed camera and a millimeter-wave radar. The high-speed camera is used to capture images or videos at a high frame rate to capture pedestrian behavior and movement. The millimeter-wave radar is used to emit millimeter waves and receive reflected signals. By analyzing information such as the time and intensity of the reflected signals, information such as the pedestrian's position, speed, and direction is obtained. The crossing warning device is a sensor, traffic light, or other warning device installed on the road to remind vehicles to avoid pedestrians or to remind pedestrians not to cross. This includes sound warning systems, light warning systems, or vibration warning systems, which use sound, light, or vibration signals to alert vehicles and pedestrians.

[0029] The system uses millimeter-wave radar to detect pedestrian crossing behavior in the first waiting area, which is the area in front of the zebra crossing where pedestrians wait to cross the road. Within the first waiting area, the millimeter-wave radar emits millimeter waves and receives the reflected signals. By analyzing the reflected signals, the system determines the pedestrian's behavior pattern. When a change in the pedestrian's position information is detected, indicating that the pedestrian has crossed the boundary of the first waiting area, it is considered that a crossing behavior has occurred. After detecting the crossing behavior, the system obtains the pedestrian's position information through the millimeter-wave radar, including the pedestrian's coordinate information. Then, by analyzing the pedestrian's speed, angle, and other information obtained by the millimeter-wave radar, and combining it with the pedestrian's coordinate information, the system generates the pedestrian's motion information, such as the pedestrian's speed and direction. In summary, the system obtains the pedestrian coordinate information and pedestrian motion information of the first pedestrian, which refers to any one of the pedestrians waiting to cross the road in front of the zebra crossing.

[0030] Based on the pedestrian coordinate information and pedestrian movement information, pedestrian trajectory prediction is performed. First, the acquired pedestrian coordinate information and pedestrian movement information are preprocessed, including data cleaning, filtering, and standardization, to ensure the accuracy and reliability of the data. Then, features related to the pedestrian trajectory are extracted from the preprocessed data, including the pedestrian's position, speed, acceleration, and direction. Algorithms such as recurrent neural networks, long short-term memory networks, or decision trees are used to construct a prediction model. The extracted features are input into the prediction model, and the model learns based on these features and generates a predicted trajectory. The predicted trajectory is used to represent the pedestrian's movement path or behavior pattern in the future, including predicted trajectories in a first waiting area and a second waiting area. The second waiting area refers to the deceleration area before the zebra crossing, where pedestrians can slow down and prepare to enter the zebra crossing. A predicted trajectory appearing in the first waiting area indicates that the pedestrian crossed the line and then returned to the first waiting area to wait. A predicted trajectory appearing in the second waiting area indicates that the pedestrian crossed the line and then continued to move forward along the zebra crossing direction.

[0031] When the first pedestrian enters the second waiting area, a high-speed camera starts working on the predicted trajectory in the second waiting area to capture the pedestrian's movements and obtain a set of pedestrian action images, including the pedestrian's walking, running, jumping and other actions, as well as the time sequence and spatial location information of these actions.

[0032] Pedestrian intent analysis is performed based on the aforementioned set of pedestrian action images. First, the captured pedestrian action images are preprocessed, including image cleaning, filtering, and segmentation, to extract pedestrian features and behavioral patterns, such as position, speed, direction, and action type. Machine learning or deep learning algorithms are then used to analyze the extracted pedestrian features and behavioral patterns to identify the pedestrian's intent. For example, if a pedestrian is accelerating towards a crosswalk, it is determined that the pedestrian intends to cross the road. When the first pedestrian is determined to be crossing the crosswalk, a crossing warning device is activated. This device alerts oncoming vehicles to yield to the pedestrian through sound, light, or other warning signals. Upon receiving the warning signal, the driver slows down, stops, or yields to the pedestrian based on the actual situation. This achieves the technical effect of accurately detecting and predicting pedestrian behavior, thereby improving safety at crosswalks.

[0033] In a preferred embodiment provided in this application, it further includes:

[0034] A preset distance is determined before the target zebra crossing, and a vehicle speed detection device, such as radar or a camera sensor, is installed at this distance to detect vehicle speed. When a vehicle approaches the zebra crossing, the speed detection device measures its speed and transmits the result to the system. The system receives the speed result and compares it to the preset speed. If the detected speed is greater than or equal to the preset speed (i.e., the vehicle is traveling too fast), a pedestrian crossing warning device is activated. This device emits sound and light signals to remind pedestrians not to cross the road. After receiving the warning, pedestrians adjust their behavior accordingly, such as waiting for the vehicle to pass or choosing an alternative route. This preferred embodiment, by detecting vehicle speed and reminding pedestrians, reduces traffic accidents caused by excessive vehicle speed, thus improving pedestrian safety when crossing the road.

[0035] In another preferred embodiment provided in this application, the detection of line-crossing behavior in the first waiting area is performed using millimeter-wave radar, including:

[0036] Two pillars are set on both sides of the zebra crossing in the first waiting area. The first pillar is used to install millimeter-wave radar, which includes a signal transmitting module and a signal receiving module. The second pillar acts as a reflector to reflect millimeter-wave signals.

[0037] The system continuously transmits a first signal through the signal transmitting module of the millimeter-wave radar and continuously receives a first reflected signal through the signal receiving module of the millimeter-wave radar. The system automatically records the time points of transmitting the first signal and receiving the first reflected signal, and calculates the time difference between the two, i.e., the first time difference. This time difference is used to reflect whether the pedestrian is in the first waiting area.

[0038] When the first time difference decreases, it indicates that a pedestrian is obstructing the signal propagation between the two pillars. The pedestrian is located at the boundary of the first waiting area, and at this point, it is determined that there has been a crossing behavior. This preferred embodiment can accurately detect pedestrian crossing behavior, achieving the technical effect of improving the accuracy and reliability of detection.

[0039] In another preferred embodiment provided in this application, when a line-crossing behavior is detected, pedestrian coordinate information and pedestrian movement information of the first pedestrian are generated, including:

[0040] When a line-crossing behavior is detected, a second time difference is acquired. This time difference reflects the distance between the pedestrian and the millimeter-wave radar. Using this second time difference, the distance between the first pedestrian and the millimeter-wave radar is calculated based on the relationship between distance, speed, and time. The pedestrian's coordinate information is then determined based on the distance calculation result. The second time difference is continuously measured to obtain its duration. The speed of the first pedestrian is calculated based on this duration, and this speed is used to generate pedestrian motion information, thereby obtaining the pedestrian's coordinate information and motion information. This preferred embodiment, by accurately calculating the pedestrian's position and motion information, can better determine their motion state and behavior pattern, achieving the technical effect of improving the accuracy and reliability of line-crossing detection.

[0041] In another preferred embodiment provided in this application, when the duration of the second time difference exceeds a preset threshold, it indicates that the pedestrian has been staying on the line for too long, and it is determined that the pedestrian is staying on the line and has a tendency to cross the road. At this time, the system activates the crossing warning device and reminds the pedestrian to back off through sound prompts or other means. This preferred embodiment, by reminding pedestrians to back off when they are staying on the line, can effectively avoid traffic accidents and achieve the technical effect of improving road safety.

[0042] In another preferred embodiment provided in this application, pedestrian intent analysis is performed based on the pedestrian action image set, including:

[0043] Data is extracted from historical pedestrian detection records to obtain multiple sets of sample pedestrian images and corresponding sets of sample intent analysis results, which are then labeled. For example, the action labels of pedestrians in the pedestrian detection results (such as "walking", "running", "stopping") are labeled with the corresponding images.

[0044] An intent analysis model is constructed based on neural networks (convolutional neural networks, recurrent neural networks, etc.). The neural network consists of multiple neurons, each receiving input and generating output. The intent analysis model is trained under supervised supervision using multiple sets of sample pedestrian images and multiple sets of sample intent analysis results. During training, the sample pedestrian image sets and corresponding intent analysis result sets are used as input and output, respectively. The model parameters are updated using a backpropagation algorithm to make the model's predictions as close as possible to the true labels. When the model reaches convergence, it indicates that the model's predictions meet the requirements, training stops, and the final intent analysis model is obtained.

[0045] The set of pedestrian action images is input into the intent analysis model. The model automatically extracts features from the images and performs tasks such as classification or regression, ultimately outputting the predicted intent analysis result. This preferred embodiment trains the model using a neural network. Neural networks have powerful feature learning and generalization capabilities, accurately distinguishing different pedestrian actions and intentions, thus achieving the technical effect of improving the accuracy of intent analysis.

[0046] Example 2

[0047] Based on the same inventive concept as the zebra crossing yielding-to-pedestrian monitoring method in the foregoing embodiments, such as Figure 2 As shown, this application provides a monitoring system for pedestrian yielding at zebra crossings. The system and method embodiments in this application are based on the same inventive concept. The system includes:

[0048] Connection module 1, which is used to connect the pedestrian detection device and the crossing warning device of the target zebra crossing, wherein the pedestrian detection device includes a high-speed camera and a millimeter-wave radar;

[0049] Pedestrian information generation module 2 is used to detect crossing behavior in the first waiting area using millimeter-wave radar. When crossing behavior is detected, pedestrian coordinate information and pedestrian movement information of the first pedestrian are generated.

[0050] The predicted trajectory acquisition module 3 is used to predict the pedestrian trajectory based on the pedestrian coordinate information and the pedestrian movement information, and to acquire the predicted trajectory.

[0051] Pedestrian motion image set acquisition module 4 is used to capture the motion of the first pedestrian on the predicted trajectory of the second waiting area using the high-speed camera, and acquire a set of pedestrian motion images.

[0052] The pedestrian crossing prompting device activation module 5 is used to analyze pedestrian intentions based on the pedestrian action image set. When it is determined that the first pedestrian is about to cross the zebra crossing, the pedestrian crossing prompting device is activated to prompt oncoming vehicles to give way.

[0053] Furthermore, the traversal prompting device activation module 5 is used to perform the following method:

[0054] Vehicle speed detection devices are deployed at a predetermined distance from the target zebra crossing;

[0055] The vehicle speed is detected by the vehicle speed detection device to determine whether the detected vehicle speed is greater than or equal to the preset vehicle speed.

[0056] If the detected vehicle speed is greater than or equal to the preset speed, the cross-traffic warning device will be activated to remind pedestrians not to cross.

[0057] Furthermore, the pedestrian information generation module 2 is used to perform the following method:

[0058] Two pillars are set on both sides of the zebra crossing in the first waiting area. The first pillar is used to deploy millimeter-wave radar, and the second pillar is used to reflect millimeter-wave signals.

[0059] The first signal is continuously transmitted through the signal transmitting module of the millimeter-wave radar, and the first reflected signal is continuously received through the signal receiving module of the millimeter-wave radar. At the same time, the first time difference between transmitting the first signal and receiving the first reflected signal is obtained.

[0060] When the first time difference decreases, it is determined that there is a violation of the boundary.

[0061] Furthermore, the pedestrian information generation module 2 is used to perform the following method:

[0062] When a line-crossing behavior is detected, the second time difference is obtained;

[0063] The distance between the first pedestrian and the millimeter-wave radar is calculated using the second time difference, and pedestrian coordinate information is generated based on the distance calculation result;

[0064] The second time difference is continuously measured to obtain the duration of the second time difference. The speed of the first pedestrian is calculated based on the duration to generate pedestrian motion information.

[0065] Furthermore, the pedestrian information generation module 2 is used to perform the following method:

[0066] When the duration of the second time difference exceeds a preset threshold, it is determined that the pedestrian is standing on the line, and the crossing prompt device is activated to prompt the pedestrian to back away.

[0067] Furthermore, the traversal prompting device activation module 5 is used to perform the following method:

[0068] Data is extracted from historical pedestrian detection records to obtain multiple sets of sample pedestrian images and multiple sets of sample intent analysis results, which are then labeled.

[0069] An intent analysis model is built based on neural networks;

[0070] The intent analysis model is trained under supervision using the multiple sets of pedestrian images and the multiple sets of intent analysis results. When the model converges, the intent analysis model is obtained.

[0071] The set of pedestrian action images is input into the intent analysis model, and the intent analysis results are output.

[0072] It should be noted that the order of the embodiments described above is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. Furthermore, specific embodiments have been described above. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps described in the claims can be performed in a different order than that shown in the embodiments and still achieve the desired result. Additionally, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0073] The above description is only a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

[0074] This specification and accompanying drawings are merely illustrative examples of this application and are intended to cover any and all modifications, variations, combinations, or equivalents within the scope of this application. Clearly, those skilled in the art can make various alterations and modifications to this application without departing from its scope. Therefore, if such modifications and modifications fall within the scope of this application and its equivalents, this application intends to include such modifications and modifications.

Claims

1. A method for monitoring pedestrian yielding at zebra crossings, characterized in that, include: A pedestrian detection device and a crossing warning device are connected to a target zebra crossing. The pedestrian detection device includes a high-speed camera and a millimeter-wave radar. The first waiting area is detected by millimeter-wave radar. When the crossing behavior is detected, the pedestrian coordinates and movement information of the first pedestrian are generated. Based on the pedestrian coordinate information and the pedestrian movement information, perform pedestrian trajectory prediction to obtain the predicted trajectory; The predicted trajectory is used to represent the pedestrian's movement path or behavior pattern in the future, including the predicted trajectory in the first waiting area and the predicted trajectory in the second waiting area. The second waiting area refers to the deceleration area in front of the zebra crossing, which is for pedestrians to slow down and prepare to enter the zebra crossing. When the first pedestrian enters the second waiting area, the high-speed camera starts working on the predicted trajectory in the second waiting area to capture the pedestrian's movements and obtain a set of pedestrian movement images. Based on the set of pedestrian motion images, pedestrian intentions are analyzed. When it is determined that the first pedestrian is about to cross the zebra crossing, the crossing prompt device is activated to prompt oncoming vehicles to give way. Vehicle speed detection devices are deployed at a predetermined distance from the target zebra crossing; The vehicle speed is detected by the vehicle speed detection device to determine whether the detected vehicle speed is greater than or equal to the preset vehicle speed. If the detected vehicle speed is greater than or equal to the preset speed, the cross-traffic warning device will be activated to remind pedestrians not to cross. Two pillars are set on both sides of the zebra crossing in the first waiting area. The first pillar is used to deploy millimeter-wave radar, and the second pillar is used to reflect millimeter-wave signals. The first signal is continuously transmitted through the signal transmitting module of the millimeter-wave radar, and the first reflected signal is continuously received through the signal receiving module of the millimeter-wave radar. At the same time, the first time difference between transmitting the first signal and receiving the first reflected signal is obtained. When the first time difference decreases, it is determined that there is a violation of the boundary. When a line-crossing behavior is detected, the pedestrian coordinates and movement information of the first pedestrian are generated, including: When a line-crossing behavior is detected, the second time difference is obtained; The distance between the first pedestrian and the millimeter-wave radar is calculated using the second time difference, and pedestrian coordinate information is generated based on the distance calculation result; The second time difference is continuously measured to obtain the duration of the second time difference. The speed of the first pedestrian is calculated based on the duration to generate pedestrian motion information.

2. The method as described in claim 1, characterized in that, When the duration of the second time difference exceeds a preset threshold, it is determined that the pedestrian is standing on the line, and the crossing prompt device is activated to prompt the pedestrian to back away.

3. The method as described in claim 1, characterized in that, Pedestrian intent analysis is performed based on the set of pedestrian action images, including: Data is extracted from historical pedestrian detection records to obtain multiple sets of sample pedestrian images and multiple sets of sample intent analysis results, which are then labeled. An intent analysis model is built based on neural networks; The intent analysis model is trained under supervision using the multiple sets of pedestrian images and the multiple sets of intent analysis results. When the model converges, the intent analysis model is obtained. The set of pedestrian action images is input into the intent analysis model, and the intent analysis results are output.

4. A monitoring system for pedestrian yielding at zebra crossings, used to perform the method described in claim 1, characterized in that, The system includes: A connection module is used to connect the pedestrian detection device and the crossing warning device of the target zebra crossing. The pedestrian detection device includes a high-speed camera and a millimeter-wave radar. The pedestrian information generation module is used to detect crossing behavior in the first waiting area using millimeter-wave radar. When crossing behavior is detected, the module generates pedestrian coordinate information and pedestrian movement information of the first pedestrian. A predicted trajectory acquisition module is used to predict the pedestrian trajectory based on the pedestrian coordinate information and the pedestrian movement information, and to acquire the predicted trajectory. A pedestrian motion image set acquisition module is used to capture the motion of the first pedestrian on the predicted trajectory of the second waiting area using the high-speed camera, and acquire a pedestrian motion image set. The pedestrian crossing warning device activation module is used to analyze pedestrian intentions based on the pedestrian action image set. When it is determined that the first pedestrian is about to cross the zebra crossing, the pedestrian crossing warning device is activated to remind oncoming vehicles to give way.

Images