Intelligent intersection-oriented pedestrian fall visual perception and multi-level alarm system

Abstract

The application discloses a pedestrian fall visual perception and multi-level alarm system for a smart intersection, and relates to the technical field of intelligent traffic, which comprises an intersection edge visual perception module, a fall event confirmation module, an intersection area calibration and state fusion module, a multi-level alarm control module and an end-cloud collaborative tracing module. The system collects intersection videos through a camera, calls a lightweight fall detection model at the edge end to output detection frame coordinates and fall confidence, and confirms a fall event in combination with continuous frame position consistency. The system further fuses position information, time information and traffic signal states to generate intersection fall event data, triggers multi-level alarms of the scene, the roadside / platform and the emergency terminal according to the fall area, the traffic conflict relationship and the event duration state, and uploads the event data to the cloud for map annotation, historical display and image tracing. The application can improve the timeliness of the discovery of the fall event at the smart intersection and the pertinence of the disposal.

Description

Technical Field

[0001] This invention relates to the field of intelligent transportation technology, specifically to a pedestrian fall visual perception and multi-level alarm system for intelligent intersections. Background Technology

[0002] With the development of intelligent transportation and urban public safety management, video surveillance equipment, traffic signal control equipment, and intelligent intersection management platforms are typically deployed at urban intersections, pedestrian crossings, bus stops, and transportation hubs to collect traffic operation status, pedestrian crossing status, and on-site video information. When a pedestrian falls in an open traffic environment, there is a risk of secondary injury due to passing vehicles, blind spots during turns, pedestrian obstruction, or failure of rescuers to notice in time. Therefore, it is necessary to promptly detect and alert to abnormal pedestrian conditions at intersections.

[0003] Existing intersection video surveillance systems are mostly manually operated or rely on post-event recording and playback, primarily serving the functions of on-site recording and evidence collection. They typically lack the ability to actively identify pedestrian fall postures and automatically trigger alarms, making it difficult to meet the needs for 24 / 7, low-latency handling of abnormal events. While some existing visual fall detection solutions incorporate deep learning models, they largely rely on high-performance servers or cloud computing, resulting in high deployment costs. Furthermore, given the limited computing power of embedded devices at intersections and network transmission latency, it is difficult to balance real-time detection with stable alarms. Furthermore, real-world intersection scenarios are affected by factors such as changes in lighting, vehicle obstruction, pedestrian overlap, and complex backgrounds, making single-frame detection results prone to false positives or false negatives. Existing solutions typically only output a single judgment of whether a fall has occurred, rarely combining factors such as the fall location, traffic signal status, vehicle clearance direction, and event duration to differentiate risks. Alarm methods are also mostly limited to local notifications or platform recordings, lacking a tiered linkage mechanism for on-site personnel, roadside traffic participants, management platforms, and emergency terminals. Therefore, it is necessary to improve the visual perception and alarm technology for pedestrian falls at smart intersections. Summary of the Invention

[0004] To address the aforementioned technical issues, a pedestrian fall visual perception and multi-level alarm system for smart intersections is provided. This technical solution solves the problems of existing intersection monitoring relying on manual labor, delayed fall detection, single alarm method, and lack of traffic status integration and cloud-based tracking.

[0005] To achieve the above objectives, the technical solution adopted by the present invention is as follows: A pedestrian fall detection and multi-level alarm system for smart intersections includes: The intersection edge visual perception module is used to collect surveillance videos of smart intersections, preprocess video frames, and call a lightweight fall detection model on an embedded edge computing device to output the coordinates of the detection box of the suspected fall target and the fall confidence score. The fall event confirmation module is used to generate fall events based on fall confidence, the consistency of detection box positions in consecutive video frames, and preset continuous confirmation conditions. It uses alarm trigger frames as key frame images and suppresses repeated alarm triggering of the same fall event within a preset alarm cooling period. The intersection area calibration and status fusion module is used to perform semantic area calibration of the intersection in the camera's field of view, generate area boundary data including pedestrian crossing area, curb area and motor vehicle traffic conflict area, and map the detection box coordinates to the corresponding intersection area to determine the fall location. At the same time, it acquires the location information, time information and traffic signal status of the fall event, and generates intersection fall event data. The multi-level alarm control module is used to determine the alarm level based on intersection fall event data. When a fall event is generated, a level 1 on-site audible and visual alarm is triggered. When the fall location is in a pedestrian crossing area, curb area, or motor vehicle traffic conflict zone, or when a traffic conflict is determined based on the mapping relationship between the boundary data of the area and the current motor vehicle traffic direction, a level 2 roadside prompt and platform alarm are triggered. When the fall event continues for a preset escalation time, a level 3 emergency terminal alarm is triggered. The edge-cloud collaborative tracing module is used to report intersection fall event data to the cloud server, which stores and generates map annotations, historical event displays, and image retrospectives.

[0006] Preferably, in the intersection edge visual perception module, the video frame preprocessing includes: The video frames are scaled, pixel normalized, and converted to tensor format, and the preprocessed video frames are then input into the lightweight fall detection model. The detection box coordinates output by the lightweight fall detection model include the center point coordinates of the suspected fall target, the width of the detection box, and the height of the detection box. The fall confidence score is used to characterize the probability that the suspected fall target is in a fall posture.

[0007] Preferably, the lightweight fall detection model is a lightweight target detection model or a human posture detection model. The lightweight target detection model includes a YOLOv8n model, a YOLOv5s model, or a lightweight improved YOLO model. The lightweight fall detection model is deployed on the embedded edge computing device in ONNX format or a model format adapted to the embedded inference framework, and the embedded edge computing device performs local inference.

[0008] Preferably, in the fall event confirmation module, the process of generating a fall event based on fall confidence, consistency of detection box positions in consecutive video frames, and preset continuous confirmation conditions includes: The suspected fall targets are screened based on the fall confidence level, and non-maximum suppression processing is performed on the screened detection boxes. The consistency of the position of the same target in consecutive video frames is determined by the overlap (IOU) between detection boxes in adjacent video frames being greater than a preset threshold or the distance between the center points of the detection boxes being less than a preset distance threshold. The fall event is generated when the same suspected fall target meets the fall confidence condition and the detection box position consistency condition within a consecutive preset number of frames.

[0009] Preferably, during the process of the fall event confirmation module generating a fall event based on the positional consistency of the detection box coordinates in consecutive video frames, the intersection edge visual perception module obtains the detection box coordinates and fall confidence of the suspected fall target according to a preset frame skipping detection strategy. The preset frame skipping detection strategy includes: The lightweight fall detection model is invoked for inference every preset number of frames. The detection box coordinates and fall confidence scores output from the previous inference frame are used between adjacent inference frames; During the preset alarm cooling-off period, no new fall events will be output to the multi-level alarm control module for the same fall event.

[0010] Preferably, in the intersection area calibration and state fusion module, the method for obtaining the location information of the fall event includes: The latitude and longitude coordinates corresponding to the fall event can be obtained through the positioning module, or the preset intersection location information can be obtained according to the device identification deployed at the smart intersection; When the location information is obtained through the positioning module, the intersection area calibration and status fusion module performs coordinate system transformation on the original latitude and longitude coordinates output by the positioning module to generate location information for map annotation.

[0011] Preferably, in the intersection area calibration and state fusion module, the intersection fall event data includes: Event number, device identifier, detection frame coordinates, keyframe image, fall confidence level, location information, time information, and traffic signal status identifier; After determining the alarm level, the multi-level alarm control module writes the alarm level into the intersection fall event data and provides the intersection fall event data with the alarm level written into it to the end-to-cloud collaborative tracing module.

[0012] Preferably, in the multi-level alarm control module, after determining the alarm level based on the intersection fall event data, the alarm actions triggered according to the alarm level include: The Level 1 on-site audible and visual alarm includes one or more of the following: buzzer alarm, voice broadcast alarm, and on-site warning light alarm. The secondary roadside display or broadcast prompts include sending text or image prompts to roadside display screens or sending voice prompts to roadside broadcasting equipment; The smart intersection platform alarm includes sending pop-up alarms or event notifications to the smart intersection platform; The Level 3 emergency terminal alarm includes sending an alarm message containing the location information, the time information, the alarm level, and the keyframe image to a preset management terminal or emergency response terminal.

[0013] Preferably, the traffic signal status is obtained from the traffic light recognition results in the smart intersection signal control equipment, the smart intersection management platform interface, or video images; when the area corresponding to the fall location in the intersection fall event data is a pedestrian crossing area, and the current traffic signal status indicates that the vehicle passage direction corresponding to the pedestrian crossing area is in a free state, or when the area corresponding to the fall location is a vehicle passage conflict area, the multi-level alarm control module determines that there is a passage conflict between the area corresponding to the fall location and the current vehicle passage direction, and determines the alarm level to be no less than level two, triggering the level two roadside display or broadcast prompt and the smart intersection platform alarm.

[0014] Preferably, after the intersection fall event data is sent to the cloud server in the end-to-cloud collaborative tracing module, the cloud server saves the key frame image to the file storage space, writes the location information, time information, traffic signal status, fall confidence level and the alarm level into the event database, generates the map label based on the location information, and generates a historical event list and image retrospective interface based on the event database.

[0015] Compared with the prior art, the beneficial effects of the present invention are as follows: This invention proposes to invoke a lightweight fall detection model on an embedded edge computing device to perform local inference on smart intersection video frames, solving the problems of traditional intersection monitoring relying on manual viewing, high cloud computing latency, and untimely fall event detection. It proposes to generate fall events based on fall confidence, detection box position consistency, and preset continuous confirmation conditions, addressing the issue of false alarms caused by single-frame detection being susceptible to posture changes, occlusion, and complex backgrounds. It proposes to fuse location information, time information, traffic signal status, keyframe images, and fall confidence to generate intersection fall event data, addressing the problem of existing fall detection methods not incorporating intersection traffic conditions for risk identification. It proposes to trigger multi-level alarms based on fall location, vehicle travel direction, and event duration, addressing the issues of single alarm methods and lack of tiered handling. Finally, it proposes to report event data to the cloud and generate map annotations, historical displays, and image retrospectives, solving the problems of difficulty in locating, verifying, and tracing fall events. Attached Figure Description

[0016] Figure 1 This is a system framework diagram of the present invention. Detailed Implementation

[0017] The following description is intended to disclose the invention and enable those skilled in the art to implement it. The preferred embodiments described below are merely examples, and other obvious variations will occur to those skilled in the art.

[0018] Reference Figure 1 As shown, a pedestrian fall detection and multi-level alarm system for smart intersections includes: The intersection edge visual perception module is used to collect surveillance videos of smart intersections, preprocess video frames, and call a lightweight fall detection model on an embedded edge computing device to output the coordinates of the detection box of the suspected fall target and the fall confidence score. The fall event confirmation module is used to generate fall events based on fall confidence, the consistency of detection box positions in consecutive video frames, and preset continuous confirmation conditions. It uses alarm trigger frames as key frame images and suppresses repeated alarm triggering of the same fall event within a preset alarm cooling period. The intersection area calibration and status fusion module is used to perform semantic area calibration of the intersection in the camera's field of view, generate area boundary data including pedestrian crossing area, curb area and motor vehicle traffic conflict area, and map the detection box coordinates to the corresponding intersection area to determine the fall location. At the same time, it acquires the location information, time information and traffic signal status of the fall event, and generates intersection fall event data. The multi-level alarm control module is used to determine the alarm level based on intersection fall event data. When a fall event is generated, a level 1 on-site audible and visual alarm is triggered. When the fall location is in a pedestrian crossing area, curb area, or motor vehicle traffic conflict zone, or when a traffic conflict is determined based on the mapping relationship between the boundary data of the area and the current motor vehicle traffic direction, a level 2 roadside prompt and platform alarm are triggered. When the fall event continues for a preset escalation time, a level 3 emergency terminal alarm is triggered. The edge-cloud collaborative tracing module is used to report intersection fall event data to the cloud server, which stores and generates map annotations, historical event displays, and image retrospectives.

[0019] In one implementation, the intersection edge visual perception module is deployed in an intersection edge device, which may include an embedded edge computing device, a camera, a positioning module, a local alarm device, and a communication module. The embedded edge computing device may be a Raspberry Pi, an embedded industrial computer, an edge computing box, or other devices with local inference capabilities; the camera may be a USB camera, a network camera, or an existing surveillance camera at the smart intersection; the positioning module may be a GPS positioning module, a BeiDou positioning module, or a GPS / BeiDou dual-mode positioning module; the local alarm device may include one or more of a buzzer, a voice broadcast module, and on-site warning lights. A cloud server may be deployed with a data receiving interface, an event database, image storage space, and a web visualization dashboard for receiving, saving, and displaying intersection fall event data.

[0020] In one implementation, the intersection edge visual perception module continuously collects surveillance video via cameras installed at the smart intersection. The cameras can be installed on traffic light poles, roadside monitoring poles, intersection equipment boxes, or in locations that cover pedestrian crossings, curb areas, and conflict zones between vehicular and motor vehicle traffic. The video acquisition resolution can be set according to the capabilities of the on-site equipment; for example, a video stream with a resolution of 640×480 and a frame rate of 30fps can be acquired, or other resolutions and frame rates can be used.

[0021] Furthermore, in the intersection edge visual perception module, the video frame preprocessing includes: The video frames are scaled, pixel normalized, and converted to tensor formats. Specifically, the acquired video frames can be scaled to the input size of the lightweight fall detection model, for example, to 640×640; then the pixel values ​​are normalized to the [0,1] range, or to other numerical ranges required by the model inference framework; and then the image data is converted to the tensor format required for model inference, for example, to NCHW format. The preprocessed video frames are then input into the lightweight fall detection model. The lightweight fall detection model outputs detection box coordinates including the center point coordinates of the suspected fall target, the detection box width, and the detection box height. The fall confidence score characterizes the probability that the suspected fall target is in a falling posture. Using the detection box coordinates, the system can determine the position of the suspected fall target in the video frame; using the fall confidence score, the system can determine whether the suspected fall target meets the subsequent event confirmation conditions.

[0022] The lightweight fall detection model is a lightweight target detection model or a human posture detection model. The lightweight target detection model includes the YOLOv8n model, the YOLOv5s model, or a lightweight improved YOLO model. The human posture detection model can determine whether a pedestrian is in a fall posture by using human skeletal key points, human posture angles, changes in human aspect ratio, or the continuous state of posture.

[0023] The lightweight fall detection model is deployed on the embedded edge computing device in ONNX format or a model format compatible with the embedded inference framework. The embedded edge computing device can perform local inference through ONNX Runtime or other inference engines compatible with the embedded inference framework. When using local inference, the system can complete fall detection without uploading the complete video stream to the cloud server, thereby reducing communication latency and meeting the real-time alarm requirements in smart intersection scenarios.

[0024] The fall event confirmation module receives the detection box coordinates and fall confidence score output by the intersection edge visual perception module, and generates a fall event based on the fall confidence score, the consistency of the detection box position in consecutive video frames, and preset continuous confirmation conditions.

[0025] Furthermore, the suspected fall targets are screened based on the fall confidence level. When the fall confidence level is lower than a preset confidence level, the detection result can be filtered as a low-confidence detection result; when the fall confidence level reaches the preset confidence level, the detection result is used as a candidate detection result. In one specific embodiment, the preset confidence level can be set to a fall confidence level of not less than 0.8; in other embodiments, this threshold can be adjusted according to the intersection environment, model performance, and alarm sensitivity requirements. In the case of multiple overlapping candidate detection boxes in the same video frame, the fall event confirmation module performs non-maximum suppression processing on the filtered detection boxes to remove redundant detection boxes corresponding to the same target and retain the detection boxes with higher fall confidence. The consistency of the position of the same target in consecutive video frames is determined by the overlap ratio (IOU) between detection boxes in adjacent video frames being greater than a preset threshold or the distance between the center points of the detection boxes being less than a preset distance threshold; wherein, the overlap ratio (IOU) is the ratio of the intersection area to the union area of ​​two detection boxes in adjacent video frames.

[0026] Specifically, when the overlap between two detection boxes in adjacent video frames reaches a preset overlap condition, the two detection boxes can be considered to correspond to the same suspected fall target; or, when the distance between the center points of two detection boxes in adjacent video frames is less than a preset distance condition, the two detection boxes can also be considered to correspond to the same suspected fall target. This method can improve the reliability of continuous confirmation of the same suspected fall target in situations with many pedestrians, vehicle obstructions, or complex intersection backgrounds. A fall event is generated when the same suspected falling target meets both the fall confidence condition and the detection box position consistency condition within a consecutive preset number of frames. The consecutive preset number of frames can be set according to the video frame rate, edge device computing power, and alarm response requirements. For example, it can be set to generate a fall event when three consecutive inference results meet the conditions. Continuous confirmation can reduce false alarms caused by single-frame false detections, instantaneous posture changes, or accidental occlusion.

[0027] During the process of the fall event confirmation module generating a fall event based on the positional consistency of the detection box coordinates in consecutive video frames, the intersection edge visual perception module obtains the detection box coordinates and fall confidence of the suspected fall target according to a preset frame skipping detection strategy. The preset frame skipping detection strategy includes: The lightweight fall detection model is invoked for inference every preset number of frames. The detection box coordinates and fall confidence scores output from the previous inference frame are used between adjacent inference frames; For example, model inference can be performed every 5 frames, with intermediate frames using the detection results from the previous inference frame. This approach reduces the number of model inferences, lowers the processing load on embedded edge computing devices, and enables the system to maintain real-time monitoring capabilities even with limited computing power.

[0028] During the preset alarm cooling-off period, no new fall events will be output to the multi-level alarm control module for the same fall event.

[0029] Furthermore, after the fall event confirmation module generates a fall event, the system uses the alarm trigger frame as a keyframe image and records the event number, detection box coordinates, keyframe image, and generation time of the fall event. Simultaneously, a preset alarm cooldown period is initiated. During this preset cooldown period, if subsequent detection results meet the same event association condition as the already generated fall event, the fall event confirmation module will no longer output new fall events for the same fall event to the multi-level alarm control module.

[0030] Conditions for associating the same event can include: the overlap between the subsequent detection box and the detection box that has already generated the fall event reaches a preset overlap condition, or the distance between the center point of the subsequent detection box and the center point of the detection box that has already generated the fall event is less than a preset distance condition. This method avoids repeatedly triggering Level 1 alarms, repeatedly triggering roadside alarms, and repeatedly reporting to the cloud server during the continuous fall of the same pedestrian. In one specific embodiment, the preset alarm cooldown period can be set to 10 seconds, or it can be adjusted according to the pedestrian traffic at the intersection, the alarm frequency, and the equipment deployment requirements.

[0031] In the intersection area calibration and state fusion module, the methods for obtaining the location information of the fall event include: The latitude and longitude coordinates corresponding to the fall event are obtained by the positioning module, and the intersection area calibration and state fusion module reads the current valid latitude and longitude coordinates as the location information corresponding to the fall event when generating the fall event.

[0032] In one implementation, the positioning module continuously outputs positioning data. When a fall event is generated, the intersection area calibration and state fusion module reads the current valid latitude and longitude coordinates and uses these coordinates as the original location information corresponding to the fall event. Specifically, the positioning module can output positioning data conforming to the NMEA-0183 format. The intersection area calibration and state fusion module parses the positioning statements within the data and extracts information such as latitude and longitude, positioning quality, and number of satellites. When the positioning quality meets the valid positioning conditions and the latitude and longitude are not empty, the latitude and longitude are used as the original location information.

[0033] In one specific embodiment, the original latitude and longitude coordinates output by the positioning module are coordinates in the WGS-84 coordinate system. To adapt to cloud map display, the intersection area calibration and status fusion module performs coordinate system transformation on the original latitude and longitude coordinates to generate location information for map annotation. For example, in scenarios using domestic map services, the WGS-84 coordinates can be converted to GCJ-02 coordinates before being reported to the cloud server.

[0034] In another implementation, for devices fixedly deployed at smart intersections, the intersection area calibration and status fusion module obtains preset intersection location information based on the device identifier. Specifically, a correspondence table between device identifiers and intersection latitude and longitude coordinates can be pre-established; when a fall event occurs, the intersection area calibration and status fusion module reads the corresponding preset intersection location information based on the device identifier. This method is suitable for scenarios where the fixed intersection device position is stable, the positioning module signal is obstructed, or hardware costs need to be reduced.

[0035] Furthermore, the intersection area calibration and state fusion module also acquires the time information and traffic signal status corresponding to the fall event. The time information can be determined by the system time of the embedded edge computing device, network time synchronization information, or time information returned by the cloud server. The traffic signal status can be provided by the smart intersection signal control device, obtained from the smart intersection management platform interface, or obtained from the traffic light recognition results in the video image. The traffic signal status may include one or more of the following: vehicle travel direction, pedestrian crossing status, pedestrian prohibition status, signal phase number, or traffic light color.

[0036] Furthermore, the intersection area calibration and state fusion module correlates the detection box coordinates, keyframe images, fall confidence scores, location information, time information, and traffic signal status corresponding to the fall event to generate intersection fall event data. The intersection fall event data may include an event number, device identifier, detection box coordinates, keyframe images, fall confidence scores, location information, time information, and traffic signal status identifier. The event number is used to distinguish different fall events, the device identifier is used to identify the device from which the event originated, the keyframe images are used for event verification and image backtracking, and the traffic signal status identifier is used to record the intersection traffic status corresponding to the time the fall event occurred.

[0037] Furthermore, to determine whether the fall location is within a pedestrian crossing, curb area, or vehicular traffic conflict zone, the system performs intersection area calibration on the camera's field of view during deployment. Specifically, pedestrian crossings, curb areas, vehicular traffic conflict zones, and ordinary traffic areas can be pre-calibrated in the video footage captured by the camera. Each area can be represented by a polygonal pixel region, and the region number, region type, and region boundary coordinates are stored. After a fall event is generated, the intersection area calibration and state fusion module determines the fall location based on the detection frame coordinates. For example, the center point of the bottom edge of the detection frame can be used as the pedestrian's contact point with the ground, or the center point of the detection frame can be used as a target location reference point. When the center point of the bottom edge of the detection frame or the center point of the detection frame falls into a preset intersection area, that area is determined as the area corresponding to the fall location.

[0038] Furthermore, the multi-level alarm control module determines the alarm level based on the intersection fall event data. Specifically, when the fall event is generated, a level one on-site audible and visual alarm is triggered; when the intersection fall event data indicates that the fall location is in a pedestrian crossing area, curb area, or motor vehicle traffic conflict zone, or when the area corresponding to the fall location conflicts with the current motor vehicle traffic direction, a level two roadside display or broadcast prompt and a smart intersection platform alarm are triggered; when the fall event persists for a preset escalation time, a level three emergency terminal alarm is triggered.

[0039] When determining whether there is a traffic conflict between the area corresponding to the fall location and the current direction of vehicle traffic, the judgment can be made based on the area corresponding to the fall location and the traffic signal status. For example, if the fall location is in a pedestrian crossing area, and the direction of the pedestrian crossing is in a state where vehicles are allowed to pass or pedestrians are prohibited, a traffic conflict is determined to exist; if the fall location is in a vehicle traffic conflict area, a traffic conflict can be directly determined to exist; if the fall location is in a curb area and the curb area is adjacent to the area for vehicles to turn right, make a U-turn, or pull over, a traffic conflict can be determined based on a preset conflict area mapping relationship.

[0040] The preset conflict area mapping relationship includes the correspondence between area boundary data, area type and vehicle release direction; the multi-level alarm control module determines whether the area corresponding to the fall location is within the traffic conflict range corresponding to the current vehicle release direction based on the area corresponding to the fall location, the area boundary data and the current vehicle release direction.

[0041] Furthermore, the first-level on-site audible and visual alarm may include one or more of the following: buzzer alarm, voice broadcast alarm, and on-site warning light alarm; the second-level roadside display or broadcast prompt may include sending text or image prompts to roadside display screens or sending voice prompts to roadside broadcasting equipment; the smart intersection platform alarm may include sending pop-up alarms or event prompts to the smart intersection platform; and the third-level emergency terminal alarm may include sending alarm messages containing location information, time information, alarm level, and keyframe images to a preset management terminal or emergency response terminal.

[0042] Furthermore, the persistence of the fall event can be determined by the same suspected fall target still meeting the fall state within a continuous time period. Specifically, after generating the fall event, the system continues to track the same target based on the overlap of detection boxes or the distance between the center points of detection boxes in adjacent inference frames; if the same target still meets the fall confidence condition within a preset continuous judgment period, and the change in the position of the detection box is less than a preset displacement condition, then the fall event is determined to persist. If the same target no longer meets the fall confidence condition in subsequent video frames, or the same target leaves the detection area, then the fall event is determined to cease to persist.

[0043] In one implementation, after determining the alarm level, the multi-level alarm control module writes the alarm level into the intersection fall event data and provides the intersection fall event data with the alarm level written into it to the edge-cloud collaborative tracing module. The edge-cloud collaborative tracing module then sends the intersection fall event data to a cloud server. The intersection fall event data can be encapsulated as structured data, which may include event number, device identifier, keyframe image or keyframe image encoded data, detection box coordinates, fall confidence level, location information, time information, traffic signal status identifier, and alarm level.

[0044] In one specific embodiment, the edge device can compress the keyframe image into JPEG format, perform Base64 encoding, and then encapsulate it together with location information, time information, traffic signal status, fall confidence level, and alarm level into JSON data, which is then sent to the cloud server via HTTP or HTTPS. Upon receiving the intersection fall event data, the cloud server saves the keyframe image to a file storage space, writes the location information, time information, traffic signal status, fall confidence level, and alarm level into an event database, generates map annotations based on the location information, and generates a historical event list and image retrospective interface based on the event database.

[0045] In the above manner, the system can associate the visual detection results of pedestrian falls, the fall location, traffic signal status, alarm level, and cloud-based traceability data into complete intersection fall event data. It can not only achieve rapid identification of fall events and on-site alarms at the edge, but also trigger roadside prompts, platform alarms, and emergency terminal alarms based on intersection traffic risks, thereby realizing multi-level linkage alarms for traffic safety risks at smart intersections.

[0046] To verify the detection performance of the system described in this invention, tests were conducted in an intersection scenario, considering pedestrians crossing normally, moving quickly, bending over, squatting, falling, partial occlusion, and changes in lighting. During the tests, the camera captured a video stream with a resolution of 640×480 and a frame rate of 30fps. The lightweight fall detection model had an input size of 640×640, local inference was performed at the edge, and a continuous frame confirmation mechanism was used to generate fall events.

[0047] Test results show that when the fall confidence threshold is set to around 0.8, the system achieves a good balance between accuracy and false alarm rate. In intersection scenarios, the fall detection accuracy is approximately 92.5%, the false alarm rate is controlled below 4%, and the average response time from the occurrence of a fall to the triggering of an on-site alarm is less than 2 seconds. In low-light conditions, the detection confidence decreases slightly, but it remains above 75% when the camera's field of view is not completely obstructed.

[0048] The tests also showed that the continuous frame confirmation mechanism and the preset alarm cooling period can reduce false detections of a single frame and repeated alarms for the same fall event. In extreme cases such as when the camera is completely blocked or there is overexposure to strong light, false alarms or failure to detect may occur, but the system detection results can be restored to stability when the camera's field of view returns to normal.

[0049] Application examples Cameras and embedded edge computing devices are deployed near pedestrian crossings, curb areas, and conflict zones for motor vehicle traffic at smart intersections. The cameras continuously collect video footage from the intersection, while the embedded edge computing devices preprocess the video frames and use a lightweight fall detection model for local inference. When a suspected fall is detected, the system generates a fall event based on fall confidence, consistency of detection box positions in consecutive video frames, and preset continuous confirmation conditions, and uses the alarm trigger frame as the keyframe image.

[0050] After a fall event is generated, the system obtains the location information, time information and traffic signal status corresponding to the fall event, and determines whether the fall location is in a pedestrian crossing area, curb area or motor vehicle traffic conflict area, and whether the area corresponding to the fall location conflicts with the current motor vehicle traffic direction.

[0051] When a pedestrian falls in a regular waiting area, the system triggers a Level 1 on-site audible and visual alarm to alert nearby personnel to check promptly. When a pedestrian falls in a crosswalk or a conflict zone between pedestrian and vehicle traffic, or when the fall location conflicts with the current direction of vehicle traffic, the system triggers a Level 2 roadside display or broadcast alert and sends an alarm message to the smart intersection platform. When the fall event persists for a preset escalation time, the system further sends a Level 3 alarm message to a preset management terminal or emergency response terminal.

[0052] Simultaneously, the system reports keyframe images, location information, time information, traffic signal status, fall confidence level, and alarm level to the cloud server. The cloud server stores the fall event and generates map annotations, a historical event list, and an image retrospective interface, facilitating real-time handling and post-event tracing by intersection management personnel.

[0053] Through this application example, the present invention can realize automatic identification of pedestrian fall events, traffic risk correlation judgment, multi-level alarm triggering, and cloud-based traceability management in smart intersection scenarios, thereby improving the timeliness of detection and targeted handling of abnormal pedestrian events at intersections.

[0054] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are merely principles of the invention. Various changes and modifications can be made to the invention without departing from its spirit and scope, and all such changes and modifications fall within the scope of the claimed invention. The scope of protection claimed by the appended claims and their equivalents is defined.

Claims

1. A pedestrian fall visual perception and multi-level alarm system for smart intersections, characterized in that, include: The intersection edge visual perception module is used to collect surveillance videos of smart intersections, preprocess video frames, and call a lightweight fall detection model on an embedded edge computing device to output the coordinates of the detection box of the suspected fall target and the fall confidence score. The fall event confirmation module is used to generate fall events based on fall confidence, the consistency of detection box positions in consecutive video frames, and preset continuous confirmation conditions. It uses alarm trigger frames as key frame images and suppresses repeated alarm triggering of the same fall event within a preset alarm cooling period. The intersection area calibration and status fusion module is used to perform semantic area calibration of the intersection in the camera's field of view, generate area boundary data including pedestrian crossing area, curb area and motor vehicle traffic conflict area, and map the detection box coordinates to the corresponding intersection area to determine the fall location. At the same time, it acquires the location information, time information and traffic signal status of the fall event, and generates intersection fall event data. The multi-level alarm control module is used to determine the alarm level based on intersection fall event data. When a fall event is generated, a level 1 on-site audible and visual alarm is triggered. When the fall location is in a pedestrian crossing area, curb area, or motor vehicle traffic conflict zone, or when a traffic conflict is determined based on the mapping relationship between the boundary data of the area and the current motor vehicle traffic direction, a level 2 roadside prompt and platform alarm are triggered. When the fall event continues for a preset escalation time, a level 3 emergency terminal alarm is triggered. The edge-cloud collaborative tracing module is used to report intersection fall event data to the cloud server, which stores and generates map annotations, historical event displays, and image retrospectives.

2. The pedestrian fall visual perception and multi-level alarm system for smart intersections according to claim 1, characterized in that, In the intersection edge visual perception module, the video frame preprocessing includes: The video frames are scaled, pixel normalized, and converted to tensor format, and the preprocessed video frames are then input into the lightweight fall detection model. The detection box coordinates output by the lightweight fall detection model include the center point coordinates of the suspected fall target, the width of the detection box, and the height of the detection box. The fall confidence score is used to characterize the probability that the suspected fall target is in a fall posture.

3. A pedestrian fall visual perception and multi-level alarm system for smart intersections according to claim 2, characterized in that, The lightweight fall detection model is a lightweight target detection model or a human posture detection model. The lightweight target detection model includes the YOLOv8n model, the YOLOv5s model, or a lightweight improved YOLO model. The lightweight fall detection model is deployed on the embedded edge computing device in ONNX format or a model format adapted to the embedded inference framework, and the embedded edge computing device performs local inference.

4. A pedestrian fall visual perception and multi-level alarm system for smart intersections according to claim 1, characterized in that, The fall event confirmation module includes the following process for generating a fall event based on fall confidence, consistency of detection box positions in consecutive video frames, and preset continuous confirmation conditions: The suspected fall targets are screened based on the fall confidence level, and non-maximum suppression processing is performed on the screened detection boxes. The consistency of the position of the same target in consecutive video frames is determined by the overlap (IOU) between detection boxes in adjacent video frames being greater than a preset threshold or the distance between the center points of the detection boxes being less than a preset distance threshold. The fall event is generated when the same suspected fall target meets the fall confidence condition and the detection box position consistency condition within a consecutive preset number of frames.

5. A pedestrian fall visual perception and multi-level alarm system for smart intersections according to claim 1, characterized in that, During the process of the fall event confirmation module generating a fall event based on the positional consistency of the detection box coordinates in consecutive video frames, the intersection edge visual perception module obtains the detection box coordinates and fall confidence of the suspected fall target according to a preset frame skipping detection strategy. The preset frame skipping detection strategy includes: The lightweight fall detection model is invoked for inference every preset number of frames. The detection box coordinates and fall confidence scores output from the previous inference frame are used between adjacent inference frames; During the preset alarm cooling-off period, no new fall events will be output to the multi-level alarm control module for the same fall event.

6. A pedestrian fall visual perception and multi-level alarm system for smart intersections according to claim 1, characterized in that, In the intersection area calibration and state fusion module, the methods for obtaining the location information of the fall event include: The latitude and longitude coordinates corresponding to the fall event can be obtained through the positioning module, or the location of the preset intersection can be obtained through the device identifier; The positioning module outputs positioning data conforming to the NMEA-0183 format and generates valid latitude and longitude coordinates when the positioning quality meets the preset valid conditions. When latitude and longitude coordinates are used, coordinate system transformation is performed to generate location information for map labeling.

7. A pedestrian fall visual perception and multi-level alarm system for smart intersections according to claim 1, characterized in that, In the intersection area calibration and state fusion module, the intersection fall event data includes: Event number, device identifier, detection frame coordinates, keyframe image, fall confidence level, location information, time information, and traffic signal status identifier; After determining the alarm level, the multi-level alarm control module writes the alarm level into the intersection fall event data and provides the intersection fall event data with the alarm level written into it to the end-to-cloud collaborative tracing module.

8. A pedestrian fall visual perception and multi-level alarm system for smart intersections according to claim 1, characterized in that, In the multi-level alarm control module, after determining the alarm level based on the intersection fall event data, the alarm actions triggered according to the alarm level include: The Level 1 on-site audible and visual alarm includes one or more of the following: buzzer alarm, voice broadcast alarm, and on-site warning light alarm. The secondary roadside display or broadcast prompts include sending text or image prompts to roadside display screens or sending voice prompts to roadside broadcasting equipment; The smart intersection platform alarm includes sending pop-up alarms or event notifications to the smart intersection platform; The Level 3 emergency terminal alarm includes sending an alarm message containing the location information, the time information, the alarm level, and the keyframe image to a preset management terminal or emergency response terminal.

9. A pedestrian fall visual perception and multi-level alarm system for smart intersections according to claim 1, characterized in that, The traffic signal status is obtained from the traffic light recognition results of the smart intersection signal control equipment, the smart intersection management platform interface, or video images. When the area corresponding to the fall location in the intersection fall event data is a pedestrian crossing, and the current traffic signal status indicates that the vehicle passage direction corresponding to the pedestrian crossing is in a free state, or when the area corresponding to the fall location is a vehicle passage conflict zone, the multi-level alarm control module determines that there is a passage conflict between the area corresponding to the fall location and the current vehicle passage direction, and determines the alarm level to be no less than level two, triggering the level two roadside display or broadcast prompt and the smart intersection platform alarm.

10. A pedestrian fall visual perception and multi-level alarm system for smart intersections according to claim 1, characterized in that, In the edge-cloud collaborative tracing module, after the intersection fall event data is sent to the cloud server, the cloud server saves the key frame image to the file storage space, writes the location information, time information, traffic signal status, fall confidence level and the alarm level into the event database, generates the map label based on the location information, and generates a historical event list and image retrospective interface based on the event database.

Images