Pedestrian safety warning method, device and equipment and storage medium

Abstract

This invention relates to the field of image recognition and discloses a pedestrian safety early warning method, device, equipment, and storage medium. The method detects the behavior of people looking down at their phones and issues warnings in a tiered manner. The method includes: acquiring multiple consecutive frames of image data of a monitored area for a first preset duration, inputting them into a pre-trained AI posture recognition model, obtaining the behavior confidence score of each pedestrian in each frame of the image, and for each pedestrian, if the behavior confidence score continuously exceeds a preset threshold in all images for a second preset duration, then determining that the pedestrian is a target pedestrian. The method also acquires the total number of people in the monitored area and the distance between each target pedestrian and the hazard source to determine the pedestrian density coefficient and the distance weight of each target pedestrian. Combined with the average behavior confidence score of the target pedestrians, a weighted fusion algorithm is used to calculate the comprehensive risk value and determine the risk level. Finally, a preset early warning strategy is matched to generate an early warning instruction corresponding to each target pedestrian, and the early warning instruction is executed sequentially from high to low according to the comprehensive risk value.

Description

Technical Field

[0001] This invention relates to the field of image recognition, and more particularly to a pedestrian safety early warning method, apparatus, device, and storage medium. Background Technology

[0002] With the widespread use of smartphones, the phenomenon of "phubbing" has become a major public safety hazard. In key areas with high pedestrian traffic, such as school stairwells, crosswalks, and subway station stairwells, pedestrians, focused on their phone screens, often ignore their surroundings, greatly increasing the risk of tripping, falling, collisions, and even traffic accidents. Existing solutions to this problem mainly fall into three categories, but all have significant limitations.

[0003] The first category is static warning signs, such as "Watch out for steps" or "No phone calls" signs or ground stickers. These are passive reminders, heavily reliant on pedestrians' active observation. For those constantly looking down at screens, static signs are easily overlooked and cannot provide effective intervention. Additionally, convex mirrors that reveal blind spots at corners also rely on pedestrians' active observation and are easily soiled.

[0004] The second type is traditional electronic sensor alarms, such as voice prompts based on infrared (PIR) or ultrasonic sensors. When the sensor detects a moving object entering a preset area, it triggers a speaker to play a fixed voice prompt. However, this approach has a high false alarm rate and cannot distinguish between specific pedestrian behaviors (such as walking normally, bending down to tie shoelaces, or looking down at a mobile phone). It triggers the alarm whenever an object passes by, easily creating a "boy who cried wolf" effect and reducing the credibility of the warning. Furthermore, this approach lacks differentiated warnings, applying the same alert to all passing pedestrians, failing to target high-risk behaviors like looking down at a mobile phone. In addition, continuous or frequent voice prompts can cause noise pollution, making it unsuitable, especially in quiet places like hospitals and schools.

[0005] The third category is commercial AI video analytics systems, such as server-side AI behavior analysis platforms provided by large security companies. These typically capture video streams through high-definition network cameras, upload them to backend servers or the cloud, and utilize powerful GPUs for complex deep learning model inference to identify dangerous behaviors. This solution is costly; system deployment requires high-performance servers, expensive software licenses, and professional maintenance, with the cost per location often reaching tens of thousands of yuan, making large-scale deployment impossible in budget-constrained scenarios such as campuses or small public places. Furthermore, the need to upload video data to the cloud or server for processing poses a risk of personal privacy leaks. In addition, it has high requirements for network bandwidth and power supply facilities, limiting its installation and deployment.

[0006] Therefore, existing technologies still need improvement and development. Summary of the Invention

[0007] This invention provides a pedestrian safety early warning method, device, equipment, and storage medium, which are used to detect the behavior of people looking down at their phones and issue warnings in a tiered manner.

[0008] The first aspect of this invention provides a pedestrian safety early warning method, comprising: acquiring continuous multi-frame image data of a monitored area for a first preset duration; inputting each frame of the continuous multi-frame image data into a pre-trained AI posture recognition model to obtain the behavior confidence of each pedestrian in each frame; for each pedestrian, if a continuous second preset duration exists within the first preset duration, and the behavior confidence of the pedestrian is greater than a preset confidence threshold in all frames within the continuous second preset duration, then determining that the pedestrian is a target pedestrian, wherein the second preset duration is less than the first preset duration; when a target pedestrian exists, acquiring the total number of pedestrians in the monitored area and the distance between each target pedestrian and a hazard source; matching the total number of pedestrians and a preset number of pedestrians with a pedestrian flow density coefficient correlation table to determine the corresponding pedestrian flow density coefficient, and setting the distance between each target pedestrian and a hazard source. The system matches the distance with a preset distance and distance weight relationship table to determine the distance weight of each target pedestrian; it calculates the average behavioral confidence of each target pedestrian in multiple consecutive frames of image data, and based on the average behavioral confidence of each target pedestrian in multiple consecutive frames of image data, the pedestrian density coefficient, and the distance weight corresponding to each target pedestrian, it calculates the comprehensive risk value of each target pedestrian through a preset weighted fusion algorithm; it obtains the range of each risk level, matches the comprehensive risk value of each target pedestrian with the range of each risk level to determine the risk level of each target pedestrian; based on the risk level of each target pedestrian, it matches the risk warning strategy corresponding to the risk level of each target pedestrian from a preset risk warning strategy library, and generates a warning instruction corresponding to each target pedestrian based on the risk warning strategy, and executes each warning instruction in descending order of the comprehensive risk value of each target pedestrian.

[0009] Preferably, the step of acquiring continuous multi-frame image data of a first preset duration in the monitored area, inputting each frame of the continuous multi-frame image data into a pre-trained AI posture recognition model to obtain the behavior confidence of each pedestrian in each frame, and for each pedestrian, if there is a continuous second preset duration within the first preset duration, and the behavior confidence of the pedestrian is greater than a preset confidence threshold in all frames within the continuous second preset duration, then the pedestrian is determined to be the target pedestrian, wherein the second preset duration is less than the first preset duration, includes: collecting video image data in the target application scenario, constructing a dataset based on the video image data in the target application scenario, the dataset including multiple training samples, the training samples including categories of looking down at a mobile phone and normal walking, and the training samples having head, neck, wrist, and hand. Arm key point annotation; a pre-trained model is built based on YOLO, and the pre-trained model is trained using a transfer learning strategy and the dataset to obtain an AI posture recognition model; a low-power passive infrared sensor is used to continuously detect whether there is a moving human body in the monitoring area; when it is determined that there is a moving human body in the monitoring area, a series of consecutive frames of image data for a first preset duration are acquired in the monitoring area, and each frame of the consecutive frames of image data is input into the AI ​​posture recognition model to obtain the behavior confidence of each pedestrian in each frame. For each pedestrian, if there is a consecutive second preset duration within the first preset duration, and the behavior confidence of the pedestrian is greater than a preset confidence threshold in all frames within the consecutive second preset duration, then the pedestrian is determined to be the target pedestrian, and the second preset duration is less than the first preset duration.

[0010] Preferably, when a target pedestrian is present, obtaining the total number of pedestrians in the monitoring area and the distance between each target pedestrian and the hazard source includes: when a target pedestrian is present, obtaining a real-time panoramic image of the monitoring area and the distance between each target pedestrian and the hazard source; and statistically determining the total number of pedestrians in the monitoring area based on the real-time panoramic image.

[0011] Preferably, the calculation of the average behavioral confidence of each target pedestrian in consecutive multi-frame image data, and the calculation of the comprehensive risk value of each target pedestrian based on the average behavioral confidence of each target pedestrian in consecutive multi-frame image data, the pedestrian density coefficient, and the distance weight corresponding to each target pedestrian, using a preset weighted fusion algorithm, includes: calculating the comprehensive risk value of each target pedestrian using a preset comprehensive risk calculation function. The comprehensive risk calculation function is expressed as:

[0012] In the formula, This represents the overall risk value. This represents the average behavioral confidence of the target pedestrian across multiple consecutive frames of image data. This represents the population density coefficient. Indicates distance weight, , , These are the preset weighting coefficients.

[0013] Preferably, the step of obtaining the range of each risk level and matching the comprehensive risk value of each target pedestrian with the range of each risk level to determine the risk level of each target pedestrian includes: obtaining the range of each risk level and matching the comprehensive risk value of each target pedestrian with the range of each risk level; when the comprehensive risk value of the target pedestrian is less than a first threshold, the risk level of the target pedestrian is determined to be low risk; when the comprehensive risk value of the target pedestrian is greater than or equal to the first threshold and less than a second threshold, the risk level of the target pedestrian is determined to be medium risk; and when the comprehensive risk value of the target pedestrian is greater than or equal to the second threshold, the risk level of the target pedestrian is determined to be high risk.

[0014] Preferably, the step of matching a risk warning strategy corresponding to the risk level of each target pedestrian from a preset risk warning strategy library based on the risk level of each target pedestrian, generating a warning instruction corresponding to each target pedestrian based on the risk warning strategy, and executing each warning instruction sequentially according to the comprehensive risk value of each target pedestrian from high to low, includes: when the risk level of the target pedestrian is low risk, matching a first-level risk warning strategy from the preset risk warning strategy library, and generating a first-level warning instruction based on the first-level risk warning strategy, wherein the first-level warning instruction is used to control the RGB projection light to project a prompt pattern with a first color and a first flashing frequency on the ground; when the risk level of the target pedestrian is medium risk, matching a second-level risk warning strategy from the preset risk warning strategy library, and... A second-level warning command is generated based on the second-level risk warning strategy. This command controls an RGB projector to project a warning pattern with a second color and a second flashing frequency on the ground, where the second flashing frequency is higher than the first flashing frequency. When the risk level of the target pedestrian is high, a third-level risk warning strategy is matched from a preset risk warning strategy library, and a third-level warning command is generated based on this strategy. This command controls an RGB projector to project an emergency warning pattern with a third color and a third flashing frequency on the ground, and controls a directional speaker to play a preset warning voice message, where the third flashing frequency is higher than the second flashing frequency. The warning commands are executed sequentially according to the overall risk value of each target pedestrian, from highest to lowest.

[0015] Preferably, the pedestrian safety warning method further includes: acquiring the relative distance and relative speed of oncoming pedestrians in real time, and calculating the collision time based on the relative distance and relative speed of oncoming pedestrians; if the collision time is lower than a preset safety time threshold, the risk level of the oncoming pedestrian is regarded as high risk, and an emergency warning strategy is matched from a preset risk warning strategy library, and an emergency warning command is generated based on the emergency warning strategy to immediately execute the warning, wherein the emergency warning command is used to control the RGB projection lamp to project an emergency warning pattern with a fourth color and a fourth flashing frequency on the ground, and to control the directional speaker to play a preset warning voice, wherein the fourth flashing frequency is higher than or equal to the third flashing frequency.

[0016] The second aspect of the present invention provides a pedestrian safety early warning device, comprising: a judgment module, configured to acquire continuous multi-frame image data of a monitoring area for a first preset duration, input each frame of the continuous multi-frame image data into a pre-trained AI posture recognition model to obtain the behavior confidence of each pedestrian in each frame, and for each pedestrian, if a continuous second preset duration exists within the first preset duration, and the behavior confidence of the pedestrian is greater than a preset confidence threshold in all frames within the continuous second preset duration, then the pedestrian is determined to be a target pedestrian, wherein the second preset duration is less than the first preset duration; an acquisition module, configured to acquire the number of all pedestrians in the monitoring area and the distance between each target pedestrian and a hazard source when a target pedestrian exists; and a matching module, configured to match the number of all pedestrians with a preset number of pedestrians with a pedestrian flow density coefficient correlation table to determine the corresponding pedestrian flow density coefficient, and match the distance between each target pedestrian and a hazard source with a preset distance and a... The system employs a distance weighting relationship table to determine the distance weight of each target pedestrian. A calculation module calculates the average behavioral confidence of each target pedestrian across multiple consecutive frames of image data. Based on this confidence, the pedestrian density coefficient, and the distance weight corresponding to each target pedestrian, a pre-defined weighted fusion algorithm is used to calculate the comprehensive risk value of each target pedestrian. A risk determination module acquires the range of each risk level, matches the comprehensive risk value of each target pedestrian with the range of each risk level, and determines the risk level of each target pedestrian. A risk warning module matches a risk warning strategy corresponding to the risk level of each target pedestrian from a pre-defined risk warning strategy library, generates a warning instruction corresponding to each target pedestrian based on the risk warning strategy, and executes the warning instructions sequentially according to the comprehensive risk value of each target pedestrian from highest to lowest.

[0017] A third aspect of the present invention provides a pedestrian safety warning device, comprising: a memory and at least one processor, wherein the memory stores computer-readable instructions, and the memory and the at least one processor are interconnected via a circuit; the at least one processor invokes the computer-readable instructions in the memory to cause the pedestrian safety warning device to perform the various steps of the pedestrian safety warning method described above.

[0018] A fourth aspect of the present invention provides a computer-readable storage medium storing computer-readable instructions that, when executed on a computer, cause the computer to perform the steps of the pedestrian safety warning method described above.

[0019] The technical solution provided by this invention accurately identifies pedestrians who are continuously looking down at their phones by using multi-frame image continuous detection and an AI posture recognition model, effectively avoiding misjudgment in a single frame. Simultaneously, it integrates pedestrian density coefficients and distance weights, dynamically assessing the comprehensive risk value of each target pedestrian based on a weighted algorithm. This achieves multi-dimensional risk quantification from behavior, environment, and location, and matches differentiated early warning strategies according to risk levels. Early warning instructions are executed sequentially from high to low comprehensive risk values, ensuring that the most dangerous pedestrians receive priority intervention while avoiding resource conflicts and mutual interference when multiple targets receive parallel early warnings. This improves the accuracy and effectiveness of early warnings while effectively reducing the probability of pedestrian safety accidents. Attached Figure Description

[0020] Figure 1 A flowchart of a pedestrian safety early warning method provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of the pedestrian safety warning device provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of the pedestrian safety early warning device provided in an embodiment of the present invention. Detailed Implementation

[0021] The terms "first," "second," "third," "fourth," etc. (if present) in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" or "having" and any variations thereof are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0022] For ease of understanding, the specific process of the embodiments of the present invention is described below. Please refer to [link / reference]. Figure 1 An embodiment of the present invention provides a pedestrian safety early warning method comprising: S101. Acquire continuous multi-frame image data of the monitoring area for a first preset duration, input each frame of the continuous multi-frame image data into a pre-trained AI posture recognition model, obtain the behavior confidence of each pedestrian in each frame image, and for each pedestrian, if there is a continuous second preset duration within the first preset duration, and the behavior confidence of the pedestrian is greater than a preset confidence threshold in all frames within the continuous second preset duration, then determine that the pedestrian is the target pedestrian, and the second preset duration is less than the first preset duration; S102. When a target pedestrian is present, obtain the total number of pedestrians in the monitored area and the distance between each target pedestrian and the hazard source. S103. Match the total number of pedestrians with the preset number of pedestrians and the relationship table between pedestrian flow density coefficient to determine the corresponding pedestrian flow density coefficient, and match the distance between each target pedestrian and the hazard source with the preset distance and distance weight relationship table to determine the distance weight of each target pedestrian. S104. Calculate the average behavioral confidence of each target pedestrian in consecutive multi-frame image data, and based on the average behavioral confidence of each target pedestrian in consecutive multi-frame image data, the pedestrian density coefficient, and the distance weight corresponding to each target pedestrian, calculate the comprehensive risk value of each target pedestrian through a preset weighted fusion algorithm. S105. Obtain the range of each risk level, match the comprehensive risk value of each target pedestrian with the range of each risk level, and determine the risk level of each target pedestrian. S106. Based on the risk level of each target pedestrian, match the risk warning strategy corresponding to the risk level of each target pedestrian from the preset risk warning strategy library, generate the warning instruction corresponding to each target pedestrian based on the risk warning strategy, and execute each warning instruction in sequence according to the comprehensive risk value of each target pedestrian from high to low.

[0023] It is understood that the executing entity of this invention can be a pedestrian safety early warning method device, a terminal, or a server; no specific limitation is made here. This embodiment of the invention will be described using a server as an example.

[0024] In this embodiment, the hazard source can be dangerous areas such as stairwells, zebra crossings, and platform edges, or pedestrians walking / running towards the target pedestrian who may be at risk of collision, or pedestrians in blind spots at corners.

[0025] Understandably, assuming only one projection device and one directional speaker are installed in the monitored area, when two or more pedestrians have the same overall risk value, warnings are issued sequentially according to their distance from the hazard source, from closest to furthest. If the distances between pedestrians and the hazard source are still the same, warnings are issued in any order. If multiple projection devices and multiple directional speakers are installed in the monitored area, warnings are issued simultaneously to pedestrians with the same overall risk value.

[0026] This embodiment provides a pedestrian safety early warning method, which accurately filters out target pedestrians who are constantly looking down at their mobile phones through continuous multi-frame image detection and AI posture recognition model, effectively avoiding single-frame misjudgment. At the same time, it integrates pedestrian flow density coefficient and distance weight, and dynamically evaluates the comprehensive risk value of each target pedestrian based on a weighted algorithm, realizing multi-dimensional risk quantification from behavior, environment to location. Furthermore, it matches differentiated early warning strategies according to risk level, and executes early warning instructions in order from high to low comprehensive risk value. This ensures that the most dangerous pedestrians receive priority intervention, and avoids resource conflicts and mutual interference when multiple targets are warned in parallel. Thus, while improving the accuracy and effectiveness of early warning, it effectively reduces the probability of pedestrian safety accidents.

[0027] In this embodiment, step S101 involves acquiring continuous multi-frame image data of a first preset duration in the monitored area, inputting each frame of the continuous multi-frame image data into a pre-trained AI posture recognition model to obtain the behavior confidence of each pedestrian in each frame. For each pedestrian, if a continuous second preset duration exists within the first preset duration, and the behavior confidence of the pedestrian is greater than a preset confidence threshold in all frames within the continuous second preset duration, then the pedestrian is determined to be the target pedestrian. The second preset duration is less than the first preset duration. This includes: collecting video image data in the target application scenario, constructing a dataset based on the video image data in the target application scenario, the dataset including multiple training samples, the training samples including categories of looking down at a mobile phone and normal walking, and the training samples having head, neck, and Wrist and arm key points are labeled; a pre-trained model is built based on YOLO, and the pre-trained model is trained with the dataset in combination with transfer learning strategy to obtain an AI posture recognition model; a low-power passive infrared sensor is used to continuously detect whether there is a moving human body in the monitoring area; when it is determined that there is a moving human body in the monitoring area, a series of consecutive frames of image data of the monitoring area for a first preset duration are acquired, and each frame of the consecutive frames of image data is input into the AI ​​posture recognition model to obtain the behavior confidence of each pedestrian in each frame. For each pedestrian, if there is a consecutive second preset duration within the first preset duration, and the behavior confidence of the pedestrian is greater than a preset confidence threshold in all frames within the consecutive second preset duration, then the pedestrian is determined to be the target pedestrian, and the second preset duration is less than the first preset duration.

[0028] In this embodiment, video image data was first collected in three typical application scenarios: campus staircases, zebra crossings at intersections, and subway station stairwells. This covered various lighting conditions, including daytime, nighttime, backlighting, and strong light, and included multiple states such as single person, multiple people, occlusion, and walking, ensuring sample diversity and scene adaptability. Subsequently, the MakeSense annotation tool was used to manually annotate the images. Annotations were divided into two categories: category annotation (differentiating between looking down at a phone and normal walking) and key point annotation (annotating the coordinates of the head, neck, and wrists of each pedestrian), resulting in a total of 23,000 effectively annotated images. Key point annotation is the core of improving recognition accuracy. The model can accurately distinguish between looking down at a phone and other behaviors by the head tilt angle, neck flexion, and wrist holding position. For example, a head tilt >30° and wrists held in front of the chest indicate looking down at a phone, while upright walking and wrists hanging naturally indicate normal walking. This dataset closely matches real-world application scenarios, avoiding the problem of general datasets being out of touch with actual scenarios, providing high-quality samples for model training, and improving recognition accuracy from the source.

[0029] In this embodiment, the model training and inference execution stages employ a transfer learning strategy to reduce training costs, improve model scene adaptability, and achieve accurate behavior confidence output. First, a pre-trained model is built based on YOLO, which has already learned general visual features. Then, a transfer learning strategy is used to fine-tune the training on the dataset, freezing the bottom feature extraction layer and only fine-tuning the top classification and regression layers. This significantly reduces the number of training iterations. On a server equipped with an entry-level GPU, the model can complete fine-tuning convergence within 1-2 hours. The trained model can then be deployed to edge devices for inference. The trained AI pose recognition model can output the bounding box of the target pedestrian and the behavior confidence score. The behavior confidence score is the core output, ranging from 0 to 1, representing the probability that the pedestrian is looking down at a mobile phone. During inference, the visual image data of the target pedestrian is input into the AI ​​pose recognition model. The AI ​​pose recognition model analyzes the key point coordinates and pose features to quickly output the results.

[0030] In this embodiment, an ultra-low-power passive infrared sensor (HC-SR501) is used to continuously monitor the monitored area, which is the core guarantee for the system's low-power operation. The passive infrared sensor does not require an external high-frequency power supply; it determines whether a moving person has entered the area by detecting changes in infrared radiation emitted by a human body. Its operating current is only in the microamplitude range, and its standby power consumption is far lower than that of cameras and ultrasonic sensors, allowing for long-term stable operation without consuming large amounts of electricity. The sensor covers the entire monitored area, has adjustable sensitivity, and can filter out non-human interference such as pets and debris, responding only to moving human bodies to avoid false triggers. When no pedestrians enter, only the infrared sensor remains operational, while the camera and ultrasonic sensor are in a low-power sleep state, reducing device power consumption by more than 90%. When a pedestrian enters the monitored area, the passive infrared sensor detects a sudden change in infrared radiation and immediately outputs a high-level trigger signal, which is transmitted to the edge computing controller to initiate the subsequent device wake-up process.

[0031] In this embodiment, the camera is activated by a trigger signal from a passive infrared sensor. Once activated, the camera acquires multiple consecutive frames of image data within the monitored area in real time at 1080P resolution for AI pose recognition.

[0032] Understandably, the first preset duration and the second preset duration are set according to the actual situation. In this embodiment, the first preset duration is 5s and the second preset duration is 2s.

[0033] In this embodiment, each frame of the multi-frame image data captured by the camera within 5 consecutive seconds is input into a pre-trained AI pose recognition model. The AI ​​pose recognition model can output a behavior confidence score for each pedestrian in the image. That is, the probability value of the pedestrian's current posture being looking down at a mobile phone, with a value ranging from 0 to 1. For each frame of the image, the AI ​​pose recognition model will output the bounding boxes of all pedestrians in that frame and their corresponding behavior confidence scores.

[0034] Since the same pedestrian may appear in close proximity across multiple consecutive frames, the identity of the same pedestrian in different frames is associated using the Intersection over Union (IoU) of bounding boxes or target tracking algorithms (such as Simple Online Real-Time Tracking, SORT), ensuring that subsequent comparisons are of the same pedestrian's behavior confidence sequence. For each successfully associated pedestrian, their behavior confidence value across all frames is extracted. When the behavior confidence of the pedestrian in all frames for any consecutive 2 seconds is greater than a preset confidence threshold, the pedestrian is identified as a target pedestrian. In other words, it is not necessary for the pedestrian to maintain a head-down behavior throughout the entire first preset duration; as long as the head-down behavior can be sustained for a short but sufficiently stable period (e.g., 2 seconds), it is considered to pose a substantial safety risk, triggering subsequent risk assessment and warning procedures. If a pedestrian does not meet the above confidence condition for any consecutive second preset duration within the entire first preset duration (i.e., the head-down behavior is intermittent, or the head-down duration is less than the second preset duration), they are excluded from the target pedestrian list, and subsequent risk assessment and warning procedures are not triggered.

[0035] Understandably, the preset reliability threshold is set according to the actual situation, and the preset reliability threshold is 0.7.

[0036] In this embodiment, step S102, when a target pedestrian is present, obtains the total number of pedestrians in the monitoring area and the distance between each target pedestrian and the hazard source, including: when a target pedestrian is present, obtaining a real-time panoramic image of the monitoring area and the distance between each target pedestrian and the hazard source; and statistically obtaining the total number of pedestrians in the monitoring area based on the real-time panoramic image.

[0037] In this embodiment, the hazard source includes at least one of the following types: static hazard area, dynamic risk pedestrian, and blind spot risk pedestrian. A static hazard area refers to a physical area with a fixed location, such as stairwells, zebra crossings, or platform edges, where a pedestrian's misstep or intrusion could cause a fall or collision. A dynamic risk pedestrian refers to other pedestrians walking or running towards the target pedestrian, posing a risk of head-on collision. A blind spot risk pedestrian refers to other pedestrians located in visual blind spots, such as corners, behind pillars, or behind obstructions, and not directly observable by the target pedestrian. When the hazard source is a static hazard area, the straight-line distance from the target pedestrian to the edge of the area is measured using a depth camera or lidar. When the hazard source is a dynamic risk pedestrian, the relative distance and relative speed between the target pedestrian and the dynamic risk pedestrian are measured in real time using a visual target tracking algorithm or millimeter-wave radar. When the hazard source is a blind spot risk pedestrian, the relative distance between the target pedestrian and the blind spot pedestrian is estimated using auxiliary sensors (ultrasound, radar, or reflectors) placed at corners.

[0038] Specifically, when the hazard source is a static hazardous area, the straight-line distance between the target pedestrian and the edge of the static hazardous area is measured using a depth camera (such as a binocular camera or structured light camera) or a lidar array. The depth camera outputs an image containing depth information, which, combined with visual target detection and depth map matching, yields the three-dimensional position coordinates of the target pedestrian, and then calculates the Euclidean distance between the pedestrian and the preset boundary of the static hazardous area. The lidar array generates point cloud data of the monitored area through multi-beam scanning, achieving precise positioning and ranging of the target pedestrian.

[0039] When the hazard source is a dynamically risky pedestrian, the relative distance and relative velocity between the target pedestrian and the dynamically risky pedestrian are measured in real time using visual target tracking algorithms or millimeter-wave radar. Specifically, multi-target tracking algorithms such as SORT (Simple Online and Realtime Tracking) or DeepSORT are used to identify and track pedestrians in multiple consecutive frames of images. Based on camera calibration parameters, the pedestrian's foot coordinates in the image coordinate system are mapped to the world coordinate system, and the Euclidean distance between the two pedestrians is calculated as the relative distance. The relative velocity is obtained by differential smoothing the distance changes in multiple consecutive frames. Alternatively, a 60GHz or 77GHz millimeter-wave radar is used to directly measure the relative radial distance and radial velocity between the target pedestrian and the dynamically risky pedestrian by transmitting frequency-modulated continuous waves and receiving the echoes. The measured relative distance can be directly used for distance weight matching, while the relative velocity can be used to calculate the time to collision (TTC). When the TTC is lower than a preset safe time threshold (e.g., 2 seconds), an emergency warning is triggered.

[0040] When the hazard source is a pedestrian in a blind spot, auxiliary sensors placed at the corner estimate the relative distance between the target pedestrian and the pedestrian in the blind spot. These auxiliary sensors include ultrasonic sensors, millimeter-wave radar, or passive infrared sensors, installed on both sides of the corner with their emission direction along the outer side of the corner. When a pedestrian is present on the other side of the corner, the sensor can detect the echo reflected from the human body and output distance information. Alternatively, reflectors or prisms can be installed at the corner, working with a camera to identify pedestrians in the blind spot and estimate their distance through image analysis. The weighting matching rule for this distance is the same as for static hazard areas, used to quantify the risk level of a blind spot collision.

[0041] Distances between pedestrians and hazards can be measured using depth cameras or lidar arrays, or by fusing multiple ultrasonic sensors for localization. Depth cameras (such as binocular or structured light cameras) output images containing depth information. By matching visual target detection with depth maps, the 3D position of each pedestrian can be simultaneously acquired, allowing for the calculation of their straight-line distance to the hazard. Lidar arrays, through multi-beam scanning, can generate point cloud data of the monitored area, enabling precise localization and distance measurement of multiple pedestrians. If a multi-ultrasonic sensor fusion scheme is used, distances to different pedestrians can be distinguished using triangulation or time-of-arrival algorithms; the choice can be flexible based on the scenario's cost and accuracy requirements.

[0042] Simultaneously, the depth camera can also capture real-time panoramic images of the monitored area. A visual object detection algorithm processes these images to accurately count the number of pedestrians within the monitored area. This algorithm, based on the lightweight YOLO model, runs locally on edge computing devices (such as Orange Pie), eliminating the need to upload to the cloud. With a processing speed exceeding 15 frames per second, it can accurately count the number of people in the area in real time. The visual object detection algorithm performs bounding box detection on pedestrians, filtering out non-human targets such as trees, vehicles, and debris, counting only valid pedestrians with a statistical accuracy exceeding 98%. It can accurately count even in scenarios with overlapping pedestrians or partial occlusion. For example, if there are 6 pedestrians in the monitored area, the visual object detection algorithm can accurately identify and output a count of 6, which is directly used for subsequent pedestrian density coefficient matching.

[0043] In this embodiment, a passive infrared sensor is installed above or to the side of the monitored area, horizontally covering the entire passageway, for continuous detection of moving human bodies. A depth camera is installed at the center above the monitored area, providing a top-down view to capture a complete panoramic view of the monitored area and close-ups of pedestrian postures. A lidar array or ultrasonic distance sensor is installed at the edge of the hazard source (such as beside stairs or platforms), with its emission direction directed towards pedestrians within the monitored area, accurately measuring the straight-line distance between pedestrians and the hazard source.

[0044] In this embodiment, in step S103, a preset correlation table between the number of pedestrians and the pedestrian density coefficient is used to convert the real-time number of people in the monitoring area into a density coefficient that can be used for risk calculation, thereby achieving accurate quantification of environmental congestion risk. This correlation table is based on the statistical analysis of the probability of accidents caused by people looking down at their phones in public places, and adopts a three-segment rule to fully consider the amplification effect of pedestrian density on risk. When the total number of pedestrians is ≤3, the scene is sparse, the pedestrian activity space is sufficient, and the secondary risks such as collisions and trampling are extremely low. Therefore, the pedestrian density coefficient is set to 1.0, and the overall risk is not amplified. When the total number of pedestrians is 4-8, the scene is generally congested. The distance between pedestrians is shortened, and the probability of collisions while looking down increases. Therefore, the coefficient is set to 1.1, slightly amplifying the overall risk. When the total number of pedestrians is ≥9, the scene is highly congested. Pedestrians are densely distributed, and once looking down occurs, it is very easy to trigger chain accidents such as trampling and collisions, and the risk is significantly increased. Therefore, the coefficient is set to 1.2, significantly amplifying the overall risk.

[0045] For example, if the real-time count of all pedestrians in the monitored area is 10, meeting the segmentation condition of ≥9 pedestrians, the matching pedestrian density coefficient is 1.2; if the count is 5, the coefficient is 1.1; and if the count is 2, the coefficient is 1.0. This lookup and assignment method requires no complex mathematical calculations, has extremely high computational efficiency, and can be completed quickly on edge computing devices, ensuring system real-time performance. Furthermore, the coefficient values ​​have been calibrated through real-world testing in three scenarios: campuses, intersections, and subway stations, highly aligning with actual risk trends. It transforms abstract pedestrian density into concrete numerical parameters, integrating them into comprehensive risk calculations. This allows risk assessment to consider not only behavior and distance but also environmental factors, comprehensively improving the accuracy and rationality of early warning results.

[0046] In this embodiment, for each target pedestrian, a preset distance and distance weight relationship table is used to convert the distance between the target pedestrian and the hazard source into a weighted parameter that can participate in risk calculation, thereby achieving accurate quantification of the degree of location danger. This relationship table is based on the distance distribution statistics of falls and collisions caused by people looking down at their phones, and adopts a three-segment rule, strictly adhering to the principle that distance and risk are positively correlated. When the distance between the target pedestrian and the hazard source is greater than 3 meters, it is within the safe range, and the probability of an accident is extremely low. Therefore, the distance weight is set to 0.2, with a low risk contribution. When the distance between the target pedestrian and the hazard source is between 1 meter and 3 meters, it is within the warning range. The pedestrian is close to the danger zone, and an accident is likely to occur while looking down. Therefore, the distance weight is set to 0.5, with a medium risk contribution. When the distance between the target pedestrian and the hazard source is less than 1 meter, it is within the high-risk range. The pedestrian is very close to the hazard source, and a fall or collision may occur at any time. Therefore, the distance weight is set to 0.8, with a high risk contribution.

[0047] For example, if an ultrasonic sensor measures a pedestrian's distance from a subway platform door to be 0.9 meters, meeting the segmentation condition of a pedestrian's distance from a hazard source being <1 meter, a distance weight of 0.8 is assigned. If the distance is 2 meters, the weight is 0.5; and if it's 4 meters, the weight is 0.2. This lookup-based assignment method is highly efficient, capable of instantaneous calculation on edge devices, ensuring real-time system response. Furthermore, the weight values ​​are calibrated through real-world testing in three scenarios: stairs, zebra crossings, and platforms, closely aligning with actual location risks. This transforms abstract distance parameters into concrete numerical weights, integrating them into comprehensive risk calculations. This allows risk assessment to simultaneously cover behavioral, environmental, and location dimensions, comprehensively improving the risk perception accuracy of the early warning system and preventing missed warnings due to missing location risks.

[0048] In other embodiments, to further improve the smoothness and precision of the pedestrian density coefficient, the preset correlation table between the number of pedestrians and the pedestrian density coefficient can not only adopt a segmented lookup method, but also store the linear function relationship between all pedestrian numbers and the pedestrian density coefficient, realizing continuous change of the coefficient with the number of people, avoiding the abrupt changes caused by segmented assignment, and making the risk assessment more delicate and stable; the linear function is expressed as:

[0049] In the formula, Human flow density coefficient To monitor the number of all pedestrians in the area, , These are the linear coefficients calibrated according to the scenario. For example, in a campus stairwell scenario, they are obtained by fitting measured data. , The values ​​are 0.02 and 1 respectively.

[0050] Similarly, to make the changes in distance risk more continuous and natural, the pre-defined distance-distance weight relationship table can not only use a segmented lookup method, but also store the linear function relationship between distance and distance weight. This allows the weight to transition smoothly and continuously with changes in distance, improving the granularity and rationality of risk assessment. This linear function is expressed as:

[0051] In the formula, Distance weights The distance between the target pedestrian and the hazard source. , These are coefficients calibrated based on the scenario. For example, in a subway platform scenario, these coefficients are obtained through measured fitting. , The values ​​are 1 and 0.27, respectively.

[0052] In this embodiment, in step S104, the average behavioral confidence of each target pedestrian in consecutive multi-frame image data is calculated. Based on the average behavioral confidence of each target pedestrian in consecutive multi-frame image data, the pedestrian density coefficient, and the distance weight corresponding to each target pedestrian, a preset weighted fusion algorithm is used to calculate the comprehensive risk value of each target pedestrian. This includes: calculating the comprehensive risk value of each target pedestrian using a preset comprehensive risk calculation function. The comprehensive risk calculation function is expressed as follows:

[0053] In the formula, This represents the overall risk value. This represents the average behavioral confidence of the target pedestrian across multiple consecutive frames of image data. This represents the population density coefficient. Indicates distance weight, , , The weighting coefficients are preset and calibrated using real-world data from three scenarios: campus, intersection, and subway station. These weighting coefficients can be flexibly adjusted according to the scenario; for example, subway stations emphasize distance risk, so the γ value can be increased, while campuses emphasize behavioral risk, so the γ value can be increased. This value is highly versatile. In basic scenarios, The value is 0.4. The value is 0.2. With a value of 0.4, this configuration allows behavioral confidence (looking down) and distance weight (dangerous distance) to dominate, while the crowd density coefficient (environmental congestion) serves as an auxiliary correction, which aligns with the core causes of accidents involving people looking down.

[0054] For example, the average behavioral confidence in a series of consecutive image frames. (Clearly looking down at a phone), crowd density coefficient (Dense crowd), distance weight (Adjacent to the hazard source), substituting into the comprehensive risk calculation function yields a comprehensive risk value of 0.94. The comprehensive risk value is continuously quantifiable and can accurately reflect the true degree of danger posed to pedestrians.

[0055] In this embodiment, step S105, obtaining the range of each risk level and matching the comprehensive risk value of each target pedestrian with the range of each risk level to determine the risk level of each target pedestrian, includes: obtaining the range of each risk level and matching the comprehensive risk value of each target pedestrian with the range of each risk level; when the comprehensive risk value of the target pedestrian is less than a first threshold, the risk level of the target pedestrian is determined to be low risk; when the comprehensive risk value of the target pedestrian is greater than or equal to the first threshold and less than a second threshold, the risk level of the target pedestrian is determined to be medium risk; and when the comprehensive risk value of the target pedestrian is greater than or equal to the second threshold, the risk level of the target pedestrian is determined to be high risk.

[0056] In this embodiment, a three-level risk level range, pre-loaded based on actual measurements, is used as the basis for determining the comprehensive risk value. The risk level ranges are established based on statistical analysis of the probability of accidents caused by pedestrians looking down at their phones in three major scenarios: campuses, intersections, and subway stations. They are divided into three standardized ranges. A first threshold of 0.4 and a second threshold of 0.8 are used as critical values. After tens of thousands of actual tests, these values ​​can accurately distinguish between safe, alert, and emergency states. The threshold settings are scientifically reasonable, avoiding false or missed warnings caused by grading bias. Specifically, a comprehensive risk value < 0.4 indicates a low-risk range, meaning pedestrians pose no obvious danger and only gentle warnings are needed; 0.4 ≤ comprehensive risk value < 0.8 indicates a medium-risk range, meaning pedestrians are in a alert state and stronger warnings are needed; and a comprehensive risk value ≥ 0.8 indicates a high-risk range, meaning pedestrians are in an emergency state and immediate emergency warnings are needed.

[0057] In this embodiment, for each target pedestrian, the calculated comprehensive risk value is compared with a preset range to quickly determine the pedestrian's risk level, providing a basis for subsequent early warning strategy matching. For example, a comprehensive risk value of 0.25, less than 0.4, is determined to be low risk; a comprehensive risk value of 0.6, between 0.4 and 0.8, is determined to be medium risk; and a comprehensive risk value of 0.94, greater than or equal to 0.8, is determined to be high risk.

[0058] In this embodiment, in step S106, based on the risk level of each target pedestrian, a risk warning strategy corresponding to the risk level of each target pedestrian is matched from a preset risk warning strategy library, and a warning instruction corresponding to each target pedestrian is generated based on the risk warning strategy. The warning instructions are then executed sequentially according to the comprehensive risk value of each target pedestrian from high to low, including: when the risk level of the target pedestrian is low risk, a first-level risk warning strategy is matched from the preset risk warning strategy library, and a first-level warning instruction is generated based on the first-level risk warning strategy. The first-level warning instruction is used to control the RGB projection lamp to project a prompt pattern with a first color and a first flashing frequency onto the ground; when the risk level of the target pedestrian is medium risk, a second-level risk warning strategy is matched from the preset risk warning strategy library. The system first establishes a risk warning strategy and generates a second-level warning command based on the second-level risk warning strategy. The second-level warning command controls RGB projection lights to project a warning pattern with a second color and a second flashing frequency on the ground, where the second flashing frequency is higher than the first flashing frequency. When the risk level of the target pedestrian is high, a third-level risk warning strategy is matched from the preset risk warning strategy library, and a third-level warning command is generated based on the third-level risk warning strategy. The third-level warning command controls RGB projection lights to project an emergency warning pattern with a third color and a third flashing frequency on the ground, and controls directional speakers to play a preset warning voice, where the third flashing frequency is higher than the second flashing frequency. The warning commands are executed sequentially according to the comprehensive risk value of each target pedestrian from high to low.

[0059] In this embodiment, the first color is green and the first flashing frequency is 1Hz; the second color is yellow and the second flashing frequency is 3Hz; and the third color is red and the third flashing frequency is 8Hz.

[0060] In this embodiment, low risk indicates that the pedestrian is in a safe state and only requires a slight reminder. Therefore, the command controls the RGB projection light to project a prompt pattern of the first color (green) and the first flashing frequency (1Hz) on the ground. The pattern is a soft style such as the outline of steps and safety signs, without voice broadcast to avoid disturbing the surrounding environment. Green is an internationally recognized safety color, and the slow flashing frequency of 1Hz is soft and not dazzling. It only forms a visual prompt under the pedestrian's feet, directly entering the line of sight of pedestrians who are looking down, reminding them to pay attention to their surroundings. It achieves the reminder effect without noise pollution or strong light interference, making it suitable for scenarios with high requirements for quietness, such as schools and parks.

[0061] Medium-risk indicates pedestrians are in a state of alert and require clear reminders. The system controls RGB projection lights to project a warning pattern in a second color (yellow) with a second flashing frequency (3Hz). Yellow is an internationally recognized warning color, and the 3Hz flashing frequency is higher than low-risk, increasing the visual warning intensity. There is no voice announcement. The fast-flashing yellow pattern is eye-catching and can quickly attract the attention of pedestrians looking down, reminding them to stay away from the danger zone. The warning intensity is moderate, improving the reminder effect without causing strong light interference, making it suitable for semi-dense scenarios such as intersections and parks.

[0062] High risk indicates that the pedestrian is in an emergency and needs to be forcibly awakened. The command controls the RGB projection lights to project an emergency warning pattern in the third color (red) and with the third flashing frequency (8Hz). At the same time, the directional speaker plays a preset warning voice (such as "Watch your step" or "Red light is dangerous"). Red is an internationally recognized emergency color, and the 8Hz flashing frequency is highly visually impactful. The sound from the directional speaker is confined to a narrow angle of ±15°, only emitting sound towards the target pedestrian and not disturbing the surrounding area.

[0063] In this embodiment, the pedestrian safety warning method further includes: acquiring the relative distance and relative speed of oncoming pedestrians in real time, and calculating the collision time based on the relative distance and relative speed of oncoming pedestrians; if the collision time is lower than a preset safety time threshold, the risk level of the oncoming pedestrian is regarded as high risk, and an emergency warning strategy is matched from a preset risk warning strategy library, and an emergency warning command is generated based on the emergency warning strategy to immediately execute the warning. The emergency warning command is used to control the RGB projection lamp to project an emergency warning pattern with a fourth color and a fourth flashing frequency on the ground, and to control the directional speaker to play a preset warning voice, wherein the fourth flashing frequency is higher than or equal to the third flashing frequency.

[0064] In this embodiment, the relative distance and speed of oncoming pedestrians are extracted from the image using a visual target tracking algorithm (such as SORT / DeepSORT), or the relative distance and speed of oncoming pedestrians are collected using millimeter-wave radar.

[0065] The relative distance and velocity of oncoming pedestrians are extracted from images using visual target tracking algorithms (such as SORT / DeepSORT). This involves real-time acquisition of multiple consecutive frames of image data within the monitored area using a depth camera or a standard RGB camera (if multiple consecutive frames of image data within the monitored area have already been acquired, this data can be used directly). Visual target detection algorithms (such as YOLOv8) are used to detect the bounding boxes of all pedestrians in each frame. Multi-target tracking algorithms (such as SORT and DeepSORT) are then used to identify the same pedestrian in consecutive frames, forming their respective motion trajectories. Based on the trajectory data, the relative distance between two oncoming pedestrians in three-dimensional space is calculated. First, the pedestrian's foot coordinates in the image coordinate system are mapped to the world coordinate system using camera calibration parameters to obtain the pedestrian's true position. Then, the Euclidean distance is calculated. The relative velocity is obtained by differential smoothing the distance changes over multiple consecutive frames.

[0066] Millimeter-wave radar is used to collect the relative distance and relative speed of oncoming pedestrians. This is achieved by installing millimeter-wave radar (such as 60GHz or 77GHz vehicle-mounted radar) on either side or above the monitored area. Millimeter-wave radar transmits frequency-modulated continuous wave (FMCW) and receives the echo, allowing simultaneous measurement of the distance, speed, and angle of multiple targets. It features a narrow radar beam (typically ±5°), high angular resolution (1°~3°), and the ability to stably distinguish and track two oncoming pedestrians, outputting the relative radial distance and radial speed of each target. Because millimeter-wave radar directly measures velocity (Doppler effect), it eliminates the need for multi-frame differential analysis, resulting in a fast response time (milliseconds) and high robustness, unaffected by lighting conditions, fog, or dust.

[0067] For example, if two pedestrians walking towards each other with their heads down have a relative distance of d = 2 meters and a relative speed of v = 1 m / s, the calculated TTC = 2 seconds.

[0068] In this embodiment, if the collision time is lower than a preset safe time threshold, the two oncoming pedestrians are determined to be in an emergency danger state. At this time, regardless of whether the pedestrian is identified as a target pedestrian, their risk level is directly regarded as high risk, and their comprehensive risk value is forcibly set to the maximum value of 1.0 to ensure that they receive the highest priority in the subsequent warning instruction sorting. Subsequently, an emergency warning strategy is matched from the preset risk warning strategy library to generate an emergency warning instruction. The RGB projection light is controlled to project an emergency warning pattern with a fourth color (such as red flash) and a fourth flashing frequency (such as 8Hz or higher) on the ground. At the same time, the directional speaker is controlled to play a preset warning voice (such as please pay attention to oncoming pedestrians and avoid them). The fourth flashing frequency is higher than or equal to the aforementioned third flashing frequency (the warning frequency for high-risk pedestrians looking down) to highlight the urgency of the head-on collision.

[0069] Understandably, the preset safety time threshold is based on pedestrian reaction time calibration and is the shortest safe time to avoid head-on collisions. In this embodiment, the preset safety time threshold is 2 seconds.

[0070] This step is the emergency avoidance judgment stage. The calculated collision time is compared with a preset safe time threshold. If it is lower than the threshold, all oncoming pedestrians are directly considered target pedestrians and classified as high-risk, triggering an emergency warning for priority response. When TTC < 2 seconds, the target pedestrian cannot avoid the collision in time and must be immediately and forcibly awakened. For example, if the oncoming pedestrian's TTC = 1 second < 2 seconds, both are directly classified as high-risk, skipping the behavioral confidence and risk calculation process, immediately issuing a warning, and forcibly waking both to avoid a head-on collision. This mechanism has higher priority than conventional risk assessment, enabling rapid response to emergency collision risks without increasing system complexity, and seamlessly integrating with existing processes. It is applicable to all oncoming pedestrian scenarios; regardless of whether the pedestrian is looking down, as long as there is a collision risk, an emergency warning can be issued, comprehensively covering the behavioral and motion risks of those looking down at their phones, significantly improving safety.

[0071] If only one projection device and one directional speaker are installed in the monitored area, warnings will be issued to two pedestrians in any order. If multiple projection devices and multiple directional speakers are installed in the monitored area, warnings will be issued to two pedestrians simultaneously.

[0072] The pedestrian safety warning method in the embodiments of the present invention has been described above. The device in the embodiments of the present invention is described below. Please refer to [link / reference]. Figure 2 The implementation methods of the pedestrian safety warning device in this invention include: The judgment module 201 is used to acquire continuous multi-frame image data of a first preset duration in the monitored area, input each frame of the continuous multi-frame image data into a pre-trained AI posture recognition model, obtain the behavior confidence of each pedestrian in each frame image, and for each pedestrian, if there is a continuous second preset duration within the first preset duration, and the behavior confidence of the pedestrian is greater than a preset confidence threshold in all frames within the continuous second preset duration, then the pedestrian is determined to be the target pedestrian, and the second preset duration is less than the first preset duration; The acquisition module 202 is used to acquire the total number of pedestrians in the monitoring area and the distance between each target pedestrian and the hazard source when a target pedestrian is present. The matching module 203 is used to match the total number of pedestrians with a preset number of pedestrians and a relationship table of pedestrian flow density coefficients to determine the corresponding pedestrian flow density coefficients, and to match the distance between each target pedestrian and the hazard source with a preset distance and distance weight relationship table to determine the distance weight of each target pedestrian. The calculation module 204 is used to calculate the average behavioral confidence of each target pedestrian in a series of consecutive frames of image data, and to calculate the comprehensive risk value of each target pedestrian based on the average behavioral confidence of each target pedestrian in a series of consecutive frames of image data, the pedestrian density coefficient, and the distance weight corresponding to each target pedestrian through a preset weighted fusion algorithm. The risk determination module 205 is used to obtain the range of each risk level, match the comprehensive risk value of each target pedestrian with the range of each risk level, and determine the risk level of each target pedestrian. The risk warning module 206 is used to match the risk warning strategy corresponding to the risk level of each target pedestrian from the preset risk warning strategy library based on the risk level of each target pedestrian, generate a warning instruction corresponding to each target pedestrian based on the risk warning strategy, and execute each warning instruction in sequence according to the comprehensive risk value of each target pedestrian from high to low.

[0073] In this embodiment, multi-frame image continuous detection and AI posture recognition model are used to accurately screen out target pedestrians who are constantly looking down at their mobile phones, effectively avoiding misjudgment in a single frame. At the same time, pedestrian density coefficient and distance weight are integrated, and a weighted algorithm is used to dynamically evaluate the comprehensive risk value of each target pedestrian, realizing multi-dimensional risk quantification from behavior, environment to location. Differentiated early warning strategies are matched according to risk level (low risk green slow flash, medium risk yellow fast flash, high risk red burst flash plus directional voice), and early warning instructions are executed sequentially from high to low comprehensive risk value. This ensures that the most dangerous pedestrians receive priority intervention, and avoids resource conflicts and mutual interference when multiple targets are warned in parallel. Thus, while improving the accuracy and effectiveness of early warning, it effectively reduces the probability of pedestrian safety accidents.

[0074] Figure 2 The structure of the pedestrian safety warning device shown does not constitute a limitation on the pedestrian safety warning device, and can implement the steps of the pedestrian safety warning methods provided in the above-described method embodiments.

[0075] above Figure 2 The pedestrian safety warning device in the embodiments of the present invention will be described in detail from the perspective of modular functional entities. The pedestrian safety warning device in the embodiments of the present invention will be described in detail from the perspective of hardware processing.

[0076] Figure 3This is a schematic diagram of a pedestrian safety warning device provided in an embodiment of the present invention. The device 300 can vary significantly due to different configurations or performance characteristics. It may include one or more central processing units (CPUs) 310 (e.g., one or more processors) and a memory 320, and one or more storage media 330 (e.g., one or more mass storage devices) for storing application programs 333 or data 332. The memory 320 and storage media 330 can be temporary or persistent storage. The program stored in the storage media 330 may include one or more modules (not shown), each module including a series of instruction operations on the device 300. Furthermore, the processor 310 may be configured to communicate with the storage media 330 and execute the series of instruction operations stored in the storage media on the device 300.

[0077] Device 300 may also include one or more power supplies 340, one or more wired or wireless network interfaces 350, one or more input / output interfaces 360, and / or one or more operating systems 331, such as Windows Server, Mac OS X, Unix, Linux, FreeBSD, etc.

[0078] This invention also provides a computer-readable storage medium, which can be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium. The computer-readable storage medium stores instructions that, when executed on a computer, cause the computer to perform the steps of a pedestrian safety warning method.

[0079] Those skilled in the art will clearly understand that, for the sake of convenience and brevity, the specific working process of the system, device, or unit described above can be referred to the corresponding process in the foregoing method embodiments, and will not be repeated here.

[0080] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0081] The above-described embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit it. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A pedestrian safety early warning method, characterized in that, The pedestrian safety early warning method includes: Acquire continuous multi-frame image data of the monitored area for a first preset duration, input each frame of the continuous multi-frame image data into a pre-trained AI posture recognition model, obtain the behavior confidence of each pedestrian in each frame image, for each pedestrian, if there is a continuous second preset duration within the first preset duration, and the behavior confidence of the pedestrian is greater than a preset confidence threshold in all frames within the continuous second preset duration, then determine that the pedestrian is the target pedestrian, and the second preset duration is less than the first preset duration; When a target pedestrian is present, obtain the total number of pedestrians in the monitored area and the distance between each target pedestrian and the hazard source; The number of all pedestrians is matched with the preset number of pedestrians and the relationship table of pedestrian flow density coefficient to determine the corresponding pedestrian flow density coefficient. The distance between each target pedestrian and the hazard source is matched with the preset distance and distance weight relationship table to determine the distance weight of each target pedestrian. The average behavioral confidence of each target pedestrian in consecutive multi-frame image data is calculated, and based on the average behavioral confidence of each target pedestrian in consecutive multi-frame image data, the pedestrian density coefficient, and the distance weight corresponding to each target pedestrian, the comprehensive risk value of each target pedestrian is calculated by a preset weighted fusion algorithm. Obtain the range of each risk level, match the comprehensive risk value of each target pedestrian with the range of each risk level, and determine the risk level of each target pedestrian; Based on the risk level of each target pedestrian, a risk warning strategy corresponding to the risk level of each target pedestrian is matched from a preset risk warning strategy library. Based on the risk warning strategy, a warning instruction corresponding to each target pedestrian is generated. The warning instructions are executed in sequence according to the comprehensive risk value of each target pedestrian from high to low.

2. The pedestrian safety early warning method according to claim 1, characterized in that, The process of acquiring continuous multi-frame image data of a first preset duration in the monitored area, inputting each frame of the continuous multi-frame image data into a pre-trained AI pose recognition model to obtain the behavior confidence score of each pedestrian in each frame, and for each pedestrian, if there is a continuous second preset duration within the first preset duration, and the behavior confidence score of the pedestrian is greater than a preset confidence threshold in all frames within the continuous second preset duration, then the pedestrian is determined to be the target pedestrian, where the second preset duration is less than the first preset duration, includes: Collect video and image data in the target application scenario, and construct a dataset based on the video and image data in the target application scenario. The dataset includes multiple training samples, including categories of looking down at a mobile phone and normal walking, and the training samples have key point annotations of the head, neck, wrist, and arm. A pre-trained model is built based on YOLO, and the pre-trained model is trained using a transfer learning strategy and the dataset to obtain an AI pose recognition model. The system continuously detects the presence of moving human bodies within the monitored area using a low-power passive infrared sensor. When it is determined that there is a moving human body in the monitored area, the system acquires a series of multiple frames of image data for a first preset duration in the monitored area. Each frame of the series of multiple frames of image data is input into the AI ​​posture recognition model to obtain the behavior confidence of each pedestrian in each frame. For each pedestrian, if there is a continuous second preset duration within the first preset duration, and the behavior confidence of the pedestrian is greater than a preset confidence threshold in all frames within the continuous second preset duration, then the pedestrian is determined to be the target pedestrian, and the second preset duration is less than the first preset duration.

3. The pedestrian safety early warning method according to claim 1, characterized in that, When a target pedestrian is present, the method of obtaining the total number of pedestrians within the monitored area and the distance between each target pedestrian and the hazard source includes: When a target pedestrian is present, acquire real-time panoramic images of the monitored area and the distance between each target pedestrian and the hazard source; The number of all pedestrians in the monitored area is obtained based on real-time panoramic image statistics.

4. The pedestrian safety early warning method according to claim 1, characterized in that, The calculation obtains the average behavioral confidence of each target pedestrian in multiple consecutive frames of image data. Based on the average behavioral confidence of each target pedestrian in multiple consecutive frames of image data, the pedestrian density coefficient, and the distance weight corresponding to each target pedestrian, a preset weighted fusion algorithm is used to calculate the comprehensive risk value of each target pedestrian. This includes: calculating the comprehensive risk value of each target pedestrian using a preset comprehensive risk calculation function. The comprehensive risk calculation function is expressed as follows: In the formula, This represents the overall risk value. This represents the average behavioral confidence of the target pedestrian across multiple consecutive frames of image data. Indicates the population density coefficient. Indicates distance weight, , , These are the preset weighting coefficients.

5. The pedestrian safety early warning method according to claim 1, characterized in that, The process of obtaining the range of each risk level, matching the comprehensive risk value of each target pedestrian with the range of each risk level, and determining the risk level of each target pedestrian includes: Obtain the range of each risk level, and match the comprehensive risk value of each target pedestrian with the range of each risk level; When the overall risk value of the target pedestrian is less than the first threshold, the risk level of the target pedestrian is determined to be low risk. When the overall risk value of the target pedestrian is greater than or equal to the first threshold and less than the second threshold, the risk level of the target pedestrian is determined to be medium risk. When the overall risk value of the target pedestrian is greater than or equal to the second threshold, the risk level of the target pedestrian is determined to be high risk.

6. The pedestrian safety early warning method according to claim 1, characterized in that, Based on the risk level of each target pedestrian, a risk warning strategy corresponding to the risk level of each target pedestrian is matched from a preset risk warning strategy library. Based on the risk warning strategy, a warning instruction corresponding to each target pedestrian is generated. The warning instructions are then executed sequentially according to the comprehensive risk value of each target pedestrian from high to low, including: When the risk level of the target pedestrian is low risk, a first-level risk warning strategy is matched from the preset risk warning strategy library, and a first-level warning instruction is generated based on the first-level risk warning strategy. The first-level warning instruction is used to control the RGB projection light to project a prompt pattern with a first color and a first flashing frequency on the ground. When the risk level of the target pedestrian is medium risk, a second-level risk warning strategy is matched from the preset risk warning strategy library, and a second-level warning instruction is generated based on the second-level risk warning strategy. The second-level warning instruction is used to control the RGB projection light to project a warning pattern with a second color and a second flashing frequency on the ground. The second flashing frequency is higher than the first flashing frequency. When the risk level of the target pedestrian is high risk, a third-level risk warning strategy is matched from the preset risk warning strategy library, and a third-level warning instruction is generated based on the third-level risk warning strategy. The third-level warning instruction is used to control the RGB projection lamp to project an emergency warning pattern with a third color and a third flashing frequency on the ground, and to control the directional speaker to play a preset warning voice. The third flashing frequency is higher than the second flashing frequency. And according to the comprehensive risk value of each target pedestrian, from high to low, each warning instruction is executed in sequence.

7. The pedestrian safety early warning method according to claim 6, characterized in that, Also includes: The relative distance and relative speed of oncoming pedestrians are acquired in real time, and the collision time is calculated based on the relative distance and relative speed of oncoming pedestrians. If the collision time is lower than the preset safe time threshold, the risk level of the oncoming pedestrian is considered high risk, and an emergency warning strategy is matched from the preset risk warning strategy library. An emergency warning command is generated based on the emergency warning strategy to immediately execute the warning. The emergency warning command is used to control the RGB projection light to project an emergency warning pattern with a fourth color and a fourth flashing frequency on the ground, and to control the directional speaker to play a preset warning voice. The fourth flashing frequency is higher than or equal to the third flashing frequency.

8. A pedestrian safety early warning device, characterized in that, include: The judgment module is used to acquire continuous multi-frame image data of a first preset duration in the monitored area, input each frame of the continuous multi-frame image data into a pre-trained AI posture recognition model, obtain the behavior confidence of each pedestrian in each frame, and for each pedestrian, if there is a continuous second preset duration within the first preset duration, and the behavior confidence of the pedestrian is greater than a preset confidence threshold in all frames within the continuous second preset duration, then the pedestrian is determined to be the target pedestrian, and the second preset duration is less than the first preset duration; The acquisition module is used to acquire the total number of pedestrians in the monitored area and the distance between each pedestrian and the hazard source when a target pedestrian is present. The matching module is used to match the total number of pedestrians with a preset number of pedestrians and a relationship table of pedestrian flow density coefficients to determine the corresponding pedestrian flow density coefficients, and to match the distance between each target pedestrian and the hazard source with a preset distance and distance weight relationship table to determine the distance weight of each target pedestrian. The calculation module is used to calculate the average behavioral confidence of each target pedestrian in a series of consecutive frames of image data, and to calculate the comprehensive risk value of each target pedestrian based on the average behavioral confidence of each target pedestrian in a series of consecutive frames of image data, the pedestrian flow density coefficient and the distance weight corresponding to each target pedestrian through a preset weighted fusion algorithm. The risk determination module is used to obtain the range of each risk level, match the comprehensive risk value of each target pedestrian with the range of each risk level, and determine the risk level of each target pedestrian. The risk warning module is used to match the risk warning strategy corresponding to the risk level of each target pedestrian from the preset risk warning strategy library, generate a warning instruction corresponding to each target pedestrian based on the risk warning strategy, and execute each warning instruction in order of the comprehensive risk value of each target pedestrian from high to low.

9. A pedestrian safety early warning device, characterized in that, It includes a memory and at least one processor, wherein the memory stores computer-readable instructions; The at least one processor invokes the computer-readable instructions in the memory to perform the steps of the pedestrian safety warning method as described in any one of claims 1-7.

10. A computer-readable storage medium storing computer-readable instructions thereon, characterized in that, When the computer-readable instructions are executed by a processor, they implement the steps of the pedestrian safety warning method as described in any one of claims 1-7.

Images