Deep learning-based platform safety line grading early warning method and system

Abstract

The invention discloses a platform safety line grading early warning method and system based on deep learning, and belongs to the technical field of computer vision and public safety. The method comprises the following steps: initializing a system; acquiring video stream data of a platform monitoring area in real time and performing image preprocessing; identifying and positioning pedestrians in the video frame by using the target detection and segmentation model; based on the identified foot position information of the pedestrian, judging whether the pedestrian crosses a line or not; utilizing a multi-task attribute identification model to identify the attributes of the line-crossing pedestrians; calculating a minimum line-crossing distance of the line-crossing pedestrians and a target density factor of a line-crossing area, generating a comprehensive threat score, and generating a priority queue; and generating and executing a self-adaptive multi-mode safety reminder. According to the invention, the automatic, precise and graded management of the platform line crossing behavior is realized, the timeliness, accuracy and humanization level of early warning can be effectively improved, and the method is suitable for the safety protection of public traffic scenes such as high-speed railway stations and subway stations.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and public safety technology, specifically to a platform safety line hierarchical early warning method and system based on deep learning. Background Technology

[0002] In high-speed rail stations, train stations, and other similar locations, the yellow safety warning line along the platform edge is a crucial facility for ensuring passenger safety. Its core function is to delineate a safe zone for waiting passengers, preventing two main types of risks: First, during the waiting period before the train arrives, passengers may cross the line due to crowding, playing, or negligence, posing a direct risk of accidentally falling onto the tracks. Second, during the entire process of the train entering the platform, the high-speed train generates a powerful suction airflow, and the gap between the train and the platform is extremely small; at this time, passengers who have crossed the line face the fatal danger of being swept into the airflow or scraping against the train.

[0003] Currently, preventing passengers from crossing the line relies mainly on visual observation and verbal reminders from platform staff. However, during peak passenger flow periods, it is difficult for staff to monitor all areas simultaneously, leading to oversights. Some stations have installed simple infrared or laser sensor alarm devices, but these technologies cannot distinguish the attributes of the intruders, such as whether they are adults or children, domestic or international passengers, nor can they intelligently prioritize reminders when multiple people cross the line at the same time. This results in low reminder efficiency and is prone to false alarms (such as luggage touching), which can cause passenger annoyance. Summary of the Invention

[0004] The technical problem this invention aims to solve is to overcome the shortcomings of existing technologies and provide a platform safety early warning method and system capable of automatically and in real-time detecting line-crossing behavior and providing differentiated and prioritized voice prompts based on target attributes, thereby significantly improving the efficiency and user-friendliness of platform safety management. To solve the above problem, this invention adopts the following technical solution:

[0005] First, this invention proposes a deep learning-based method for hierarchical early warning of platform safety lines, comprising the following steps:

[0006] S1. Complete system initialization through camera calibration and warning line calibration;

[0007] S2. Real-time acquisition of video stream data from the platform monitoring area and image preprocessing;

[0008] S3. Use deep learning object detection and segmentation models to identify and locate pedestrians in video frames;

[0009] S4. Based on the identified pedestrian foot position information, determine whether the pedestrian has crossed the preset safety warning line;

[0010] S5. Use a multi-task attribute recognition model to identify the attributes of pedestrians crossing the line, including age, gender, and nationality;

[0011] S6. Calculate the minimum crossing distance of pedestrians who cross the line and the target density factor of the crossing area, and generate a comprehensive threat score based on the dynamic priority algorithm. Sort all pedestrians who cross the line according to the comprehensive threat score and generate a priority queue.

[0012] S7. Generate and execute adaptive multimodal safety alerts based on the priority queue and pedestrian attributes.

[0013] Preferably, the target detection and segmentation model in step S3 adopts the YOLOv8-Seg model to obtain the bounding box and pixel-level mask of the pedestrian.

[0014] Preferably, in step S4, determining whether a preset safety warning line has been crossed specifically includes: extracting a mask of the pedestrian's foot area through an instance segmentation model, or obtaining the ankle joint coordinates through a human key point detection model, and determining its positional relationship with the preset warning line polygon.

[0015] Preferably, the dynamic priority algorithm in step S6 specifically includes:

[0016] Risk levels are determined based on the minimum distance crossed the line. ; Calculate the number of targets per unit area in the area that crosses the line as the density factor. ; through formula Calculate the overall threat score ; Generate a priority queue by sorting the threats in descending order, where and It is an adjustable weighting coefficient.

[0017] Preferably, the adaptive multimodal reminder in step S7 specifically includes:

[0018] Select either the Chinese or English voice library based on the nationality attribute;

[0019] A personalized nickname is generated based on the combination of the age and gender attributes;

[0020] Select the voice urgency level and the intensity of the audible and visual alarms based on the threat score;

[0021] If the target density factor of the crossed-line area is higher than a set threshold, a group warning broadcast is triggered.

[0022] Meanwhile, this invention proposes a platform safety line graded early warning system based on deep learning, comprising:

[0023] The image acquisition module is used to acquire the surveillance video stream of the platform area;

[0024] The preprocessing module, connected to the image acquisition module, is used to perform image correction and enhancement processing on the video stream;

[0025] The core analysis module, connected to the preprocessing module, is used to perform pedestrian target detection and instance segmentation on the processed video frames, determine whether pedestrians have crossed the line based on the segmentation results, and identify the attributes of pedestrians who have crossed the line.

[0026] The decision module, connected to the core analysis module, is used to calculate a comprehensive threat score and generate a priority queue based on the attributes of the pedestrian who crossed the line, the minimum crossing distance, and the target density of the crossing area.

[0027] The execution module, connected to the decision module, is used to generate and execute adaptive multimodal safety alerts based on the priority queue and pedestrian attributes.

[0028] Preferably, the core analysis module integrates multiple deep learning models, including the YOLOv8-Seg model for object detection, the OpenPose model for keypoint detection, and the ResNet model for attribute classification.

[0029] Furthermore, the present invention proposes a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method described in the present invention.

[0030] Meanwhile, the present invention proposes an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the computer program is executed, it implements the steps of the method described in the present invention.

[0031] Finally, the present invention provides a computer program product comprising a computer program / instructions that, when executed by a processor, implement the steps of the method described in the present invention.

[0032] The beneficial effects of adopting the above technical solution are as follows:

[0033] 1. This invention, through multi-target detection and attribute recognition algorithms, achieves automated and precise 24 / 7 monitoring of line-crossing behavior. This helps address oversights caused by staff fatigue or blind spots during peak passenger flow periods, providing valuable time to prevent safety incidents. Simultaneously, intelligent management reduces train delays or operational disruptions due to safety incidents, ensuring the smooth operation of the public transportation network and generating significant indirect economic benefits.

[0034] 2. This invention triggers differentiated voice reminders through attribute recognition, avoiding the discomfort caused by single and rigid warnings. It is more easily accepted by different groups of people, reflects humanistic care, significantly improves passenger acceptance and cooperation, and enhances public safety awareness.

[0035] 3. The system proposed in this invention can replace the monitoring tasks that previously required multiple staff members to perform in fixed positions, thus reducing human resource costs. Simultaneously, the dynamic priority decision-making mechanism ensures that, in complex scenarios, limited alerting resources are allocated to the highest-risk targets, maximizing safety benefits.

[0036] 4. The system architecture proposed in this invention is modular, which can be easily integrated into existing station security systems and can be extended to similar scenarios such as subway stations and airports, and has broad application value. Attached Figure Description

[0037] Figure 1 The flowchart of a platform safety line hierarchical early warning method based on deep learning is presented in this invention.

[0038] Figure 2 This invention relates to an overall structural diagram of a platform safety line hierarchical early warning system based on deep learning.

[0039] Figure 3 This is a schematic diagram illustrating the principle of line crossing judgment in the embodiment.

[0040] Figure 4 This is a flowchart of the dynamic priority decision-making algorithm involved in the embodiment.

[0041] Figure 5 This is a schematic diagram of the system deployment involved in the embodiment. Detailed Implementation

[0042] The technical solution of the present invention will be described in detail below with reference to the accompanying drawings.

[0043] Example 1: This example is a specific implementation of a platform safety line hierarchical early warning method and system based on deep learning, such as... Figure 1 As shown, the steps include:

[0044] S1. Complete system initialization through camera calibration and warning line calibration;

[0045] S2. Real-time acquisition of video stream data from the platform monitoring area and image preprocessing;

[0046] S3. Use deep learning object detection and segmentation models to identify and locate pedestrians in video frames;

[0047] S4. Based on the identified pedestrian foot position information, determine whether the pedestrian has crossed the preset safety warning line;

[0048] S5. Use a multi-task attribute recognition model to identify the attributes of pedestrians crossing the line, including age, gender, and nationality;

[0049] S6. Calculate the minimum crossing distance of pedestrians who cross the line and the target density factor of the crossing area, and generate a comprehensive threat score based on the dynamic priority algorithm. Sort all pedestrians who cross the line according to the comprehensive threat score and generate a priority queue.

[0050] S7. Generate and execute adaptive multimodal safety alerts based on the priority queue and pedestrian attributes.

[0051] The target detection and segmentation in step S3 uses the YOLOv8-Seg model to obtain the bounding box and pixel-level mask of the pedestrian.

[0052] The line crossing judgment in step S4 specifically involves: extracting the pedestrian foot area mask through the instance segmentation model, or obtaining the ankle joint coordinates through the human key point detection model, and judging its positional relationship with the preset warning line polygon.

[0053] The dynamic priority algorithm in step S6 specifically includes:

[0054] Risk levels are determined based on the minimum distance crossed the line. ; Calculate the number of targets per unit area in the area that crosses the line as the density factor. ; through formula Calculate the overall threat score ; Generate a priority queue by sorting the threats in descending order, where and It is an adjustable weighting coefficient.

[0055] The adaptive multimodal reminder in step S7 specifically includes:

[0056] Select Chinese or English voice libraries based on nationality; generate personalized titles based on a combination of age and gender attributes; select the urgency level of the voice and the intensity of the sound and light alarm based on the threat score; trigger a group warning broadcast for high-density crowds crossing the line.

[0057] Meanwhile, this embodiment proposes a system, such as Figure 2 As shown, it specifically includes:

[0058] The image acquisition module is used to acquire the surveillance video stream of the platform area;

[0059] The preprocessing module, connected to the image acquisition module, is used to perform image correction and enhancement processing on the video stream;

[0060] The core analysis module, connected to the preprocessing module, is used to perform pedestrian target detection and instance segmentation on the processed video frames, determine whether pedestrians have crossed the line based on the segmentation results, and identify the attributes of pedestrians who have crossed the line.

[0061] The decision module, connected to the core analysis module, is used to calculate a comprehensive threat score and generate a priority queue based on the attributes of the pedestrian who crossed the line, the minimum crossing distance, and the target density of the crossing area.

[0062] The execution module, connected to the decision module, is used to generate and execute adaptive multimodal safety alerts based on the priority queue and pedestrian attributes.

[0063] The core of the system lies in constructing an image acquisition module, a preprocessing module, a core analysis module, a decision-making module, and an execution module, specifically including:

[0064] Phase 1: System Initialization and Configuration

[0065] Step 1.1, Camera Calibration and Adjustment:

[0066] This stage is completed before system startup, providing a foundation and benchmark for real-time analysis. A calibration board is used to capture images at multiple positions and angles within the camera's field of view, acquiring a set of images. The intrinsic parameter matrix (focal length, principal point) and distortion coefficients of the camera are calculated using the `calibrateCamera` function in the OpenCV vision library. These parameters are saved for real-time correction of all subsequent video frames.

[0067] Step 1.2, Marking the safety warning line area:

[0068] The yellow safety line area requiring monitoring is precisely defined in the video feed. From the calibrated video, the administrator uses a human-computer interface to click sequentially with the mouse to connect the points, forming a polygonal area that closely follows the edge of the platform, serving as the warning zone. The coordinates of the vertices of this polygon are saved in the system configuration file. This line will be the reference for subsequent judgments on whether the line has been crossed.

[0069] Phase Two: Real-time Data Processing and Analysis Cycle

[0070] Step 2.1, Data Acquisition:

[0071] The system reads video streams from pre-calibrated high-definition network cameras. Video frames are extracted according to a preset frame rate. The system receives signals from the train's position sensors and remains silent during the boarding and alighting period when the train is at a complete stop in the station, to avoid continuous alarm prompts during passenger boarding and alighting; it maintains core analysis and executes real-time alarm reminders before the train enters the stationary area.

[0072] Step 2.2, Image Preprocessing:

[0073] Each frame of the image is corrected, enhanced, and scaled to the fixed input size required by the model.

[0074] Step 2.3, Object Detection and Instance Segmentation:

[0075] The preprocessed image is input into a pre-trained YOLOv8-Seg (YOLOv8 with segmentation capabilities) deep learning model, which outputs a bounding box and a corresponding pixel-level mask for each pedestrian. The mask can more accurately represent the shape of the person, especially the position of the feet.

[0076] Step 2.4, Line Crossing Judgment:

[0077] Accurately determine if a pedestrian's feet have entered the warning zone marked in step 1.2. For each pedestrian detected in step 2.3, extract the set of pixels at the bottom of the pedestrian instance segmentation mask (approximate foot landing point), and call the human keypoint detection model OpenPose to obtain the coordinates of the pedestrian's left and right ankles. Determine if any of these foot points fall inside the warning line polygon marked in step 1.2. If at least one point falls inside, the pedestrian is determined to have "crossed the line" and marked as a target to be processed, proceeding to the next step. Otherwise, ignore the pedestrian. For details on the principle of crossing the line here, please refer to [link to relevant documentation]. Figure 3 .

[0078] Step 2.5, Attribute Identification and Line Crossing Distance Calculation:

[0079] The system identifies the attributes of individuals who cross the boundary line and quantifies the severity of their transgression. Attribute identification involves cropping the bounding box region of the pedestrian's image and inputting it into a multi-task classification model, ResNet. The model outputs multiple attribute labels in parallel: nationality (domestic / foreign), age (adult / child), and gender (male / female). The distance calculation involves calculating the shortest vertical distance D from the foot point identified as having crossed the line to the boundary of the warning line polygon. This distance D is the pedestrian's minimum crossing distance.

[0080] Phase Three: Intelligent Decision-Making and Prioritization

[0081] The main purpose of this stage is to process all targets that have crossed the line and determine the order in which they will be alerted. The specific decision-making and sorting algorithm is shown in Figure 4.

[0082] Step 3.1: Calculate the target density factor ρ:

[0083] Assess the congestion level in the current area where pedestrians have crossed the line; a greater risk of mass movement necessitates more urgent intervention. Count the total number N of all pedestrians marked as "crossed the line" in the current frame. Calculate the total area of ​​these pedestrians (this can be estimated by the union of the bounding boxes of all pedestrians crossing the line, or by directly using the area of ​​a pre-defined fixed warning zone). Calculate the density factor: . The higher the value, the more concentrated the risk in the area.

[0084] Step 3.2: Calculate the overall threat score. :

[0085] Generate a quantifiable, comparable threat score for each target that crosses the line. Map the minimum crossing distance D calculated in step 2.5 to a distance level. :

[0086] if This is considered a Level 1 risk. (Approaching the restricted area).

[0087] if This is considered a level two risk. .

[0088] if Then it is the observation level. They are usually not added to the priority queue.

[0089] Calculate the overall threat score using the formula: .

[0090] in and It is an adjustable weighting coefficient used to balance the importance of individual risk and group risk.

[0091] Step 3.3: Generate a dynamic priority queue:

[0092] refer to Figure 4 The flowchart shown illustrates the dynamic prioritization decision-making algorithm, which determines the order of alerts based on the severity of the threat. All... Pedestrians who cross the line are classified according to their overall threat score. Sort the data in descending order. Create a priority alert queue. The target with the highest score will be alerted first.

[0093] Phase Four: Adaptive Reminder Execution and Feedback

[0094] This phase translates the decision-making results into concrete, humane warning actions.

[0095] Step 4.1: Generate adaptive reminder instructions:

[0096] Generate the most appropriate alert content based on target attributes and threat score. Retrieve the target that currently needs alerting from the priority queue.

[0097] Language content synthesis includes:

[0098] Choose either the Chinese or English voice library based on your nationality.

[0099] Based on age and gender, appropriate titles can be generated, such as Mr., Ms., and Little Friend.

[0100] Based on threat score Adjust the urgency level of the voice and the intensity of the sound and light. For example:

[0101] Ji Gao: "Urgent! The gentleman in the blue shirt, please step back immediately!"

[0102] Medium: "Lady, please do not cross the line, please be careful."

[0103] Group risk ( (Value exceeds the set threshold): "Attention multiple passengers, please do not crowd, take a step back!"

[0104] The sound and light mode selection controls the sound and light alarms in the corresponding area of ​​the station according to the threat level, using different flashing frequencies (fast flash / slow flash) and alarm tones (rapid / gradual).

[0105] Step 4.2, Execute reminders and system linkage:

[0106] The voice warning generated in step 4.1 is played through directional speakers deployed in the corresponding areas of the platform.

[0107] Simultaneously trigger the multi-color alarm lights in the area to provide visual warning.

[0108] High-risk alerts are sent to the station control center via network interface to alert staff to intervene.

[0109] After completing this round of processing, the system immediately jumps back to step 2.1 to obtain the next frame of video data and start a new round of processing to achieve real-time monitoring and early warning.

[0110] Furthermore, the system's core analysis module integrates multiple deep learning models, including the YOLOv8-Seg model for object detection, the OpenPose model for keypoint detection, and the ResNet model for attribute classification. The system also includes an interface for communication with the station system, enabling communication with the station control center and the acquisition of passenger nationality information to improve attribute recognition accuracy.

[0111] refer to Figure 5 The deployment diagram shown illustrates the system deployed on a high-speed railway station platform. The hardware configuration mainly includes: a 2-megapixel high-definition network camera; an NVIDIA Jetson Xavier edge computing device; a directional acoustic speaker array; and multi-color LED strobe alarm lights.

[0112] The software implementation is as follows:

[0113] 1. System initialization:

[0114] Camera calibration: The intrinsic parameter matrix and distortion coefficients were obtained using Zhang Zhengyou's calibration method;

[0115] Warning line marking: Precisely delineate the polygonal warning area in the video footage;

[0116] Model loading: Pre-trained YOLOv8-Seg, OpenPose, and ResNet-18 models.

[0117] 2. Real-time processing flow

[0118] The video stream is processed at 15fps with a resolution of 1920×1080.

[0119] Image preprocessing: distortion correction, histogram equalization, and size scaling;

[0120] Object detection: YOLOv8-Seg model inference to obtain pedestrian bounding boxes and masks;

[0121] Line crossing judgment: based on the positional relationship between key foot points and the warning area;

[0122] Attribute recognition: The multi-task ResNet model outputs age, gender, and nationality;

[0123] 3. Decision-making and execution

[0124] Calculate the threat score for each target that crosses the line (α = 0.7, β = 0.3).

[0125] Generate a priority queue;

[0126] Synthesize personalized voice prompts to trigger audio-visual alarms.

[0127] Single person crossing the line: Identified as a foreign adult female, English warning: "Madam, danger! Please step back!"

[0128] Multiple people crossing the line: Five people were detected crossing the line simultaneously, with one child being the closest. The child was given priority for warning with a Chinese prompt: "Little friend, please step back behind the yellow line!"

[0129] Crowdedness: Density Factor If the threshold is exceeded, an announcement will be made: "Passengers, please note that you should not crowd and should stay away from the platform warning line."

[0130] If the overall threat score S exceeds the set threshold, it is determined that the person is about to fall off the platform. A high-risk alarm is then sent to the station control center via the network interface to alert staff to intervene.

[0131] Example 2: This example proposes a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method described in this invention.

[0132] Example 3: This example proposes an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the computer program is executed, it implements the steps of the method described in this invention.

[0133] Example 4: This example proposes a computer program product, including a computer program / instructions, which, when executed by a processor, implements the steps of the method described in this invention.

[0134] It should be noted that the processing flow of embodiments 2-4 corresponds to the specific steps of the method provided in embodiment 1 of the present invention, and has the corresponding functional modules and beneficial effects of the method. Technical details not described in detail in this embodiment can be found in the method provided in embodiment 1 of the present invention.

[0135] The program code used to implement the methods of this application may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing device, such that when executed by the processor or controller, the functions / operations specified in the flowcharts and / or block diagrams are implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0136] The specific implementation schemes described above further illustrate the purpose, technical solution, and beneficial effects of the present invention. It should be understood that the above descriptions are merely specific implementation schemes of the present invention and are not intended to limit the scope of the present invention. Any equivalent changes and modifications made by those skilled in the art without departing from the concept and principles of the present invention should fall within the scope of protection of the present invention.

Claims

1. A platform safety line graded early warning method based on deep learning, characterized in that, Includes the following steps: S1. Complete system initialization through camera calibration and warning line calibration; S2. Real-time acquisition of video stream data from the platform monitoring area and image preprocessing; S3. Use deep learning object detection and segmentation models to identify and locate pedestrians in video frames; S4. Based on the identified pedestrian foot position information, determine whether the pedestrian has crossed the preset safety warning line; S5. Use a multi-task attribute recognition model to identify the attributes of pedestrians crossing the line, including age, gender, and nationality; S6. Calculate the minimum crossing distance of pedestrians who cross the line and the target density factor of the crossing area, and generate a comprehensive threat score based on the dynamic priority algorithm. Sort all pedestrians who cross the line according to the comprehensive threat score and generate a priority queue. S7. Generate and execute adaptive multimodal safety alerts based on the priority queue and pedestrian attributes.

2. The method according to claim 1, characterized in that, The target detection and segmentation model in step S3 uses the YOLOv8-Seg model to obtain the bounding box and pixel-level mask of the pedestrian.

3. The method according to claim 1, characterized in that, Step S4 determines whether the preset safety warning line has been crossed. Specifically, this includes: extracting the pedestrian foot area mask through the instance segmentation model, or obtaining the ankle joint coordinates through the human key point detection model, and determining its positional relationship with the preset warning line polygon.

4. The method according to claim 1, characterized in that, The dynamic priority algorithm in step S6 specifically includes: Risk levels are determined based on the minimum distance crossed the line. ; Calculate the number of targets per unit area in the area that crosses the line as the density factor. ; through formula Calculate the overall threat score ; Generate a priority queue by sorting the threats in descending order, where and It is an adjustable weighting coefficient.

5. The method according to claim 1, characterized in that, The adaptive multimodal reminder in step S7 specifically includes: Select either the Chinese or English voice library based on the nationality attribute; A personalized nickname is generated based on the combination of the age and gender attributes; Select the voice urgency level and the intensity of the audible and visual alarms based on the threat score; If the target density factor of the crossed-line area is higher than a set threshold, a group warning broadcast is triggered.

6. A platform safety line graded early warning system based on deep learning, characterized in that, include: The image acquisition module is used to acquire the surveillance video stream of the platform area; The preprocessing module, connected to the image acquisition module, is used to perform image correction and enhancement processing on the video stream; The core analysis module, connected to the preprocessing module, is used to perform pedestrian target detection and instance segmentation on the processed video frames, determine whether pedestrians have crossed the line based on the segmentation results, and identify the attributes of pedestrians who have crossed the line. The decision module, connected to the core analysis module, is used to calculate a comprehensive threat score and generate a priority queue based on the attributes of the pedestrian who crossed the line, the minimum crossing distance, and the target density of the crossing area. The execution module, connected to the decision module, is used to generate and execute adaptive multimodal safety alerts based on the priority queue and pedestrian attributes.

7. The system according to claim 6, characterized in that, The core analysis module integrates multiple deep learning models, including the YOLOv8-Seg model for object detection, the OpenPose model for keypoint detection, and the ResNet model for attribute classification.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the method as described in any one of claims 1 to 5.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the computer program is executed, it implements the steps of the method as described in any one of claims 1 to 5.

10. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, they implement the steps of the method according to any one of claims 1 to 5.

Images