A visual detection early warning method and system for passenger flow density monitoring

Abstract

The application discloses a visual detection early warning method and system for passenger flow density monitoring, comprising the following steps: receiving visual data continuously uploaded by a user based on a matching interface, and reading the visual data frame by frame according to a preset sampling interval to generate a visual data matrix comprising a plurality of frames of images; inputting the visual data matrix into a pre-trained visual detection model to output the number of trunk parts and the trunk part confidence in each frame of image, and the number of heads and the head confidence. The application can reduce the algorithm calculation complexity, trigger the alarm switching through the cumulative frame number, avoid the flexible early warning of transient interference, divide the normal state, the crowded state and the severe crowded state, generate state update data, output to the monitoring platform to realize the traceable state feedback, and further realize the multi-dimensional data acquisition and provide accurate data support for data review.

Description

Technical Field

[0001] This invention relates to the field of passenger flow density monitoring technology, and more specifically, to a visual detection and early warning method and system for passenger flow density monitoring. Background Technology

[0002] In densely populated areas such as subway stations, train stations, and airports, sudden surges in passenger flow not only easily lead to stampedes, reduced traffic efficiency, and operational safety issues such as capacity mismatch, but also cause a sharp increase in cooling or ventilation energy consumption due to higher population density, significantly increasing the air conditioning load in densely populated areas. Therefore, passenger flow density monitoring based on computer vision and sensor technology is of great application value. It can count the number of people in real time and accurately calculate the population density per unit area, providing managers with risk warnings and decision-making basis to assist in safety measures such as diversion and capacity adjustment. It can also provide data support for the dynamic control of air conditioning systems, enabling functions such as on-demand cooling and optimized unit operation strategies. In this way, while preventing safety hazards, it can significantly reduce the ineffective energy consumption of air conditioning systems and improve overall energy utilization efficiency.

[0003] An existing passenger flow detection method, disclosed in publication number CN120509617A, includes: collecting human movement direction data within a monitoring area using a sensor array, generating a time-series data stream after signal conditioning; initializing the passenger flow density network parameter population using chaotic mapping, optimizing the passenger flow density network parameter population using nonlinear decreasing pitch, and injecting random disturbances into the passenger flow density network. However, this patent relies on a dedicated sensor array, making it incompatible with existing network camera monitoring systems. Its algorithm model is overly complex, with high computational complexity, making real-time processing difficult on ordinary devices. Furthermore, it suffers from a rigid and difficult-to-trace early warning feedback mechanism and limited data collection dimensions, hindering accurate retrospective analysis.

[0004] Based on this, this application proposes a visual detection and early warning method and system for passenger flow density monitoring. Summary of the Invention

[0005] This invention aims to overcome the problems of existing passenger flow detection methods being incompatible with established network camera monitoring systems, having high computational complexity in algorithm models, having rigid and difficult-to-trace early warning feedback mechanisms, and having limited data collection dimensions that prevent accurate review.

[0006] To achieve the above objectives, the present invention provides a visual detection and early warning method and system for passenger flow density monitoring.

[0007] According to a first aspect of the present invention, a visual detection and early warning method for passenger flow density monitoring is provided, comprising: Based on the matching interface, the system receives visual data continuously uploaded by users and reads the visual data frame by frame according to the preset sampling interval to generate a visual data matrix including several frames of images. The visual data matrix is ​​input into a pre-trained visual detection model to output the number of torso and torso confidence, as well as the number of heads and head confidence in each frame of the image. The number of torso and head is weighted and fused. A benchmark term is defined based on the sum of the number of torso and head, or the larger value of the number of torso and head is selected, and a correction term is defined based on the smaller value. The number of people in each frame of the image is determined by the proportional deviation between the benchmark term and the correction term. The passenger flow density of each frame of the image is calculated by combining the pre-acquired activity area of ​​the monitoring area. The passenger flow density of each frame of the image is compared with the preset high-order congestion threshold, and an alarm sign or warning sign is generated for each frame of the image. When the cumulative number of alarm signs or warning signs reaches the alarm switching threshold, the image is switched to a warning or alarm state. Based on preset low-order congestion thresholds and preset high-order congestion thresholds, the passenger flow density of each frame of image is divided into normal state, congested state and severely congested state, and state update data is generated and output to the monitoring platform for corresponding actions.

[0008] Optionally, the method of receiving continuously uploaded visual data from the user based on the matching interface and reading the visual data frame by frame according to a preset sampling interval to generate a visual data matrix including several frames of images specifically includes: Based on the matching interface of the monitoring system compatible with users, it receives visual data continuously uploaded by users, including videos and / or images; Visual data is read frame by frame according to a sampling interval of 0.5s to 5s, the image sequence is extracted and converted into a storage matrix format to generate a visual data matrix including several frames of images.

[0009] Optionally, the step of inputting the visual data matrix into a pre-trained visual detection model to output the number of torsos and torso confidence scores, as well as the number of heads and head confidence scores in each frame of the image, specifically includes: Based on the pre-acquired training set, the YOLO model is iteratively trained to form a visual detection model; The visual data matrix is ​​input into a pre-trained visual detection model to output the number of torsos and torso confidence scores, as well as the number of heads and head confidence scores in each frame of the image.

[0010] Optionally, the weighted fusion processing of the number of torsos and heads, either based on the sum of the number of torsos and heads, or by selecting the larger value of the number of torsos and heads to define a benchmark term and the smaller value to define a correction term, determines the number of people in each frame of the image based on the proportional deviation between the benchmark term and the correction term, and calculates the passenger flow density of each frame of the image in conjunction with the pre-acquired activity area of ​​the monitoring region, specifically including: Determine the number of people in each frame of the image based on the number of torsos and heads: If the number of torsos and heads is both 0, then the number of people is 0. If the number of torsos alone is 0 or the number of heads alone is 0, then the number of people is the sum of the number of torsos and the number of heads. If neither the number of torso nor the number of heads is 0, then the larger value between the number of torso and the number of heads is selected to define the baseline term, and the smaller value is defined as the correction term. The number of people in each frame of the image is determined by the proportional deviation between the baseline term and the correction term. The passenger flow density for each frame of image is calculated based on the number of people and the pre-acquired activity area of ​​the monitoring zone.

[0011] Optionally, if both the number of torsos and the number of heads are not zero, the larger value between the number of torsos and the number of heads is selected to define a baseline term, and the smaller value is defined as a correction term. The number of people in each frame is determined by the proportional deviation between the baseline term and the correction term, specifically including: Based on benchmark terms and corrections Calculate the proportional deviation factor : and dynamic weights : In the formula: A represents the weighting coefficient. B represents the reference attenuation coefficient; Then determine the number of people P in each frame of the image: Optionally, the step of comparing the passenger flow density of each frame of image with a preset high-order congestion threshold and generating an alarm or warning indicator for each frame of image, so as to switch to a warning or alarm state when the cumulative number of alarm or warning indicators reaches the alarm switching threshold, specifically includes: The passenger flow density of each frame of the image is compared with a preset high-order congestion threshold. When the passenger flow density exceeds a preset high-order congestion threshold, an alarm indicator is generated for the current frame image. When the passenger flow density is less than the preset high-order congestion threshold, a warning sign is generated for the current frame image; When the cumulative number of frames for alarm or warning indicators reaches the alarm switching threshold, the system switches to warning or alarm status.

[0012] Optionally, based on preset low-order congestion thresholds and preset high-order congestion thresholds, the passenger flow density of each frame of image is divided into normal state, congested state, and severely congested state, and state update data is generated and output to the monitoring platform for corresponding processing, specifically including: Obtain the passenger flow density of the current frame image. If the passenger flow density of the current frame image is less than a preset low-order congestion threshold, the passenger flow density of the current frame image is classified as normal. If the passenger flow density of the current frame image is greater than a preset low-order congestion threshold but less than a preset high-order congestion threshold, the passenger flow density of the current frame image is classified as a congested state. If the passenger flow density of the current frame image is greater than the preset high-order congestion threshold, the passenger flow density of the current frame image is classified as a severely congested state. Status update data is generated based on the status division of the current frame image and output to the monitoring platform for corresponding actions.

[0013] Optionally, the step of generating state update data based on the state division of the current frame image and outputting it to the monitoring platform for corresponding processing includes: Based on the state division of the current frame image, the current frame image, along with its number of people, passenger density, alarm status, and passenger congestion status, is output to the Web UI of the monitoring platform. The status update data includes the current frame image, along with its number of people, passenger density, alarm status, and passenger congestion status. The corresponding monitoring status background color is displayed on the Web UI based on the current frame image and its number of people, passenger density, alarm status, and passenger congestion status.

[0014] Optionally, in addition to displaying the corresponding monitoring status background color in the Web UI based on the current frame image and its number of people, passenger density, alarm status, and passenger congestion status, the method also includes: The system can store the received current frame image along with its number of people, passenger density, alarm status, and passenger congestion status, and configure corresponding timestamps. Alternatively, it can store the current frame image along with its number of people, passenger density, alarm status, and passenger congestion status and configure corresponding timestamps only when the alarm status changes.

[0015] According to a second aspect of the present invention, a visual detection and early warning system for passenger flow density monitoring is also provided, for implementing the visual detection and early warning method for passenger flow density monitoring as described in any of the preceding claims, the visual detection and early warning system comprising: Data acquisition module: It can receive visual data continuously uploaded by users based on the matching interface, and read the visual data frame by frame according to the preset sampling interval to generate a visual data matrix including several frames of images. Model detection module: Used to input the visual data matrix into the pre-trained visual detection model to output the number of torso and torso confidence, as well as the number of heads and head confidence in each frame of the image; Passenger flow density calculation module: used to perform weighted fusion processing on the number of torso and head, to define a benchmark term based on the sum of the number of torso and head, or to define a correction term based on the larger value of the number of torso and head, and to determine the number of people in each frame of the image by the proportional deviation between the benchmark term and the correction term, and to calculate the passenger flow density of each frame of the image by combining the pre-acquired activity area of ​​the monitoring area. The alarm status determination module can compare the passenger flow density of each frame of image with the preset high-order congestion threshold and generate an alarm icon or warning icon for each frame of image. When the cumulative number of alarm icons or warning icons reaches the alarm switching threshold, the system switches to a warning or alarm status. Passenger congestion status classification module: Based on preset low-level congestion thresholds and preset high-level congestion thresholds, it can classify the passenger flow density of each frame of image into normal state, congested state, and severely congested state, and generate status update data to output to the monitoring platform for corresponding handling.

[0016] Compared with the prior art, the present invention has the following beneficial effects: The visual detection and early warning method for passenger flow density monitoring proposed in this invention receives continuously uploaded visual data from users through a matching interface to achieve compatibility with existing monitoring systems. It then outputs the number of torsos and their confidence scores, as well as the number of heads and their confidence scores, through a pre-trained visual detection model. Weighted fusion calculations are performed based on the proportional deviation between defined baseline and correction terms, thereby reducing the computational complexity of the algorithm. Furthermore, alarm switching is triggered by accumulating frame counts to achieve flexible early warnings that avoid instantaneous interference. Additionally, by classifying normal, crowded, and severely crowded states and generating state update data, which is output to the monitoring platform, traceable state feedback is achieved. This enables multi-dimensional data collection and provides accurate data support for data review.

[0017] The visual detection and early warning system for passenger flow density monitoring proposed in this invention belongs to the same general inventive concept as the visual detection and early warning method for passenger flow density monitoring in this invention. It should at least have the same technical effect as the visual detection and early warning method for passenger flow density monitoring. Therefore, this invention will not elaborate further here.

[0018] As can be seen from the above, the technical solution of the present invention can effectively solve the problems that existing passenger flow detection methods are incompatible with the established network camera monitoring system, have high computational complexity in algorithm models, have rigid and difficult-to-trace early warning feedback mechanisms, and have limited data collection dimensions that make accurate review impossible.

[0019] Other features and advantages of the present invention will be described in detail in the following detailed description section. Attached Figure Description

[0020] Figure 1 This is a flowchart illustrating a visual detection and early warning method for passenger flow density monitoring according to an embodiment of the present invention. Figure 2 This is a schematic diagram of a visual detection and early warning system for passenger flow density monitoring according to an embodiment of the present invention. Detailed Implementation

[0021] To enable those skilled in the art to more fully understand the technical solutions of the present invention, exemplary embodiments of the present invention will be described more comprehensively and in detail below with reference to the accompanying drawings. Obviously, the one or more embodiments of the present invention described below are merely one or more specific ways to implement the technical solutions of the present invention, and are not exhaustive. It should be understood that other ways belonging to a general inventive concept can be used to implement the technical solutions of the present invention, and should not be limited to the embodiments described exemplary. Based on one or more embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0022] Reference Figure 1 The present invention provides a visual detection and early warning method for passenger flow density monitoring, comprising: Step S1: Receive the visual data continuously uploaded by the user based on the matching interface, and read the visual data frame by frame according to the preset sampling interval to generate a visual data matrix including several frames of images. Step S2: Input the visual data matrix into the pre-trained visual detection model to output the number of torso and torso confidence, as well as the number of heads and head confidence in each frame of the image. Step S3: Perform weighted fusion processing on the number of torso and head, and define a benchmark term based on the sum of the number of torso and head, or select the larger value of the number of torso and head and define a correction term based on the smaller value. Determine the number of people in each frame of the image by the proportional deviation between the benchmark term and the correction term, and calculate the passenger flow density of each frame of the image in combination with the pre-acquired activity area of ​​the monitoring area. Step S4: Compare the passenger flow density of each frame image with the preset high-order congestion threshold, and generate an alarm icon or warning icon for each frame image, so that when the cumulative number of alarm icons or warning icons reaches the alarm switching threshold, the system switches to warning or alarm status. Step S5: Based on the preset low-order congestion threshold and the preset high-order congestion threshold, the passenger flow density of each frame image is divided into normal state, congested state and severe congestion state, and the state update data is generated and output to the monitoring platform to perform corresponding actions.

[0023] The visual detection and early warning method for passenger flow density monitoring proposed in this invention receives continuously uploaded visual data from users through a matching interface to achieve compatibility with existing monitoring systems. It then outputs the number of torsos and their confidence scores, as well as the number of heads and their confidence scores, through a pre-trained visual detection model. Weighted fusion calculations are performed based on the proportional deviation between defined baseline and correction terms, thereby reducing the computational complexity of the algorithm. Furthermore, alarm switching is triggered by accumulating frame counts to achieve flexible early warnings that avoid instantaneous interference. Additionally, by classifying normal, crowded, and severely crowded states and generating state update data, which is output to the monitoring platform, traceable state feedback is achieved. This enables multi-dimensional data collection and provides accurate data support for data review.

[0024] In one embodiment, in step S1, visual data continuously uploaded by the user is received based on the matching interface, and the visual data is read frame by frame according to a preset sampling interval to generate a visual data matrix including several frames of images, specifically including: Step S11: Based on the matching interface of the monitoring system compatible with users, receive the visual data continuously uploaded by users, wherein the visual data includes videos and / or images; Step S12: Read the visual data frame by frame according to the sampling interval of 0.5s~5s, extract the image sequence and convert it into a storage matrix format to generate a visual data matrix including several frames of images.

[0025] Specifically, the matching interface allows users to directly reuse their existing monitoring systems without the need for additional dedicated sensor arrays, achieving zero hardware cost compatibility and significantly reducing the application cost of passenger flow density monitoring. Furthermore, the sampling time can be flexibly adjusted within a 0.5s to 5s sampling interval according to the real-time requirements of the monitoring scenario. The visual data matrix provides a standard input format for subsequent monitoring implementation, thereby ensuring the smoothness and accuracy of subsequent data processing.

[0026] It is worth noting that the matching interface can be the VideoCapture interface in the OpenCV library. OpenCV provides a general interface for reading video streams or local video files. It can connect to and read video frames of a network camera monitoring system built on the Real Time Streaming Protocol (RTSP) and Real Time Messaging Protocol (RTMP), thereby enabling the reading and decoding of the user's monitoring system, extracting image sequences and converting them into a storage matrix format.

[0027] In one specific embodiment, the preset sampling interval is 1 second, and one frame of image is sampled each time.

[0028] In one embodiment, in step S2, the visual data matrix is ​​input into a pre-trained visual detection model to output the number of torsos and torso confidence scores, as well as the number of heads and head confidence scores in each frame of the image. Specifically, this includes: Step S21: Based on the pre-acquired training set, iteratively train the single-stage real-time object detection model (You Only LookOnce, YOLO) to form a visual detection model; Step S22: Input the visual data matrix into the pre-trained visual detection model to output the number of torso and torso confidence, as well as the number of heads and head confidence in each frame image.

[0029] Specifically, the formation of the visual detection model can avoid the waste of computing power caused by the recognition of the general YOLO detection model, and can accurately output the number of torsos and the confidence of torsos, as well as the number of heads and the confidence of heads for the user's monitoring scenario, thereby providing reliable data support in two dimensions for subsequent weighted fusion counting.

[0030] It is worth noting that the visual detection model formed by the YOLO detection model has the characteristics of single-stage detection and can achieve millisecond-level inference, balancing detection efficiency and accuracy, thus adapting to the real-time sampling needs of users' existing monitoring systems.

[0031] Furthermore, the inference parameters of the visual detection model include: confidence threshold and intersection over union (IOU) non-maximum suppression threshold. Among them, the confidence threshold is the set minimum confidence standard. Only when the confidence score of the torso or head target is greater than 0.3 will it be judged as a valid detection result. Fuzzy recognitions below 0.3 will be directly filtered out.

[0032] The Intersection over Union (IOU) threshold is the standard for determining duplicate detections. When the IOU of two detection boxes is greater than 0.5, they are identified as the same target, and only the detection box with the highest confidence is retained to remove redundant results, thereby obtaining accurate and non-duplicative detection data for the torso and head.

[0033] In one specific embodiment, step S21: based on the pre-acquired training set, the YOLO detection model is iteratively trained to form a visual detection model, specifically including: Step S211: Collect the user's visual data in advance, extract each frame of the visual data, and label the torso and head in each frame of the image separately to generate a training set; Step S212: Output the training set into the YOLO detection model and iteratively train it to form a visual detection model that can output the number of torso and torso confidence, as well as the number of heads and head confidence.

[0034] Specifically, by collecting visual data from actual user monitoring scenarios and specifically labeling targets in the torso and head categories, a training set is generated and the YOLO detection model is iteratively trained. This enables the resulting visual detection model to accurately adapt to the user's monitoring perspective, environmental characteristics, and crowd gathering patterns, avoiding the problem of missed or false detections of targets in specific scenarios by the general YOLO detection model.

[0035] In one embodiment, in step S3, a weighted fusion process is performed on the number of torsos and heads. A baseline term is defined based on the sum of the torso and head counts, or by selecting the larger value between the torso and head counts, and a correction term is defined based on the smaller value. The number of people in each frame is determined by the proportional deviation between the baseline and correction terms. The passenger flow density of each frame is then calculated by combining this with the pre-acquired activity area of ​​the monitoring region. Specifically, this includes: Step S31: Determine the number of people in each frame of the image based on the number of torsos and heads. If the number of torsos and heads is both 0, then the number of people is 0. If the number of torsos alone is 0 or the number of heads alone is 0, then the number of people is the sum of the number of torsos and the number of heads. If neither the number of torso nor the number of heads is 0, then the larger value between the number of torso and the number of heads is selected to define the baseline term, and the smaller value is defined as the correction term. The number of people in each frame of the image is determined by the proportional deviation between the baseline term and the correction term. Step S32: Calculate the passenger flow density for each frame of image based on the number of people and the pre-acquired activity area of ​​the monitoring area.

[0036] In one specific embodiment, in step S31, if both the number of torsos and the number of heads are not zero, the larger value between the number of torsos and the number of heads is selected to define a baseline term, and the smaller value is defined as a correction term. The number of people in each frame of the image is determined by the proportional deviation between the baseline term and the correction term, specifically including: Based on benchmark terms and corrections Calculate the proportional deviation factor : and dynamic weights : In the formula: A represents the weighting coefficient. B represents the reference attenuation coefficient; Then determine the number of people P in each frame of the image: Specifically, by using weighted fusion processing to obtain the number of people in each frame of the image, the directness and accuracy of the number calculation can be guaranteed in simple scenarios such as no one or only a single human body part being visible. In complex scenarios where the number of torsos and heads are present at the same time, the weights are adjusted based on the ratio deviation between the two. The larger value of the number of torsos and heads is selected to define the benchmark term, and the smaller value is defined as the correction term. This avoids the problem of missed detection when single-dimensional detection is obscured by crowds. At the same time, the influence of the correction term is controlled by the weight coefficient A and the benchmark attenuation coefficient B, so that the number calculation results can be adapted to different crowd density scenarios, significantly improving the accuracy and scenario adaptability of crowd counting.

[0037] Furthermore, the weighting coefficient A ranges from 0.1 to 0.9, and the baseline attenuation coefficient B ranges from 0.01 to 0.1. If the number of people P is not an integer, it is rounded to the nearest integer.

[0038] It is worth noting that the specific values ​​of the weight coefficient A and the baseline attenuation coefficient B are obtained based on the offline grid search parameter optimization method. First, a validation dataset with labeled real-number data is collected for the specific application scenario. Then, within the range of values ​​for the weight coefficient A and the baseline attenuation coefficient B, all possible parameter combinations are iterated. Each set of parameters is substituted into the weighted fusion counting formula, and the images in the validation dataset are inferred and calculated using a visual detection model. The mean absolute error is used as the evaluation metric, and the parameter combination corresponding to the minimum mean absolute error is selected as the weight coefficient A and the baseline attenuation coefficient B for the current frame image. The specific principle of the offline grid search parameter optimization method is existing technology, and will not be elaborated further here.

[0039] In one specific embodiment, the weighting coefficient A is 0.25 and the baseline attenuation coefficient B is 0.01.

[0040] In one specific embodiment, in step S32, the passenger flow density of each frame of image is calculated based on the number of people and the pre-acquired activity area of ​​the monitoring area, specifically including: The passenger flow density for each frame of image is calculated based on the ratio of the number of people to the pre-acquired activity area of ​​the monitoring area.

[0041] Specifically, the activity area of ​​the monitoring area is configured based on the Region of Interest (ROI) method, which determines the actual area of ​​the target area from the monitoring screen in the user's monitoring system.

[0042] By calculating passenger flow density using the ratio of the number of people to the activity area of ​​the monitored area, the calculation logic is kept simple and efficient, without the need for complex spatial conversions or nonlinear calculations. This makes it suitable for the real-time processing needs of ordinary equipment. Furthermore, by calibrating the actual area of ​​the target area, it avoids density deviations caused by relying solely on the pixel area of ​​the image. This allows the passenger flow density results to accurately reflect the actual congestion level of the monitored area, providing a reliable quantitative basis for subsequent congestion status classification and delayed early warning.

[0043] In one embodiment, in step S4, the passenger flow density of each frame of image is compared with a preset high-order congestion threshold, and an alarm icon or warning icon is generated for each frame of image. When the cumulative number of alarm icons or warning icons reaches the alarm switching threshold, the system switches to a warning or alarm state. Specifically, this includes: Step S41: Compare the passenger flow density of each frame with a preset high-order congestion threshold. When the passenger flow density exceeds a preset high-order congestion threshold, an alarm indicator is generated for the current frame image. When the passenger flow density is less than the preset high-order congestion threshold, a warning sign is generated for the current frame image; Step S42: When the cumulative number of frames of alarm or warning indicators reaches the alarm switching threshold, switch to warning or alarm status.

[0044] Specifically, alarm and warning signs are distinguished by a preset high-order congestion threshold, and the state switching condition is based on the cumulative number of frames reaching the alarm switching threshold. This not only enables the determination of congestion status by using the preset high-order congestion threshold and alarm switching threshold, allowing managers to perceive congestion trends in advance and make emergency preparations, but also avoids false alarm switching caused by fluctuations in passenger flow density in a single frame image through the cumulative frame confirmation mechanism, effectively eliminating invalid alarms caused by interference factors such as detection noise and instantaneous occlusion.

[0045] In one specific embodiment, the alarm switching threshold is 5 frames.

[0046] In one embodiment, in step S5, based on a preset low-order congestion threshold and a preset high-order congestion threshold, the passenger flow density of each frame of image is divided into normal state, congested state, and severely congested state, and state update data is generated and output to the monitoring platform to perform corresponding actions, specifically including: Step S51: Obtain the passenger flow density of the current frame image. If the passenger flow density of the current frame image is less than a preset low-order congestion threshold, the passenger flow density of the current frame image is classified as normal. If the passenger flow density of the current frame image is greater than a preset low-order congestion threshold but less than a preset high-order congestion threshold, the passenger flow density of the current frame image is classified as a congested state. If the passenger flow density of the current frame image is greater than the preset high-order congestion threshold, the passenger flow density of the current frame image is classified as a severely congested state. Step S52: Generate state update data based on the state division of the current frame image and output it to the monitoring platform to perform corresponding actions.

[0047] Specifically, by classifying status and outputting the status data to the monitoring platform, refined hierarchical management of passenger congestion can be achieved. This allows monitoring personnel to intuitively distinguish different levels of congestion and take proactive measures such as guidance and diversion to prevent escalation into severe congestion. Furthermore, it provides standardized status data input to the monitoring platform, triggering corresponding level-based actions. The standardized status classification also provides a unified dimension for subsequent historical data analysis, facilitating the summarization of time-based distribution patterns for different congestion levels and aiding in the development of more precise passenger flow control strategies.

[0048] The preset low-order congestion threshold and preset high-order congestion threshold are based on the recommended values ​​for platform personnel ρ = 0.33 m specified in the mandatory national standard GB 50157-2013 "Code for Design of Metro". 2 / person~0.75 m 2 The higher-order congestion threshold selected by the individual is 0.5 m in this embodiment. 2 / person, the low-order crowding threshold is 0.66 m 2 / people.

[0049] In one specific embodiment, in step S52, state update data is generated based on the state division of the current frame image and output to the monitoring platform to perform corresponding processing, specifically including: Step S521: Based on the state division of the current frame image, output the current frame image and its number of people, passenger flow density, alarm status and passenger flow congestion status to the web user interface (WebUI) of the monitoring platform. The status update data includes the current frame image and its number of people, passenger flow density, alarm status and passenger flow congestion status. Step S522: Display the corresponding monitoring status background color on the Web UI based on the current frame image and its number of people, passenger density, alarm status, and passenger congestion status.

[0050] Specifically, the current frame image, along with its status update data consisting of the number of people, passenger density, alarm status, and passenger congestion status, is output to the monitoring platform via the HTTP network transmission protocol and displayed on the Web UI, thereby enabling monitoring personnel to monitor the passenger flow situation in the monitoring area in a timely manner.

[0051] It is worth noting that the background color of the monitoring status can be the background color of the Web UI or the color of the status badge in the monitoring software, which is pre-defined based on the number of people, passenger density, alarm status and passenger congestion status. The specific settings can be customized according to user needs, and this invention does not limit them.

[0052] In one embodiment, after displaying the corresponding monitoring status background color on the Web UI based on the current frame image and its number of people, passenger flow density, alarm status, and passenger congestion status in step S522, the method further includes: When the alarm status is switched to the alarm status, the monitoring platform's Web Speech API Synthesis Interface (Web Speech API) is invoked to automatically broadcast a voice prompt indicating a severe congestion situation.

[0053] Specifically, the Web Speech API is invoked to enhance the alarm notification effect through voice broadcast when an alarm is triggered, helping monitoring personnel to quickly perceive emergency situations and improve emergency response efficiency.

[0054] In one embodiment, in step S522, while displaying the corresponding monitoring status background color on the Web UI based on the current frame image and its number of people, passenger flow density, alarm status, and passenger congestion status, the method further includes: The system can store the received current frame image along with its number of people, passenger density, alarm status, and passenger congestion status, and configure corresponding timestamps. Alternatively, it can store the current frame image along with its number of people, passenger density, alarm status, and passenger congestion status and configure corresponding timestamps only when the alarm status changes.

[0055] Specifically, the status update data stored in this embodiment can be configured and stored with timestamps for each frame of image, as well as its number of people, passenger flow density, alarm status, and passenger flow congestion status. Alternatively, timestamps can be configured and stored only when the alarm status switches to the alarm state. At the same time, each frame of image stored is named in PNG format with timestamps, and the number of people, passenger flow density, alarm status, and passenger flow congestion status are written to a CSV log to distinguish it from the storage of status update data in non-alarm states, thereby facilitating rapid retrieval and source tracing analysis by monitoring personnel.

[0056] To meet the data storage needs of different scenarios: full storage enables refined historical trend analysis, making it convenient for managers to review changes in passenger flow at any time period; alarm-only storage can reduce unnecessary data occupation, focus on the evidence retention needs of key alarms, and ensure the standardization of stored data by aligning with unified archiving formats and timestamp rules.

[0057] Furthermore, this embodiment can retrieve status update data based on the monitoring platform, and statistically analyze the alarm frequency, average passenger flow density, and maximum passenger flow density of the monitored area according to the timestamp, or display the trend curve of the number of people changing according to the selected time period, and present it in the form of charts through the Web UI, thereby supporting review and decision-making functions such as alarm frequency statistics and viewing of the number of people trends.

[0058] Accordingly, such as Figure 2 As shown, embodiments of the present invention also provide a visual detection and early warning system for passenger flow density monitoring, used to implement the visual detection and early warning method for passenger flow density monitoring as described in any of the above embodiments. The visual detection and early warning system includes: Data acquisition module: It can receive visual data continuously uploaded by users based on the matching interface, and read the visual data frame by frame according to the preset sampling interval to generate a visual data matrix including several frames of images. Model detection module: Used to input the visual data matrix into the pre-trained visual detection model to output the number of torso and torso confidence, as well as the number of heads and head confidence in each frame of the image; Passenger flow density calculation module: used to perform weighted fusion processing on the number of torso and head, to define a benchmark term based on the sum of the number of torso and head, or to define a correction term based on the larger value of the number of torso and head, and to determine the number of people in each frame of the image by the proportional deviation between the benchmark term and the correction term, and to calculate the passenger flow density of each frame of the image by combining the pre-acquired activity area of ​​the monitoring area. The alarm status determination module can compare the passenger flow density of each frame of image with the preset high-order congestion threshold and generate an alarm icon or warning icon for each frame of image. When the cumulative number of alarm icons or warning icons reaches the alarm switching threshold, the system switches to a warning or alarm status. Passenger congestion status classification module: Based on preset low-level congestion thresholds and preset high-level congestion thresholds, it can classify the passenger flow density of each frame of image into normal state, congested state, and severely congested state, and generate status update data to output to the monitoring platform for corresponding handling.

[0059] The visual detection and early warning system for passenger flow density monitoring proposed in this invention belongs to the same general inventive concept as the visual detection and early warning method for passenger flow density monitoring in this invention. It should at least have the same technical effect as the visual detection and early warning method for passenger flow density monitoring. Therefore, this invention will not elaborate further here.

[0060] While one or more embodiments of the present invention have been described above, those skilled in the art will recognize that the present invention can be implemented in any other form without departing from its spirit and scope. Therefore, the embodiments described above are illustrative and not restrictive, and many modifications and substitutions will be apparent to those skilled in the art without departing from the spirit and scope of the invention as defined in the appended claims.

Claims

1. A visual detection and early warning method for passenger flow density monitoring, characterized in that, include: Based on the matching interface, the system receives visual data continuously uploaded by users and reads the visual data frame by frame according to the preset sampling interval to generate a visual data matrix including several frames of images. The visual data matrix is ​​input into a pre-trained visual detection model to output the number of torso and torso confidence, as well as the number of heads and head confidence in each frame of the image. The number of torso and head is weighted and fused. A benchmark term is defined based on the sum of the number of torso and head, or the larger value of the number of torso and head is selected, and a correction term is defined based on the smaller value. The number of people in each frame of the image is determined by the proportional deviation between the benchmark term and the correction term. The passenger flow density of each frame of the image is calculated by combining the pre-acquired activity area of ​​the monitoring area. The passenger flow density of each frame of the image is compared with the preset high-order congestion threshold, and an alarm sign or warning sign is generated for each frame of the image. When the cumulative number of alarm signs or warning signs reaches the alarm switching threshold, the image is switched to a warning or alarm state. Based on preset low-order congestion thresholds and preset high-order congestion thresholds, the passenger flow density of each frame of image is divided into normal state, congested state and severely congested state, and state update data is generated and output to the monitoring platform for corresponding actions.

2. The visual detection and early warning method for passenger flow density monitoring according to claim 1, characterized in that, The process of receiving continuously uploaded visual data from users via a matching interface and reading the visual data frame by frame according to a preset sampling interval to generate a visual data matrix including several frames of images specifically includes: Based on the matching interface of the monitoring system compatible with users, it receives visual data continuously uploaded by users, including videos and / or images; Visual data is read frame by frame according to a sampling interval of 0.5s to 5s, the image sequence is extracted and converted into a storage matrix format to generate a visual data matrix including several frames of images.

3. The visual detection and early warning method for passenger flow density monitoring according to claim 2, characterized in that, The step of inputting the visual data matrix into a pre-trained visual detection model to output the number of torsos and torso confidence scores, as well as the number of heads and head confidence scores in each frame of the image, specifically includes: Based on the pre-acquired training set, the YOLO model is iteratively trained to form a visual detection model; The visual data matrix is ​​input into a pre-trained visual detection model to output the number of torsos and torso confidence scores, as well as the number of heads and head confidence scores in each frame of the image.

4. The visual detection and early warning method for passenger flow density monitoring according to claim 3, characterized in that, The weighted fusion processing of the number of torsos and heads, using either the sum of the torso and head counts or the larger value of the torso and head counts as a baseline term and the smaller value as a correction term, determines the number of people in each frame of the image based on the proportional deviation between the baseline and correction terms. This is combined with the pre-acquired activity area of ​​the monitoring region to calculate the passenger flow density of each frame of the image. Specifically, this includes: Determine the number of people in each frame of the image based on the number of torsos and heads: If the number of torsos and heads is both 0, then the number of people is 0. If the number of torsos alone is 0 or the number of heads alone is 0, then the number of people is the sum of the number of torsos and the number of heads. If neither the number of torso nor the number of heads is 0, then the larger value between the number of torso and the number of heads is selected to define the baseline term, and the smaller value is defined as the correction term. The number of people in each frame of the image is determined by the proportional deviation between the baseline term and the correction term. The passenger flow density for each frame of image is calculated based on the number of people and the pre-acquired activity area of ​​the monitoring zone.

5. The visual detection and early warning method for passenger flow density monitoring according to claim 4, characterized in that, If both the number of torsos and the number of heads are not zero, then the larger value between the number of torsos and the number of heads is selected to define a baseline term, and the smaller value is defined as a correction term. The number of people in each frame of the image is determined by the proportional deviation between the baseline term and the correction term, specifically including: Based on benchmark terms and corrections Calculate the proportional deviation factor : and dynamic weights : In the formula: A represents the weighting coefficient. B represents the reference attenuation coefficient; Then determine the number of people P in each frame of the image:

6. The visual detection and early warning method for passenger flow density monitoring according to claim 5, characterized in that, The step of comparing the passenger flow density of each frame of image with a preset high-order congestion threshold and generating an alarm or warning indicator for each frame of image, so as to switch to a warning or alarm state when the cumulative number of alarm or warning indicators reaches the alarm switching threshold, specifically includes: The passenger flow density of each frame of the image is compared with a preset high-order congestion threshold. When the passenger flow density exceeds a preset high-order congestion threshold, an alarm indicator is generated for the current frame image. When the passenger flow density is less than the preset high-order congestion threshold, a warning sign is generated for the current frame image; When the cumulative number of frames for alarm or warning indicators reaches the alarm switching threshold, the system switches to warning or alarm status.

7. The visual detection and early warning method for passenger flow density monitoring according to claim 6, characterized in that, Based on preset low-order congestion thresholds and preset high-order congestion thresholds, the passenger flow density of each frame of image is divided into normal state, congested state, and severely congested state, and state update data is generated and output to the monitoring platform for corresponding actions, specifically including: Obtain the passenger flow density of the current frame image. If the passenger flow density of the current frame image is less than a preset low-order congestion threshold, the passenger flow density of the current frame image is classified as normal. If the passenger flow density of the current frame image is greater than a preset low-order congestion threshold but less than a preset high-order congestion threshold, the passenger flow density of the current frame image is classified as a congested state. If the passenger flow density of the current frame image is greater than the preset high-order congestion threshold, the passenger flow density of the current frame image is classified as a severely congested state. Status update data is generated based on the status division of the current frame image and output to the monitoring platform for corresponding actions.

8. The visual detection and early warning method for passenger flow density monitoring according to claim 7, characterized in that, The process of generating state update data based on the state division of the current frame image and outputting it to the monitoring platform for corresponding processing includes: Based on the state division of the current frame image, the current frame image, along with its number of people, passenger density, alarm status, and passenger congestion status, is output to the Web UI of the monitoring platform. The status update data includes the current frame image, along with its number of people, passenger density, alarm status, and passenger congestion status. The corresponding monitoring status background color is displayed on the Web UI based on the current frame image and its number of people, passenger density, alarm status, and passenger congestion status.

9. The visual detection and early warning method for passenger flow density monitoring according to claim 8, characterized in that, The method of displaying the corresponding monitoring status background color in the Web UI based on the current frame image and its number of people, passenger density, alarm status, and passenger congestion status also includes: The system can store the received current frame image along with its number of people, passenger density, alarm status, and passenger congestion status, and configure corresponding timestamps. Alternatively, it can store the current frame image along with its number of people, passenger density, alarm status, and passenger congestion status and configure corresponding timestamps only when the alarm status changes.

10. A visual detection and early warning system for monitoring passenger flow density, characterized in that, For implementing the visual detection and early warning method for passenger flow density monitoring as described in any one of claims 1-9, the visual detection and early warning system includes: Data acquisition module: It can receive visual data continuously uploaded by users based on the matching interface, and read the visual data frame by frame according to the preset sampling interval to generate a visual data matrix including several frames of images. Model detection module: Used to input the visual data matrix into the pre-trained visual detection model to output the number of torso and torso confidence, as well as the number of heads and head confidence in each frame of the image; Passenger flow density calculation module: used to perform weighted fusion processing on the number of torso and head, to define a benchmark term based on the sum of the number of torso and head, or to define a correction term based on the larger value of the number of torso and head, and to determine the number of people in each frame of the image by the proportional deviation between the benchmark term and the correction term, and to calculate the passenger flow density of each frame of the image by combining the pre-acquired activity area of ​​the monitoring area. The alarm status determination module can compare the passenger flow density of each frame of image with the preset high-order congestion threshold and generate an alarm icon or warning icon for each frame of image. When the cumulative number of alarm icons or warning icons reaches the alarm switching threshold, the system switches to a warning or alarm status. Passenger congestion status classification module: Based on preset low-level congestion thresholds and preset high-level congestion thresholds, it can classify the passenger flow density of each frame of image into normal state, congested state, and severely congested state, and generate status update data to output to the monitoring platform for corresponding handling.

Images