Real-time monitoring system and method for pedestrian density based on multi-source fusion and conservation constraints

Abstract

This invention discloses a real-time crowd density monitoring system and method based on multi-source fusion and conservation constraints; belonging to the field of crowd flow and density monitoring technology; the system is based on deployed multi-source heterogeneous sensors, and generates high-precision entry counts through cross-validation and confidence fusion, providing globally consistent density data for downstream emergency evacuation systems; multi-source sensor cross-validation and self-calibration statistics realize automatic fault detection and performance calibration; a global density consistency constraint inference technology based on the law of crowd flow conservation outputs a globally self-consistent regional density distribution; inter-regional crowd flow transfer probability matrix and dynamic density inference technology infer sensor blind zone density and provide short-term prediction; multi-granularity scene adaptive monitoring mode switching technology automatically switches the acquisition frequency and sensor combination according to the downstream emergency level; standardized density data output interface technology realizes loose coupling between upstream and downstream systems; this invention improves the accuracy and real-time performance of crowd density monitoring in space.

Description

Technical Field

[0001] This invention belongs to the field of pedestrian flow and density monitoring technology, specifically relating to a real-time pedestrian flow density monitoring system and method based on multi-source fusion and conservation constraints. Background Technology

[0002] With the continuous advancement of urbanization in my country, large semi-enclosed, densely populated venues such as commercial complexes and shopping malls have become core carriers of urban public spaces and commercial services. Meanwhile, similar venues such as airport terminals, high-speed rail station waiting halls, stadiums, large general hospitals, campus public buildings, and office building complexes share common characteristics: complex spatial structures, multiple entrances and exits with interconnected zones, high personnel mobility, and significant instantaneous passenger flow aggregation effects. In daily operations, real-time and accurate pedestrian density monitoring data is the core foundational data for venue passenger flow management, business layout optimization, and operational efficiency improvement. In emergency scenarios such as fires and public safety incidents, globally consistent and highly reliable spatial distribution data of pedestrian density is the core input for emergency evacuation navigation systems to conduct route planning, hazard avoidance control, and joint emergency decision-making; its accuracy and consistency directly affect the life and property safety of people within the venue.

[0003] Currently, the technical solutions for monitoring pedestrian density in shopping malls can be mainly divided into three categories: pedestrian counting solutions based on a single sensor, pedestrian statistics solutions based on simple fusion of multiple sensors, and intelligent counting solutions based on video AI. Among these issues, there are limitations such as the inherent accuracy ceiling of a single sensor and the lack of cross-validation and self-calibration mechanisms; all currently available technologies rely on a single type of sensor for people counting or density estimation, and each sensor has inherent accuracy limitations; infrared sensors introduce occlusion errors when multiple people pass by in parallel; WiFi probes are affected by fluctuations in carry rate, leading to estimation bias; video AI experiences accuracy degradation in well-lit and densely populated scenes; existing multi-sensor solutions only perform simple averaging or take the maximum value, lacking a dynamic calibration mechanism based on cross-validation; and when the performance of a sensor drifts or malfunctions, the system cannot automatically detect and correct it, resulting in a continuous decline in data quality.

[0004] The density of each area is estimated independently, lacking global consistency constraints. Existing regional density estimation techniques allow each WiFi probe to independently estimate the number of people in its covered area, with no constraints between regional data. However, shopping malls, as semi-enclosed spaces, have a fundamental physical conservation relationship: the sum of the cumulative number of people entering and leaving each entrance (i.e., the net flow) should equal the total number of people remaining in the mall; the sum of the number of people in each area on each floor should equal the number of people on that floor; and the sum of the number of people on each floor should equal the total number of people in the mall. Existing technology does not utilize this conservation relationship, resulting in the sum of the number of people in each area potentially being much greater or less than the total number of people in the mall as counted at the entrances and exits, leading to data contradictions. This inconsistent data, when transmitted to the downstream evacuation system, will distort the basis for evacuation decisions.

[0005] Uneven WiFi probe coverage creates density blind spots; the complex internal structure of shopping malls limits WiFi probe deployment due to building structure and cost constraints, making it difficult to achieve uniform coverage across all areas; some areas, such as connecting passages, stairwells, and small shops, may lack sensor coverage, creating density blind spots; existing technologies lack the ability to infer blind spot density using spatial relationships between areas, and the spatial integrity of density data cannot be guaranteed.

[0006] The monitoring accuracy and frequency are fixed, and there is a lack of adaptive switching capability. Existing technologies use fixed data collection frequencies and accuracy levels, without distinguishing the differentiated needs of daily operations and emergency evacuation scenarios. A 15-minute granularity is sufficient for daily operations, but emergency evacuation scenarios require real-time density data accurate to the second level and down to the channel level. Existing technologies lack the ability to adaptively switch monitoring modes according to the emergency level of the downstream evacuation system, and cannot provide higher accuracy and higher frequency density data during emergencies. Summary of the Invention

[0007] This invention provides a real-time pedestrian density monitoring system and method based on multi-source fusion and conservation constraints. It employs three heterogeneous sensors to achieve multi-source sensing cross-validation and self-calibration statistical methods, improving the accuracy, anti-interference, and fault tolerance of pedestrian counting at entrances from the source. It pioneers a three-level constraint system based on the law of pedestrian flow conservation and a global density consistency constraint inference method, fundamentally eliminating the core pain points of spatiotemporal contradictions and cumulative error drift in existing technologies, providing self-consistent and reliable pedestrian density distribution data for downstream emergency evacuation navigation systems. Based on a directed graph modeling-based inter-regional pedestrian flow transfer probability matrix and dynamic density inference method, it addresses monitoring blind spots without sensor coverage by relying on adjacent areas. This invention enables accurate inference of density in blind spots using known density data and transition probability matrices, filling the gap in spatial coverage of density data. Simultaneously, it allows for short-term trend prediction of pedestrian density based on the transition probability matrix, enabling early assessment of passenger flow aggregation risks. Furthermore, this invention achieves adaptive switching of pedestrian flow monitoring modes across multiple granular scenarios, significantly improving the system's adaptability to various scenarios, including daily operations, passenger flow warnings, and emergency response. Finally, this invention designs a standardized density data output interface for downstream applications, allowing downstream systems to subscribe to density data at different levels and granularities on demand, without needing to concern themselves with the details of underlying sensor deployment and fusion algorithm implementation. This achieves loose coupling between upstream and downstream systems, significantly reducing the adaptation cost and reusability threshold of the solution.

[0008] To achieve the above-mentioned technical objectives, the present invention is implemented through the following technical solution: A real-time crowd density monitoring system based on multi-source fusion and conservation constraints includes: The multi-source sensor cross-validation and self-calibration people flow statistics module includes: The multi-source heterogeneous sensing and acquisition unit includes an entrance sensor group, an area sensor group, and an auxiliary sensor group for heterogeneous pedestrian counting; each sensor independently generates a count value and confidence level. The cross-validation fault detection unit performs pairwise cross-comparison verification on the counting results of the multi-source heterogeneous sensors, identifies sensors with abnormal deviations, and reduces their confidence level. The confidence-weighted fusion unit is used to perform normalized weighted fusion based on the real-time confidence of each sensor, and output the fusion count value and global confidence. The historical accuracy feedback self-calibration unit is used to periodically update the basic weights of each sensor based on the historical deviation statistics of the sensor itself, so as to achieve long-term self-calibration. The conservation constraint density inference module includes: The three-level conservation constraint modeling unit is used to establish a set of constraint equations for the conservation relationship of pedestrian flow in shopping malls at three levels: mall level, floor level, and area level. The entrance / exit anchor point constraint unit is used to calculate the net flow based on the entrance / exit fusion count results, and serves as the global total number of people anchor point constraint. The regional constraint optimization correction unit is used to solve for the globally consistent number of people corrected in each region by using the sensor estimates of each region as observations, under the conditions of satisfying conservation equality constraints and non-negativity constraints. The floor occupancy estimation unit is used to combine the net direct traffic flow from entrances and exits with the vertical traffic flow across floors, and to allocate the total number of people to each floor. The inter-regional population flow probability matrix and dynamic density inference module includes: A directed graph modeling unit is used to abstract each region in space into several nodes, the physical channels between regions into edges, and the nodes and edges together to construct a spatial topology. The online learning unit for transfer probability is used to continuously update the transfer probability matrix P[i][j] based on the statistics of cross-regional transfer events of devices, using an online learning method, which represents the probability that a person in region i will transfer to region j within a unit time window; The blind zone density inference unit is used to infer the population density in areas with no sensor coverage or weak coverage signals by utilizing the density of neighboring nodes and the transition probability. Short-term density prediction unit, used to predict the density distribution of each region in multiple future time windows based on the transition probability matrix; The multi-granularity scene adaptive pedestrian flow monitoring mode switching module includes: The three-level monitoring mode definition unit is used to define at least three-level monitoring modes, including daily mode, early warning mode and emergency mode, with each mode corresponding to different data collection cycles and sensor combinations; The passive triggering switching unit is used to receive the level change signal of the downstream emergency evacuation system and passively trigger the switching of the monitoring mode. A sensor combination dynamic adjustment unit is used to dynamically adjust the number and combination of activated sensors according to the monitoring mode; The downstream system density data output interface module includes: Multi-level data construction units are used to organize density data into a unified data structure according to four levels: entrance level, area level, floor level, and shopping mall level. Quality metadata annotation unit, used to attach quality metadata such as confidence level, data freshness and data source to each piece of output data; The on-demand subscription push unit is used to support downstream systems to subscribe to different levels of density data on demand; Version compatibility adaptation units are used to ensure backward compatibility with new versions when upgrading data structures.

[0009] Preferably, the entrance sensor group is deployed at all entrances and exits of the semi-enclosed space, serving as high-precision data anchor points for the entire pedestrian flow monitoring system; the entrance sensor group includes at least: Two-way infrared sensors are installed on both sides of the entrance channel, with at least one set at each entrance, including light curtain A and light curtain B. For wide channels, multiple sets are deployed side by side. WiFi probes are deployed on the ceiling directly above the entrance, one for each entrance, and are not reused with area probes; High-definition cameras are deployed on the ceiling directly above the entrance, one at each entrance, installed at a downward angle; The regional sensor array is deployed within each functional area of ​​the semi-enclosed space, providing raw observations for conservation constraint correction and device trajectory data for transition probability learning; the regional sensor array includes: WiFi+BLE dual-mode probes are deployed in the center of the ceiling of each functional area, with at least one WiFi+BLE dual-mode probe deployed in each functional area. The auxiliary sensor group is deployed at the inter-floor connection nodes to provide floor-level conservation constraints and inter-floor pedestrian flow data; the auxiliary sensor group includes: One-way infrared counters are deployed at the top and bottom of each escalator, with one counter at each end of each escalator.

[0010] Preferably, each of the multi-source sensors independently collects raw data for the current period, and the system evaluates the confidence level of each source based on the characteristics of the raw data; the cross-validation fault detection unit cross-compares the multi-source technologies pairwise, identifies sensors with abnormal deviations based on a "two-to-one" voting mechanism, and outputs the status of each sensor and the adjusted confidence level. The cross-validation is implemented according to the following process: a. Extract the three-source counting values: count_ir, count_wifi, and count_video; where count_ir is the infrared counting value, count_wifi is the WiFi estimated number of people, and count_video is the video AI counting value; b. Calculate the pairwise relative deviation δ(a,b) = |ab| / avg(a,b); where a and b represent different two-source count data respectively; c. Determine if the "two-to-one" condition is met: the deviation between two sources is <θ_cross and the deviation of the third source is >θ_fault; d. If the condition is met, the third source is marked as a suspected fault, and the confidence level is reduced to C_min (0.1). e. When all three sources are inconsistent, i.e., all three pairs of deviations are greater than θ_cross, a manual alarm is triggered. The confidence dynamic weighted fusion unit, based on the adjusted confidence normalized weighting, fuses the three-source counts to output the fused count result and the global confidence; The dynamic weighted fusion of confidence scores is implemented through the following process: a. Obtain the adjusted confidence scores for each sensor: conf_ir, conf_wifi, and conf_video. b. Normalize the confidence scores into weights: weight_i = conf_i / (conf_ir + conf_wifi + conf_video); c. Obtain the fused count by weighted summation: fusedCount = ROUND(Σweight_i × count_i); d. Calculate the global confidence score: globalConfidence = Σweight_i × conf_i; The historical accuracy feedback self-calibration unit periodically evaluates the historical accuracy of each sensor and updates the basic weights for subsequent fusion. Weight self-calibration is achieved through the following process: a. Calibration is triggered every T_calibrate minutes; b. Backtrack all records within this period and calculate the average deviation rate of each sensor: deviation_i = avg(|count_i - fusedCount| / fusedCount); c. Converting deviation rate to accuracy rate: accuracy_i = max(0, 1 - deviation_i); d. Progressively update the base weights W_i = W_i×(1 +η×(accuracy_i-Σaccuracy / 3)); where accuracy_i is the historical accuracy of the sensor; e. Normalization ensures that the sum of the weights is 1 and limits the range of a single weight to [W_min, W_max].

[0011] Preferably, the process for calculating the conservation constraint density is as follows: The first phase involves collecting multi-source data, obtaining the fusion counts In_k, Out_k, and netFlow_k of each entrance and exit, obtaining the raw estimated number of people R_z in each area from WiFi probes, and calculating the first-level conservation anchor point N_total = Σ_k netFlow_k. The second stage involves constructing a set of constraint equations, establishing first-level conservation constraints: the sum of the number of people in each area equals the total number of people; second-level conservation constraints: the sum of the number of people in each area within a floor equals the number of people on that floor; and non-negativity constraints. The third stage of constrained least squares solution aims to minimize the correction magnitude and uses the third-level conservation constraint to solve for the optimal density distribution that satisfies global consistency through the Lagrange multiplier method and the projected gradient method. The fourth stage involves quality assessment and density calculation, calculating the conserved residual assessment and correcting the quality, and converting the corrected population into density and heat level outputs. The specific algorithm implementation process for global conservation constraint correction is as follows: a. Construct a constrained optimization problem: the objective function is minΣ_z (N_z - R_z)² / σ_z²; b. Solve the equality-constrained subproblems using the Lagrange multiplier method, and construct the KKT equation system to obtain analytical solutions; c. Projection gradient method for handling non-negative constraints: negative values ​​are clipped to zero, and the difference is redistributed to the positive value region according to the weights. d. Calculate the conserved residual = |ΣN_z - N_total| / N_total, and evaluate the correction quality; e. Convert the corrected number of people into density and heat level outputs.

[0012] Preferably, the mall level is a first-order conservation law, where the total number of people remaining in the mall = the sum of the net flow of each entrance and exit = the cumulative number of people entering each entrance and exit - the cumulative number of people leaving each entrance and exit; calculated statistically by the entrance and exit anchor point constraint unit. The floor level follows a second-order conservation law, where the number of people on a floor equals the sum of the number of people in each area of ​​that floor; this is calculated statistically by the floor number estimation unit; the floor number estimation method is as follows: a. Calculate the net inflow of each floor: Net inflow from external direct entrances + Vertical traffic inflow - Vertical traffic outflow; b. Allocate N_total according to the proportion of positive net inflow to total positive net inflow for each floor; c. Rounding error correction to ensure that the sum of the number of people on each floor equals N_total; d. When vertical transportation data is missing, it degenerates into a uniform distribution; The regional level is a three-level conservation system, where the total number of people equals the sum of the number of people on each floor; this is calculated statistically by the regional constraint optimization and correction unit.

[0013] Preferably, the transition probability learning unit utilizes the dual-mode sniffing capability of the area WiFi probes to track the movement trajectory of devices between different areas; when a device's WiFi detection request or BLE broadcast signal disappears from the probe in area i and subsequently reappears in the probe in area j, it is recorded as a transition event from i to j; by statistically analyzing a large number of transition events, the transition probability between each pair of areas is calculated, and an exponential moving average is used for online updates to continuously adapt to changes in pedestrian flow patterns; the following algorithm is used for implementation: a. Initialize the transition counting matrix count[n][n]; b. Count the number of cross-region transfer events within the current time window Δt_transfer; c. Normalize to the current window transition probability P_current, and maintain the historical probability for regions without transition events; d. Exponential moving average fusion: P_new = γ × P_current + (1-γ) × P_history; e. Apply minimum probability guarantee P_min to the physical channel edges, and normalize the rows to ensure that the sum of each row is 1; The blind zone density inference unit is designed for blind zone nodes without sensor coverage. It uses the set N(b) of nodes with observed data within the K_neighbor order neighborhood and their corresponding transition probabilities to infer the pedestrian density in the blind zone using a weighted average. The inference formula comprehensively considers the density values ​​of neighboring nodes, the transition probability from the neighborhood to the blind zone, and the confidence level of the density data of the neighboring nodes themselves. The following algorithm is used to implement this: a. Identify the current blind zone set (areas without observation data); b. Layer-by-layer diffusion inference: Start with the first-order neighborhood (directly adjacent); c. If the first-order neighborhood data is insufficient, expand to the second-order neighborhood, up to a maximum of K_neighbor order; d. Inference weight = transition probability × observation confidence, and inference confidence decays exponentially by the number of hops (20% / hop). e. Extreme blind spots are degraded to the average density of the same floor as a safety net and are marked as low confidence. The short-term density prediction unit, based on the density distribution vector D(t) and transition probability matrix P of each region at the current moment, can predict the density distribution of the next time window: D(t+1)≈P^T ×D(t). This prediction result allows downstream systems to perceive density change trends in advance, providing a forward-looking basis for evacuation route planning. The following algorithm is used for implementation: a. Obtain the current density vector D(t) for each region; b. Matrix multiplication iteration: D(t+k) = (PT)k×D(t); c. Consider the expected net flow of entrances and exits during the forecast period and inject it into the entrance area; d. Apply non-negative constraints to ensure that the density is physically reasonable.

[0014] Preferably, the multi-granularity scene adaptive pedestrian flow monitoring mode switching module is implemented as follows: Step 1: Receive the emergency level signal, analyze the emergency level sent by the downstream emergency evacuation system, the emergency level includes: NORMAL / ALERT / EMERGENCY, and map it to the target monitoring mode; Step 2: Application mode configuration. Adjust the acquisition frequency according to the target mode: 15 minutes for daily use, 1 minute for early warning, and 5 seconds for emergency use. The sensor combination adopts dual-source, tri-source low frame, and tri-source high frame. Fusion strategy and output granularity, the output granularity includes: area level, floor level, and channel level. Step 3: Perform data acquisition and run the sensor acquisition and fusion process according to the current mode configuration; Step four involves a mode downgrade check. Once the emergency level recovers and a stable observation period of three consecutive data collection cycles is completed, the mode will automatically downgrade level by level. If the mode upgrade is performed, it will be executed immediately without waiting.

[0015] Preferably, the algorithm for implementing the mode switching is as follows: a. Receive downstream emergency level signals NORMAL / ALERT / EMERGENCY; b. Map the emergency response level to the target monitoring mode, where NORMAL is mapped to DAILY, ALERT is mapped to ALERT, and EMERGENCY is mapped to EMERGENCY. c. Determine if a switch is needed: If the target mode is equal to the current mode, keep it unchanged; d. Mode upgrade handling: Immediately switch to the target mode and adjust the data acquisition strategy to match the needs of downstream systems; e. Mode downgrade handling: Accumulate stable counts, and downgrade step by step after reaching 3 collection cycles, but skipping levels is not allowed. Adjust the collection strategy to match the needs of daily scenarios. f. Output new mode and corresponding configuration parameters, wherein the configuration parameters include at least: acquisition period, sensor combination, fusion strategy and output granularity.

[0016] Preferably, the downstream system density data output interface module organizes the density data into a unified structure based on a multi-level data construction unit, a quality metadata annotation unit, and an on-demand subscription push unit, categorizing the density data into four levels: entry level, region level, floor level, and shopping mall level. Each data entry carries three quality metadata items: confidence level, freshness, and source. Data at the corresponding level is pushed according to the downstream subscription configuration. The specific implementation process is as follows: a. Construct entry-level data: Based on the acquired fusion counts of each entry point, calculate the entry, exit, and net flow data, and add confidence and freshness tags; b. Constructing regional-level data: Merging corrected and inferred data, calculating heat levels, and labeling data sources, including: OBSERVED, INFERRED, etc. CORRECTED; c. Aggregate floor-level data: Calculate the total number of people on each floor, average density, and highest density area, and build a corridor density map for downstream path planning; d. Aggregate mall-level data: Statistics on total number of people, average density, distribution on each floor, conservation residuals, and current monitoring mode; e. Package and push to downstream systems according to subscription level.

[0017] Preferably, the heat level is classified according to the following method: The continuous pedestrian density values ​​are discretized into heat levels of 1 to 5, which makes it easier for downstream systems to understand and display them. Level 1 (vacant) corresponds to a density of <0.3 people / m²; Level 2 (Comfortable) corresponds to a density of 0.3~0.6 people / ㎡; Level 3 (relatively crowded) corresponds to a density of 0.6~1.0 people / m²; Level 4 (Crowded) corresponds to a density of 1.0~1.5 people / m²; Level 5 (Dangerous Crowding) corresponds to a density of ≥ 1.5 people / m².

[0018] Another objective of this invention is to provide a method for real-time monitoring of pedestrian density in shopping malls based on multi-source fusion and conservation constraints, comprising: S1: Multi-source sensor cross-validation and self-calibration pedestrian flow statistics: S1.1 At the entrance of the shopping mall, simultaneously collect independent count values ​​and confidence levels from no fewer than three heterogeneous pedestrian counting sensors with independent error sources; S1.2 Perform pairwise cross-comparison of the counting results of the three sensors to identify sensors with abnormal deviations and reduce their confidence level; S1.3 Perform normalized weighted fusion based on the real-time confidence scores adjusted by each sensor, and output the fusion count value and global confidence score; S1.4 The basic weights of each sensor are periodically updated based on historical deviation statistics to achieve long-term self-calibration; S2: Inference based on the global density consistency constraint of the law of pedestrian flow conservation: S2.1 Calculate the net flow of each entrance and exit based on the merged count results of each entrance, and accumulate the current total number of people in the mall to obtain the anchor point; S2.2 Establish a three-level conservation constraint equation system: mall-level conservation, floor-level conservation, and non-negativity constraints; S2.3 Using the sensor estimates for each region as observed values, and under the conditions of satisfying conservation equality constraints and non-negativity constraints, solve for the globally consistent number of people corrected in each region; S2.4 Combines the net flow of direct access from entrances and exits with the vertical traffic flow across floors to distribute the total number of people to each floor; S3: Probability matrix and dynamic density inference of inter-regional population flow: S3.1 Construct the various areas and passages of the shopping mall into a directed graph topology; S3.2 Based on the statistics of cross-regional equipment transfer events, the transfer probability matrix is ​​continuously updated using an online learning method; S3.3 Infer pedestrian density in sensor-free areas by using neighborhood node density and transition probability; S3.4 Predicting the density distribution of each region in multiple future time windows based on the transition probability matrix; S4: Multi-granularity scene adaptive monitoring mode switching: S4.1 Receives emergency level signals from downstream emergency evacuation systems; When upgrading from S4.2 mode, immediately switch to the corresponding higher-level monitoring mode; When downgrading to S4.3 mode, the downgrading process proceeds step by step after a preset stabilization period. S5: Standardized density data output: S5.1 The density data is organized into a unified data structure according to four levels: entrance level, area level, floor level, and shopping mall level; S5.2 Attach quality metadata such as confidence level, data freshness, and data source to each data entry; S5.3 Push density data of the corresponding level to downstream subscribers according to the output cycle of the current monitoring mode.

[0019] The beneficial effects of this invention are: This invention overcomes the inherent accuracy ceiling of a single sensor by using a multi-source sensor cross-validation and self-calibration method, significantly improving the accuracy of fusion counting. Furthermore, it can automatically detect and correct sensor performance drift or malfunctions, ensuring long-term stable data quality.

[0020] The global constraint inference method for pedestrian flow conservation of the present invention establishes a three-level constraint equation system by utilizing the physical conservation law of semi-enclosed space, which fundamentally eliminates the data contradictions caused by independent estimation of each region and provides globally self-consistent density distribution data for the downstream evacuation system.

[0021] The inter-regional transfer probability inference method effectively infers the pedestrian density in sensorless coverage areas through directed graph modeling and transfer matrix learning, eliminating density blind spots and achieving complete spatial coverage.

[0022] The multi-granularity scenario adaptive monitoring mode switching mechanism operates in a low-cost mode during daily operations and automatically switches to a high-precision, high-frequency mode during emergencies, balancing operating costs and emergency needs.

[0023] The standardized density data output interface enables loosely coupled connection and independent evolution of upstream and downstream systems. When the sensor hardware of this system is upgraded or the fusion algorithm is optimized, the downstream system can enjoy the accuracy improvement without any changes. Attached Figure Description

[0024] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0025] Figure 1 This is an overview block diagram of the sensor deployment architecture of the present invention; Figure 2 This is a schematic diagram showing the overall flow of sensor data in this invention; Figure 3 This is a flowchart illustrating the overall process of multi-source sensor cross-validation and self-calibration in this invention. Figure 4 This is the overall flowchart of the global density consistency constraint inference based on the conservation of people flow in this invention; Figure 5 This is a flowchart of the inter-regional pedestrian flow transfer probability matrix and dynamic density inference process of the present invention; Figure 6 This is a flowchart of the multi-granularity scene adaptive pedestrian flow monitoring mode switching process of the present invention. Detailed Implementation

[0026] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0027] Example 1 A real-time crowd density monitoring system based on multi-source fusion and conservation constraints includes: The multi-source sensor cross-validation and self-calibration people flow statistics module includes: The multi-source heterogeneous sensing and acquisition unit includes an entrance sensor group, an area sensor group, and an auxiliary sensor group for heterogeneous pedestrian counting; each sensor independently generates a count value and confidence level. like Figure 1 As shown, the entrance sensor array is deployed at all entrances and exits of semi-enclosed spaces like shopping malls, serving as high-precision data anchors for the entire pedestrian flow monitoring system; the entrance sensor array includes at least: Two-way infrared sensors are installed on both sides of the entrance channel, with at least one set at each entrance, including light curtain A and light curtain B. For wide channels, multiple sets are deployed side by side. Each set covers a channel width ≤3m and counts entry, exit, and trigger timestamps. As the first data source for three-source fusion, it provides directional counting. WiFi probes are deployed on the ceiling directly above each entrance, one for each entrance only, and are not reused with area probes; the coverage area is 15-20m in radius of the entrance, and provides a list of device MAC addresses, number of devices, and signal strength data. As a second data source for three-source fusion, it provides device count estimation. High-definition cameras are deployed on the ceiling directly above the entrance, one at each entrance, installed at a downward angle; the coverage area includes the entrance passage and the area in front, providing video stream data for video AI processing, serving as the third data source for the fusion of three sources; The regional sensor array is deployed within each functional area of ​​the semi-enclosed space, providing raw observations for conservation constraint correction and device trajectory data for transition probability learning; the regional sensor array includes: WiFi+BLE dual-mode probes are deployed in the center of the ceiling of each functional area, with at least one WiFi+BLE dual-mode probe deployed in each functional area; each probe has a coverage radius of 30~50m and provides a list of WiFi device MAC addresses, device quantity data, and BLE device broadcast events within the area. An auxiliary sensor array is deployed at inter-floor connection nodes to provide floor-level conservation constraints and inter-floor pedestrian flow data; the auxiliary sensor array includes: One-way infrared counters are deployed at the top and bottom of each escalator, with one unit at each end. The coverage area is the width of the escalator entrance, monitoring the number of people going up and down the escalator. The cross-validation fault detection unit performs pairwise cross-validation on the counting results of multi-source heterogeneous sensors, identifies sensors with abnormal deviations, and reduces their confidence level. Each of the multi-source sensors independently collects raw data for the current period, and the system evaluates the confidence level of each source based on the characteristics of the raw data. The cross-validation fault detection unit cross-compares the multi-source technologies pairwise, identifies sensors with abnormal deviations based on a "two-to-one" voting mechanism, and outputs the status of each sensor and the adjusted confidence level. Cross-validation is implemented as follows: a. Extract the three-source counting values: count_ir, count_wifi, and count_video; where count_ir is the infrared counting value, count_wifi is the WiFi estimated number of people, and count_video is the video AI counting value; b. Calculate the pairwise relative deviation δ(a,b) = |ab| / avg(a,b); where a and b represent different two-source count data respectively; c. Determine if the "two-to-one" condition is met: the deviation between two sources is <θ_cross and the deviation of the third source is >θ_fault; d. If the condition is met, the third source is marked as a suspected fault, and the confidence level is reduced to C_min (0.1). e. When all three sources are inconsistent, i.e., all three pairs of deviations are greater than θ_cross, a manual alarm is triggered. The aforementioned "two-to-one" voting mechanism leverages the independent nature of the three sensor error sources: infrared sensor (physical occlusion error), WiFi (carry-rate statistical error), and video (illumination / density recognition error). The probability of two sensors simultaneously exhibiting the same deviation is extremely low. When two sources yield consistent results while the third source shows excessive deviation, the third source is automatically flagged as a suspected fault. Through cross-verification of multi-source sensors, second-level automatic identification and isolation of sensor faults are achieved, eliminating the need for manual inspection. Faulty sensors retain a weak weight of 0.1 to mitigate voting misjudgments. Alarms are triggered in mutually exclusive three-source scenarios to prevent misjudgments. The confidence-weighted fusion unit is used to perform normalized weighted fusion based on the real-time confidence of each sensor, and output the fusion count value and global confidence. The dynamic weighted fusion of confidence scores is implemented through the following process: a. Obtain the adjusted confidence scores of each sensor: conf_ir (infrared sensor), conf_wifi (WiFi sensor), and conf_video (camera). b. Normalize the confidence scores into weights: weight_i = conf_i / (conf_ir + conf_wifi + conf_video); c. Obtain the fused count by weighted summation: fusedCount = ROUND(Σweight_i × count_i); d. Calculate the global confidence score: globalConfidence = Σweight_i × conf_i; By using the adjusted confidence level after cross-validation as a dynamic weight, sensors with high confidence levels contribute more, while the impact of suspected faulty sensors (confidence level 0.1) is naturally suppressed; the global confidence level is output for downstream evaluation of data reliability. The historical accuracy feedback self-calibration unit is used to periodically update the basic weights of each sensor based on the historical deviation statistics of the sensor itself, so as to achieve long-term self-calibration; this unit periodically evaluates the historical accuracy of each sensor and feeds back to update the basic weights for subsequent fusion. Weight self-calibration is achieved through the following process: a. Calibration is triggered every T_calibrate minutes; b. Backtrack all records within this period and calculate the average deviation rate of each sensor: deviation_i = avg(|count_i - fusedCount| / fusedCount); c. Converting deviation rate to accuracy rate: accuracy_i = max(0, 1 - deviation_i); d. Progressively update the base weights W_i = W_i×(1 +η×(accuracy_i-Σaccuracy / 3)); where accuracy_i is the historical accuracy of the sensor; e. Normalization ensures that the sum of the weights is 1 and limits the range of a single weight to [W_min, W_max]. The system periodically evaluates the historical deviation between each sensor and the fusion result, and gradually adjusts the base weights through the learning rate. The weights of sensors with good long-term performance are gradually increased, while those with poor performance are gradually decreased. This allows the system to automatically adapt to sensor aging and environmental changes without the need for manual parameter tuning. The learning rate mechanism avoids drastic weight oscillations, and the upper and lower limits of the weights prevent any one source from completely dominating the system. The conservation constraint density inference module includes: The three-level conservation constraint modeling unit is used to establish a set of constraint equations for the conservation relationship of pedestrian flow in shopping malls at three levels: mall level, floor level, and area level. The entrance / exit anchor point constraint unit is used to calculate the net flow based on the entrance / exit fusion count results, and serves as the global total number of people anchor point constraint. The regional constraint optimization correction unit is used to solve for the globally consistent number of people corrected in each region by using the sensor estimates of each region as observations, under the conditions of satisfying conservation equality constraints and non-negativity constraints. The floor occupancy estimation unit is used to combine the net direct traffic flow from entrances and exits with the vertical traffic flow across floors, and to allocate the total number of people to each floor. The conservation relationship of pedestrian flow is established into a set of constraint equations at three levels: mall level, floor level, and area level. The high-precision bidirectional counting at entrances and exits is used as the anchor point to constrain the total number of people worldwide. The independent estimates of each area are then globally optimized and corrected. The correction algorithm aims to minimize the correction amplitude and uses the three-level conservation equations as constraints to solve for the optimal density distribution that satisfies global consistency. The process of calculating the conservation constraint density includes four stages: The first phase involves collecting multi-source data, obtaining the fusion counts In_k, Out_k, and netFlow_k of each entrance and exit, obtaining the raw estimated number of people R_z in each area from WiFi probes, and calculating the first-level conservation anchor point N_total = Σ_k netFlow_k. The second stage involves constructing a set of constraint equations, establishing first-level conservation constraints (sum of people in each area = total number of people), second-level conservation constraints (sum of people in each area within a floor = number of people on the floor), and non-negativity constraints. The third stage of constrained least squares solution aims to minimize the correction magnitude and uses the third-level conservation constraint to solve for the optimal density distribution that satisfies global consistency through the Lagrange multiplier method and the projected gradient method. The fourth stage involves quality assessment and density calculation, calculating the conserved residual assessment and correcting the quality, and converting the corrected population into density and heat level outputs. The specific algorithm implementation process for global conservation constraint correction is as follows: a. Construct a constrained optimization problem: the objective function is minΣ_z (N_z - R_z)² / σ_z²; b. Solve the equality-constrained subproblems using the Lagrange multiplier method, and construct the KKT equation system to obtain analytical solutions; c. Projection gradient method for handling non-negative constraints: negative values ​​are clipped to zero, and the difference is redistributed to the positive value region according to the weights. d. Calculate the conserved residual = |ΣN_z - N_total| / N_total, and evaluate the correction quality; e. Convert the corrected number of people into density and heat level output; The three levels are divided as follows: For shopping malls, the first-level conservation rule applies: the total number of people remaining in the mall = the sum of net flow at each entrance / exit = the cumulative number of people entering at each entrance / exit - the cumulative number of people leaving; calculated statistically by the entrance / exit anchor point constraint unit. The floor level follows a second-order conservation law: the number of people on a floor equals the sum of the number of people in each area of ​​that floor; the number of people on a floor is calculated from the statistical unit of the floor-level population calculation; the process of calculating the number of people on a floor is as follows: a. Calculate the net inflow of each floor: Net inflow from external direct entrances + Vertical traffic inflow - Vertical traffic outflow; b. Allocate N_total according to the proportion of positive net inflow to total positive net inflow for each floor; c. Rounding error correction to ensure that the sum of the number of people on each floor equals N_total; d. When vertical transportation data is missing, it degenerates into a uniform distribution; The regional level is based on three levels of conservation, with the total number of people equal to the sum of the number of people on each floor; this is calculated statistically by the regional constraint optimization correction unit. The inter-regional population flow probability matrix and dynamic density inference module includes: Directed graph modeling unit is used to abstract each region in space into several nodes, the physical channels between regions into edges, and several nodes and edges to construct a spatial topology. The online learning unit for transfer probability is used to continuously update the transfer probability matrix P[i][j] based on the statistics of cross-regional transfer events of devices, using an online learning method, which represents the probability that a person in region i will transfer to region j within a unit time window; The online learning unit for transition probability utilizes the dual-mode sniffing capability of regional WiFi probes to track the movement trajectory of devices between different areas. When a device's WiFi detection request or BLE broadcast signal disappears from the probe in area i and subsequently reappears in the probe in area j, it is recorded as a transition event from i to j. By statistically analyzing a large number of transition events, the transition probability between each pair of areas is calculated, and an exponential moving average is used for online updates to continuously adapt to changes in pedestrian flow patterns. The following algorithm is used for implementation: a. Initialize the transition counting matrix count[n][n]; b. Count the number of cross-region transfer events within the current time window Δt_transfer; c. Normalize to the current window transition probability P_current, and maintain the historical probability for regions without transition events; d. Exponential moving average fusion: P_new = γ × P_current + (1-γ) × P_history; e. Apply minimum probability guarantee P_min to the physical channel edges and perform normalization to ensure that the sum of each row is 1; The matrix continuously adapts to changes in pedestrian flow patterns; the exponential smoothing mechanism avoids drastic oscillations caused by a single abnormal event; and the minimum probability guarantee prevents the probability of a certain channel from degenerating to zero, thus causing blind spot inference to fail. The blind zone density inference unit is used to infer the population density in areas with no sensor coverage or weak coverage signals by utilizing the density of neighboring nodes and the transition probability. The blind zone density inference unit targets nodes in blind zones without sensor coverage. It uses the set N(b) of nodes with observed data within the K_neighbor order neighborhood and their corresponding transition probabilities to infer the pedestrian density in the blind zone using a weighted average. The inference formula comprehensively considers the density values ​​of neighboring nodes, the transition probability from the neighborhood to the blind zone, and the confidence level of the density data of the neighboring nodes themselves. The following algorithm is used for implementation: a. Identify the current blind zone set (areas without observation data); b. Layer-by-layer diffusion inference: Start from the first-order neighborhood, which is the adjacent direct neighborhood; c. If the first-order neighborhood data is insufficient, expand to the second-order neighborhood, up to a maximum of K_neighbor order; d. Inference weight = transition probability × observation confidence, and inference confidence decays exponentially by the number of hops (20% / hop). e. Extreme blind spots are degraded to the average density of the same floor as a safety net and are marked as low confidence. By using the above blind zone estimation method, density estimates can be made for blind zones instead of simply setting them to zero; the transfer probability is used as a weight, and the inference results reflect the pattern of population flow; the confidence decay mechanism makes downstream users aware of the uncertainty of the inferred data. Short-term density prediction unit, used to predict the density distribution of each region in multiple future time windows based on the transition probability matrix; Based on the density distribution vector D(t) and transition probability matrix P of each region at the current time, the short-time density prediction unit can predict the density distribution of the next time window: D(t+1)≈P^T×D(t). This prediction result can help downstream systems perceive density change trends in advance, providing a forward-looking basis for evacuation route planning. The following algorithm is used for implementation: a. Obtain the current density vector D(t) for each region; b. Matrix multiplication iteration: D(t+k) = (PT)k×D(t); c. Consider the expected net flow of entrances and exits during the forecast period and inject it into the entrance area; d. Apply non-negative constraints to ensure that the density is physically reasonable; The multi-granularity scene adaptive pedestrian flow monitoring mode switching module includes: The three-level monitoring mode definition unit is used to define three-level monitoring modes, including daily mode, early warning mode and emergency mode. Each mode corresponds to a different data acquisition cycle and sensor combination. The passive triggering switching unit is used to receive the level change signal of the downstream emergency evacuation system and passively trigger the switching of the monitoring mode. A sensor combination dynamic adjustment unit is used to dynamically adjust the number and combination of activated sensors according to the monitoring mode; The implementation process of the multi-granularity scene adaptive pedestrian flow monitoring mode switching module is as follows: Step 1: Receive the emergency level signal, analyze the emergency level sent by the downstream emergency evacuation system. The emergency levels include: NORMAL / ALERT / EMERGENCY, and map them to the target monitoring mode. Step 2: Configure the application mode. Adjust the acquisition frequency according to the target mode: 15 minutes for daily use, 1 minute for early warning, and 5 seconds for emergency use. The sensor combination adopts dual-source, tri-source low frame, and tri-source high frame. The fusion strategy and output granularity include: area level, floor level, and channel level. Step 3: Perform data acquisition and run the sensor acquisition and fusion process according to the current mode configuration; Step 4: Mode downgrade check. When the emergency level recovers and after a stable observation period of 3 consecutive data collection cycles, the mode will automatically downgrade level by level. If the mode upgrade is performed, it will be executed immediately without waiting. The algorithm for implementing the above mode switching is as follows: a. Receive downstream emergency level signals NORMAL / ALERT / EMERGENCY; b. Map the emergency response level to the target monitoring mode, where NORMAL is mapped to DAILY, ALERT is mapped to ALERT, and EMERGENCY is mapped to EMERGENCY. c. Determine if a switch is needed: If the target mode is equal to the current mode, keep it unchanged; d. Mode upgrade handling: Immediately switch to the target mode and adjust the data acquisition strategy to match the needs of downstream systems; e. Mode downgrade handling: Accumulate stable counts, and downgrade step by step after reaching 3 collection cycles, but skipping levels is not allowed. Adjust the collection strategy to match the needs of daily scenarios. f. Output the new mode and corresponding configuration parameters. The configuration parameters should include at least: acquisition cycle, sensor combination, fusion strategy and output granularity. The design logic for mode switching is as follows: Passive triggering, not active judgment: This system does not perform any anomaly detection or early warning judgment; that is the responsibility of the downstream early warning system. It only passively switches modes based on the emergency level signal issued by the downstream emergency evacuation system. This ensures a clear boundary of responsibilities between this system and the early warning system (warning.md). Gradual upgrades and rapid downgrades: Mode upgrades (routine → early warning → emergency) require explicit instructions from downstream systems; mode downgrades (emergency → early warning → routine) automatically downgrade after the downstream system returns to normal and after a stable observation period (no new upgrade instructions for 3 data collection cycles). Core of strategy adaptation: In normal mode, upstream and downstream systems only need regional density overview, which can be met by using infrared + WiFi dual-source fusion; in early warning and emergency mode, upstream and downstream systems need higher frequency and more granular density data, so three-source cross-validation fusion is enabled to provide higher accuracy data support. The system achieves dynamic adaptation of data collection strategies to downstream system scenario requirements through multi-granularity scene adaptive pedestrian flow monitoring mode switching. The daily mode outputs regional density to meet the needs of operational overview, while the emergency mode outputs multi-granularity density to support evacuation route planning. Mode upgrades are completed in seconds to meet the emergency response time requirements of downstream systems. A stabilization period mechanism avoids frequent switching that could cause system oscillations. The system has a clear boundary of responsibilities with the downstream early warning system, focusing on data collection and output. The downstream system density data output interface module includes: Multi-level data construction units are used to organize density data into a unified data structure according to four levels: entrance level, area level, floor level, and shopping mall level. Quality metadata annotation unit, used to attach quality metadata such as confidence level, data freshness and data source to each piece of output data; The on-demand subscription push unit is used to support downstream systems to subscribe to different levels of density data on demand; Version compatibility adaptation unit is used to ensure backward compatibility with new versions when upgrading data structures; Based on multi-level data construction units, quality metadata annotation units, and on-demand subscription push units, density data is organized into a unified structure according to four levels: entry level, region level, floor level, and shopping mall level. Each data entry carries three quality metadata items: confidence level, freshness level, and source level. Data at the corresponding level is pushed according to the downstream subscription configuration. The specific implementation process is as follows: a. Construct entry-level data: Based on the acquired fusion counts of each entry point, calculate the inbound / outbound / net traffic, and add confidence and freshness tags; b. Constructing regional-level data: Merging corrected and inferred data, calculating heat levels, and labeling data sources, including: OBSERVED, INFERRED, etc. CORRECTED; c. Aggregate floor-level data: Calculate the total number of people on each floor, average density, and highest density area, and build a corridor density map for downstream path planning; d. Aggregate mall-level data: Statistics on total number of people, average density, distribution on each floor, conservation residuals, and current monitoring mode; e. Package and push the packaged products to downstream systems according to their subscription levels; The standardized output interface design scheme is as follows: Hierarchical data structure: Output data is organized in four levels—entranceLevel, zoneLevel, floorLevel, and mallLevel; each level of data can be subscribed to independently, and downstream systems can obtain it on demand. Quality metadata annotation: Each density data point must carry three quality metadata items—confidence (0~1), staleness (seconds since the most recent collection), and source (marked as OBSERVED / INFERRED / CORRECTED); downstream systems can use this to determine the reliability of the data and make corresponding decisions. Interface stability contract: Once the data structure of the standardized output interface is released, new versions must be backward compatible; newly added fields are optional and do not break the parsing logic of existing downstream systems; The popularity levels mentioned above are classified as follows: The continuous pedestrian density values ​​are discretized into heat levels of 1 to 5, which makes it easier for downstream systems to understand and display them. Level 1 (vacant) corresponds to a density of <0.3 people / m²; Level 2 (Comfortable) corresponds to a density of 0.3~0.6 people / ㎡; Level 3 (relatively crowded) corresponds to a density of 0.6~1.0 people / m²; Level 4 (Crowded) corresponds to a density of 1.0~1.5 people / m²; Level 5 (Dangerous Crowding) corresponds to a density of ≥ 1.5 people / m².

[0028] Example 2 A method for real-time monitoring of pedestrian density in shopping malls based on multi-source fusion and conservation constraints includes: S1: Multi-source sensor cross-validation and self-calibration pedestrian flow statistics: S1.1 At the entrance of the shopping mall, simultaneously collect independent count values ​​and confidence levels from no fewer than three heterogeneous pedestrian counting sensors with independent error sources; S1.2 Perform pairwise cross-comparison of the counting results of the three sensors to identify sensors with abnormal deviations and reduce their confidence level; S1.3 Perform normalized weighted fusion based on the real-time confidence scores adjusted by each sensor, and output the fusion count value and global confidence score; S1.4 The basic weights of each sensor are periodically updated based on historical deviation statistics to achieve long-term self-calibration; S2: Inference based on the global density consistency constraint of the law of pedestrian flow conservation: S2.1 Calculate the net flow of each entrance and exit based on the merged count results of each entrance, and accumulate the current total number of people in the mall to obtain the anchor point; S2.2 Establish a three-level conservation constraint equation system: mall-level conservation, floor-level conservation, and non-negativity constraints; S2.3 Using the sensor estimates for each region as observed values, and under the conditions of satisfying conservation equality constraints and non-negativity constraints, solve for the globally consistent number of people corrected in each region; S2.4 Combines the net flow of direct access from entrances and exits with the vertical traffic flow across floors to distribute the total number of people to each floor; S3: Probability matrix and dynamic density inference of inter-regional population flow: S3.1 Construct the various areas and passages of the shopping mall into a directed graph topology; S3.2 Based on the statistics of cross-regional equipment transfer events, the transfer probability matrix is ​​continuously updated using an online learning method; S3.3 Infer pedestrian density in sensor-free areas by using neighborhood node density and transition probability; S3.4 Predicting the density distribution of each region in multiple future time windows based on the transition probability matrix; S4: Multi-granularity scene adaptive monitoring mode switching: S4.1 Receives emergency level signals from downstream emergency evacuation systems; When upgrading from S4.2 mode, immediately switch to the corresponding higher-level monitoring mode; When downgrading to S4.3 mode, the downgrading process proceeds step by step after a preset stabilization period. S5: Standardized density data output: S5.1 The density data is organized into a unified data structure according to four levels: entrance level, area level, floor level, and shopping mall level; S5.2 Attach quality metadata such as confidence level, data freshness, and data source to each data entry; S5.3 Push density data of the corresponding level to downstream subscribers according to the output cycle of the current monitoring mode.

Claims

1. A real-time crowd density monitoring system based on multi-source fusion and conservation constraints, characterized in that, include: The multi-source sensor cross-validation and self-calibration people flow statistics module includes: The multi-source heterogeneous sensing and acquisition unit includes an entrance sensor group, an area sensor group, and an auxiliary sensor group for heterogeneous pedestrian counting; each sensor independently generates a count value and confidence level. The cross-validation fault detection unit performs pairwise cross-comparison verification on the counting results of the multi-source heterogeneous sensors, identifies sensors with abnormal deviations, and reduces their confidence level. The confidence-weighted fusion unit is used to perform normalized weighted fusion based on the real-time confidence of each sensor, and output the fusion count value and global confidence. The historical accuracy feedback self-calibration unit is used to periodically update the basic weights of each sensor based on the historical deviation statistics of the sensor itself, so as to achieve long-term self-calibration. The conservation constraint density inference module includes: The three-level conservation constraint modeling unit is used to establish a set of constraint equations for the conservation relationship of pedestrian flow in shopping malls at three levels: mall level, floor level, and area level. The entrance / exit anchor point constraint unit is used to calculate the net flow based on the entrance / exit fusion count results, and serves as the global total number of people anchor point constraint. The regional constraint optimization correction unit is used to solve for the globally consistent number of people corrected in each region by using the sensor estimates of each region as observations, under the conditions of satisfying conservation equality constraints and non-negativity constraints. The floor occupancy estimation unit is used to combine the net direct traffic flow from entrances and exits with the vertical traffic flow across floors, and to allocate the total number of people to each floor. The inter-regional population flow probability matrix and dynamic density inference module includes: A directed graph modeling unit is used to abstract each region in space into several nodes, the physical channels between regions into edges, and the nodes and edges together to construct a spatial topology. The online learning unit for transfer probability is used to continuously update the transfer probability matrix P[i][j] based on the statistics of cross-regional transfer events of devices, using an online learning method, which represents the probability that a person in region i will transfer to region j within a unit time window; The blind zone density inference unit is used to infer the population density in areas with no sensor coverage or weak coverage signals by utilizing the density of neighboring nodes and the transition probability. Short-term density prediction unit, used to predict the density distribution of each region in multiple future time windows based on the transition probability matrix; The multi-granularity scene adaptive pedestrian flow monitoring mode switching module includes: The three-level monitoring mode definition unit is used to define at least three-level monitoring modes, including daily mode, early warning mode and emergency mode, with each mode corresponding to different data acquisition cycles and sensor combinations; The passive triggering switching unit is used to receive the level change signal of the downstream emergency evacuation system and passively trigger the switching of the monitoring mode. A sensor combination dynamic adjustment unit is used to dynamically adjust the number and combination of activated sensors according to the monitoring mode; The downstream system density data output interface module includes: Multi-level data construction units are used to organize density data into a unified data structure according to four levels: entrance level, area level, floor level, and shopping mall level. Quality metadata annotation unit, used to attach quality metadata such as confidence level, data freshness and data source to each piece of output data; The on-demand subscription push unit is used to support downstream systems to subscribe to different levels of density data on demand; Version compatibility adaptation units are used to ensure backward compatibility with new versions when upgrading data structures.

2. The real-time crowd density monitoring system based on multi-source fusion and conservation constraints according to claim 1, characterized in that, The entrance sensor group is deployed at all entrances and exits of the semi-enclosed space, serving as a high-precision data anchor point for the entire people flow monitoring system. The inlet sensor group includes at least: Two-way infrared sensors are installed on both sides of the entrance channel, with at least one set at each entrance, including light curtain A and light curtain B. For wide channels, multiple sets are deployed side by side. WiFi probes are deployed on the ceiling directly above the entrance, one for each entrance, and are not reused with area probes; High-definition cameras are deployed on the ceiling directly above the entrance, one at each entrance, installed at a downward angle; The regional sensor array is deployed within each functional area of ​​the semi-enclosed space, providing raw observations for conservation constraint correction and device trajectory data for transition probability learning; the regional sensor array includes: WiFi+BLE dual-mode probes are deployed in the center of the ceiling of each functional area, with at least one WiFi+BLE dual-mode probe deployed in each functional area. The auxiliary sensor group is deployed at the inter-floor connection nodes to provide floor-level conservation constraints and inter-floor pedestrian flow data; the auxiliary sensor group includes: One-way infrared counters are deployed at the top and bottom of each escalator, with one counter at each end of each escalator.

3. The real-time crowd density monitoring system based on multi-source fusion and conservation constraints according to claim 1, characterized in that, Each of the multi-source sensors independently collects raw data for the current period, and the system evaluates the confidence level of each source based on the characteristics of the raw data. The cross-validation fault detection unit cross-compares the multi-source technologies pairwise, identifies sensors with abnormal deviations based on a "two-to-one" voting mechanism, and outputs the status of each sensor and the adjusted confidence level. The cross-validation is implemented according to the following process: a. Extract the three-source counting values: count_ir, count_wifi, and count_video; where count_ir is the infrared counting value, count_wifi is the WiFi estimated number of people, and count_video is the video AI counting value; b. Calculate the pairwise relative deviation δ(a,b) = |ab| / avg(a,b); where a and b represent different two-source count data respectively; c. Determine if the "two-to-one" condition is met: the deviation between two sources is <θ_cross and the deviation of the third source is >θ_fault; d. If the condition is met, the third source is marked as a suspected fault, and the confidence level is reduced to C_min (0.1). e. When all three sources are inconsistent, i.e., all three pairs of deviations are greater than θ_cross, a manual alarm is triggered. The confidence dynamic weighted fusion unit, based on the adjusted confidence normalized weighting, fuses the three-source counts to output the fused count result and the global confidence; The dynamic weighted fusion of confidence scores is implemented through the following process: a. Obtain the adjusted confidence scores for each sensor: conf_ir, conf_wifi, and conf_video. b. Normalize the confidence scores into weights: weight_i = conf_i / (conf_ir + conf_wifi + conf_video); c. Obtain the fused count by weighted summation: fusedCount = ROUND(Σweight_i × count_i); d. Calculate the global confidence score: globalConfidence = Σweight_i × conf_i; The historical accuracy feedback self-calibration unit periodically evaluates the historical accuracy of each sensor and updates the basic weights for subsequent fusion. Weight self-calibration is achieved through the following process: a. Calibration is triggered every T_calibrate minutes; b. Backtrack all records within this period and calculate the average deviation rate of each sensor: deviation_i = avg(|count_i -fusedCount| / fusedCount); c. Converting deviation rate to accuracy rate: accuracy_i = max(0, 1 - deviation_i); d. Progressively update the base weights W_i = W_i×(1 +η×(accuracy_i-Σaccuracy / 3)); where accuracy_i is the historical accuracy of the sensor; e. Normalization ensures that the sum of the weights is 1 and limits the range of a single weight to [W_min, W_max].

4. The real-time crowd density monitoring system based on multi-source fusion and conservation constraints according to claim 1, characterized in that, The process for calculating the conservation constraint density is as follows: The first phase involves collecting multi-source data, obtaining the fusion counts In_k, Out_k, and netFlow_k of each entrance and exit, obtaining the raw estimated number of people R_z in each area from WiFi probes, and calculating the first-level conservation anchor point N_total = Σ_k netFlow_k. The second stage involves constructing a set of constraint equations, establishing a first-level conservation constraint: the sum of the number of people in each area equals the total number of people; and a second-level conservation constraint: the sum of the number of people in each area within a floor equals the number of people on that floor. Non-negativity constraint; The third stage of constrained least squares solution aims to minimize the correction magnitude and uses the third-level conservation constraint to solve for the optimal density distribution that satisfies global consistency through the Lagrange multiplier method and the projected gradient method. The fourth stage involves quality assessment and density calculation, calculating the conserved residual assessment and correcting the quality, and converting the corrected population into density and heat level outputs. The specific algorithm implementation process for global conservation constraint correction is as follows: a. Construct a constrained optimization problem: the objective function is minΣ_z (N_z - R_z)² / σ_z²; b. Solve the equality-constrained subproblems using the Lagrange multiplier method, and construct the KKT equation system to obtain analytical solutions; c. Projection gradient method for handling non-negative constraints: negative values ​​are clipped to zero, and the difference is redistributed to the positive value region according to the weights. d. Calculate the conserved residual = |ΣN_z - N_total| / N_total, and evaluate the correction quality; e. Convert the corrected number of people into density and heat level outputs.

5. The real-time crowd density monitoring system based on multi-source fusion and conservation constraints according to claim 1, characterized in that, The mall level is a first-order conservation law, where the total number of people remaining in the mall = the sum of the net flow of each entrance and exit = the cumulative number of people entering each entrance and exit - the cumulative number of people leaving each entrance and exit; calculated statistically by the entrance and exit anchor point constraint unit. The floor level follows a second-order conservation law, where the number of people on a floor equals the sum of the number of people in each area of ​​that floor; this is calculated statistically by the floor number estimation unit; the floor number estimation method is as follows: a. Calculate the net inflow of each floor: Net inflow from external direct entrances + Vertical traffic inflow - Vertical traffic outflow; b. Allocate N_total according to the proportion of positive net inflow to total positive net inflow for each floor; c. Rounding error correction to ensure that the sum of the number of people on each floor equals N_total; d. When vertical transportation data is missing, it degenerates into a uniform distribution; The regional level is a three-level conservation system, where the total number of people equals the sum of the number of people on each floor; this is calculated statistically by the regional constraint optimization and correction unit.

6. The real-time crowd density monitoring system based on multi-source fusion and conservation constraints according to claim 1, characterized in that, The transition probability learning unit uses the dual-mode sniffing capability of the regional WiFi probe to track the movement trajectory of the device between different regions; when a device's WiFi detection request or BLE broadcast signal disappears from the probe in region i and then reappears in the probe in region j, it is recorded as a transition event from i to j. By statistically analyzing a large number of transfer events, the transfer probability between pairs of areas is calculated, and an exponential moving average is used for online updates to continuously adapt to changes in pedestrian flow patterns; the following algorithm is employed: a. Initialize the transition counting matrix count[n][n]; b. Count the number of cross-region transfer events within the current time window Δt_transfer; c. Normalize to the current window transition probability P_current, and maintain the historical probability for regions without transition events; d. Exponential moving average fusion: P_new = γ × P_current + (1-γ) × P_history; e. Apply minimum probability guarantee P_min to the physical channel edges, and normalize the rows to ensure that the sum of each row is 1; The blind zone density inference unit is designed for blind zone nodes without sensor coverage. It uses the set N(b) of nodes with observed data within the K_neighbor order neighborhood and their corresponding transition probabilities to infer the pedestrian density in the blind zone using a weighted average. The inference formula comprehensively considers the density values ​​of neighboring nodes, the transition probability from the neighborhood to the blind zone, and the confidence level of the density data of the neighboring nodes themselves. The following algorithm is used for implementation: a. Identify the current blind spot set; b. Layer-by-layer diffusion inference: Start from the first-order neighborhood; c. If the first-order neighborhood data is insufficient, expand to the second-order neighborhood, up to a maximum of K_neighbor order; d. Inference weight = transition probability × observation confidence, and inference confidence decays exponentially with the number of hops; e. Extreme blind spots are degraded to the average density of the same floor as a safety net and are marked as low confidence. The short-term density prediction unit, based on the density distribution vector D(t) and transition probability matrix P of each region at the current moment, can predict the density distribution of the next time window: D(t+1)≈P^T ×D(t). This prediction result allows downstream systems to perceive density change trends in advance, providing a forward-looking basis for evacuation route planning. The following algorithm is used for implementation: a. Obtain the current density vector D(t) for each region; b. Matrix multiplication iteration: D(t+k) = (PT)k×D(t); c. Consider the expected net flow of entrances and exits during the forecast period and inject it into the entrance area; d. Apply non-negative constraints to ensure that the density is physically reasonable.

7. The real-time crowd density monitoring system based on multi-source fusion and conservation constraints according to claim 1, characterized in that, The implementation process of the multi-granularity scene adaptive pedestrian flow monitoring mode switching module is as follows: Step 1: Receive the emergency level signal, analyze the emergency level sent by the downstream emergency evacuation system, the emergency level includes: NORMAL / ALERT / EMERGENCY, and map it to the target monitoring mode; Step 2: Application mode configuration. Adjust the acquisition frequency according to the target mode: 15 minutes for daily use, 1 minute for early warning, and 5 seconds for emergency use. The sensor combination adopts dual-source, tri-source low frame, and tri-source high frame. Fusion strategy and output granularity, the output granularity includes: area level, floor level, and channel level. Step 3: Perform data acquisition and run the sensor acquisition and fusion process according to the current mode configuration; Step four involves a mode downgrade check. Once the emergency level recovers and a stable observation period of three consecutive data collection cycles is completed, the mode will automatically downgrade level by level. If the mode upgrade is performed, it will be executed immediately without waiting.

8. A real-time crowd density monitoring system based on multi-source fusion and conservation constraints according to claim 7, characterized in that, The algorithm for implementing the mode switching is as follows: a. Receive downstream emergency level signals NORMAL / ALERT / EMERGENCY; b. Map the emergency response level to the target monitoring mode, where NORMAL is mapped to DAILY, ALERT is mapped to ALERT, and EMERGENCY is mapped to EMERGENCY. c. Determine if a switch is needed: If the target mode is equal to the current mode, keep it unchanged; d. Mode upgrade handling: Immediately switch to the target mode and adjust the data acquisition strategy to match the needs of downstream systems; e. Mode downgrade handling: Accumulate stable counts, and downgrade step by step after reaching 3 collection cycles, but skipping levels is not allowed. Adjust the collection strategy to match the needs of daily scenarios. f. Output new mode and corresponding configuration parameters, wherein the configuration parameters include at least: acquisition period, sensor combination, fusion strategy and output granularity.

9. A real-time crowd density monitoring system based on multi-source fusion and conservation constraints according to claim 1, characterized in that, The downstream system density data output interface module organizes density data into a unified structure based on a multi-level data construction unit, a quality metadata annotation unit, and an on-demand subscription push unit, categorizing the density data into four levels: entry level, regional level, floor level, and shopping mall level. Each data entry carries three quality metadata items: confidence level, freshness, and source. Data at the corresponding level is pushed according to the downstream subscription configuration. The specific implementation process is as follows: a. Construct entry-level data: Based on the acquired entry-level fusion counts, calculate entry, exit, and net flow data, and add confidence and freshness tags; b. Constructing regional-level data: Merging corrected and inferred data, calculating heat levels, and labeling data sources, including: OBSERVED, INFERRED, etc. CORRECTED; c. Aggregate floor-level data: Calculate the total number of people on each floor, average density, and highest density area, and build a corridor density map for downstream path planning; d. Aggregate mall-level data: Statistics on total number of people, average density, distribution on each floor, conservation residuals, and current monitoring mode; e. Package and push the packaged products to downstream systems according to their subscription levels; The heat level is classified according to the following method: The continuous pedestrian density values ​​are discretized into heat levels of 1 to 5, which makes it easier for downstream systems to understand and display them. Level 1 (vacant) corresponds to a density of <0.3 people / m²; Level 2 (Comfortable) corresponds to a density of 0.3~0.6 people / ㎡; Level 3 (relatively crowded) corresponds to a density of 0.6~1.0 people / m²; Level 4 (Crowded) corresponds to a density of 1.0~1.5 people / m²; Level 5 (Dangerous Crowding) corresponds to a density of ≥ 1.5 people / m².

10. A method for real-time monitoring of pedestrian density in shopping malls based on multi-source fusion and conservation constraints, characterized in that, include: S1: Multi-source sensor cross-validation and self-calibration pedestrian flow statistics: S1.1 At the entrance of the shopping mall, simultaneously collect independent count values ​​and confidence levels from no fewer than three heterogeneous pedestrian counting sensors with independent error sources; S1.2 Perform pairwise cross-comparison of the counting results of the three sensors to identify sensors with abnormal deviations and reduce their confidence level; S1.3 Perform normalized weighted fusion based on the real-time confidence scores adjusted by each sensor, and output the fusion count value and global confidence score; S1.4 The basic weights of each sensor are periodically updated based on historical deviation statistics to achieve long-term self-calibration; S2: Inference based on the global density consistency constraint of the law of pedestrian flow conservation: S2.1 Calculate the net flow of each entrance and exit based on the merged count results of each entrance, and accumulate the current total number of people in the mall to obtain the anchor point; S2.2 Establish a three-level conservation constraint equation system: mall-level conservation, floor-level conservation, and non-negativity constraints; S2.3 Using the sensor estimates for each region as observed values, and under the conditions of satisfying conservation equality constraints and non-negativity constraints, solve for the globally consistent number of people corrected in each region; S2.4 Combines the net flow of direct access from entrances and exits with the vertical traffic flow across floors to distribute the total number of people to each floor; S3: Probability matrix and dynamic density inference of inter-regional population flow: S3.1 Construct the various areas and passages of the shopping mall into a directed graph topology; S3.2 Based on the statistics of cross-regional equipment transfer events, the transfer probability matrix is ​​continuously updated using an online learning method; S3.3 Infer pedestrian density in sensor-free areas by using neighborhood node density and transition probability; S3.4 Predicting the density distribution of each region in multiple future time windows based on the transition probability matrix; S4: Multi-granularity scene adaptive monitoring mode switching: S4.1 Receives emergency level signals from downstream emergency evacuation systems; When upgrading to S4.2 mode, immediately switch to the corresponding higher-level monitoring mode; When downgrading to S4.3 mode, the downgrading process proceeds step by step after a preset stabilization period. S5: Standardized density data output: S5.1 The density data is organized into a unified data structure according to four levels: entrance level, area level, floor level, and shopping mall level; S5.2 Attach quality metadata such as confidence level, data freshness, and data source to each data entry; S5.3 Push density data of the corresponding level to downstream subscribers according to the output cycle of the current monitoring mode.

Images