A regional passenger flow counting method based on multi-camera cooperative perception

Abstract

The embodiment of the application relates to a kind of based on the regional people flow statistics method of multi-camera cooperative perception, comprising the following steps: S1, the real-time video stream of multiple cameras distributed in monitoring area is acquired and sampling processing is carried out, and the density index reflecting the crowd crowded degree of current scene is calculated;S2, the density index is compared with preset switching condition, and the number of people statistics strategy is executed: when the density index meets sparse scene condition, the first statistical strategy is used to count the number of people in image;When the density index meets dense scene condition, the second statistical strategy is used to count the number of people in image;S3, the space topological relation between each camera is acquired, and adjacent camera pair with physical connected relationship is screened out;S4, cross-camera repeated personnel identification is executed for adjacent camera pair, and the number of cross-camera repeated personnel is counted;And according to the single-point statistical number of each camera and the number of repeated personnel, the overall people flow of monitoring area is calculated.

Description

Technical Field

[0001] This application relates to the fields of computer vision and intelligent transportation technology, and in particular to a method for regional pedestrian flow statistics based on multi-camera collaborative perception. Background Technology

[0002] As hubs of urban public transportation, subway stations rely heavily on passenger flow statistics to ensure operational safety, improve dispatching efficiency, and optimize service quality. Accurate passenger flow monitoring helps operators promptly identify congestion risks and provides a scientific basis for adjusting train frequency.

[0003] Currently, passenger flow statistics in subway stations mainly rely on manual visual monitoring, turnstile counters, single-camera video analysis, and wireless signal probes. However, these existing technologies all have limitations in practical applications. Manual monitoring depends on the subjective judgment of security personnel, which is prone to fatigue and missed detections during long working hours, and it is difficult to make millisecond-level quantitative responses to sudden large passenger flows. Although turnstile counters can count the number of people entering and exiting the station through infrared or pressure sensors, their monitoring range is limited to the turnstile entrances and cannot cover key internal areas such as platforms and transfer passages. Furthermore, they are prone to missed counts when multiple people pass through in parallel, and cannot reflect the real-time number of people remaining in the station.

[0004] With the development of computer vision technology, deep learning-based video counting schemes have gradually become mainstream. For example, existing technologies (such as application publication number CN112632601A) propose a crowd density estimation method based on VGG-16 and dilated convolution, which switches weights by judging the occlusion rate to adapt to the carriage scene; another scheme (such as application publication number CN120656116A) integrates YOLOv7 and spatiotemporal networks to build a passenger flow statistics system. However, the above schemes are mostly limited to the field of view analysis of a single camera and lack a collaborative mechanism between multiple cameras. When passengers move between the coverage areas of different cameras, the system often treats them as new targets and counts them repeatedly, resulting in an inflated total number of people. Although some existing technologies (such as application publication number CN109902551A) introduce pedestrian re-identification (Re-ID) technology, its application is mostly limited to time series matching under the same camera and has not effectively solved the problem of pedestrian deduplication across spatial dimensions. In addition, while the method of using WiFi or Bluetooth probes to count passenger flow by MAC address has a wide coverage, its statistical accuracy is limited by the activation rate of passenger devices and signal drift issues, making it difficult to meet the needs of refined operations.

[0005] In summary, existing technologies struggle to simultaneously ensure counting accuracy in mixed sparse and dense scenarios, and they also present the challenge of repeated counting across different fields of view in multi-camera network environments. Summary of the Invention

[0006] One objective of this application is to provide a method for regional pedestrian flow statistics based on multi-camera collaborative perception, which at least solves the above-mentioned problems.

[0007] To achieve the above objectives, some embodiments of this application provide a method for regional pedestrian flow statistics based on multi-camera collaborative perception, including the following steps: S1. Acquire real-time video streams from multiple cameras distributed within the monitoring area, and sample the video streams to calculate a density index reflecting the crowding level in the current scene. S2. Compare the density index with preset switching conditions, and adaptively execute the number of people counting strategy based on the comparison result: When the density index meets the sparse scene conditions, the first statistical strategy based on object detection is used to count the number of people in the image. When the density index meets the dense scene conditions, a second statistical strategy based on density estimation is used to count the number of people in the image. S3. Obtain the spatial topology relationship between each camera, and filter out adjacent camera pairs with physical connectivity based on the spatial topology relationship; S4. For the adjacent cameras, perform cross-camera duplicate person identification, count the number of duplicate people across cameras; and combine the single-point count of people from each camera with the number of duplicate people to calculate the overall traffic flow in the monitored area.

[0008] Compared with related technologies, the solution provided in this application effectively solves the problem of accuracy in complex monitoring scenarios where a single algorithm cannot take into account changes in density. By adaptively switching target detection and density estimation strategies, it ensures accurate individual positioning during off-peak hours and guarantees anti-occlusion capability during peak hours. At the same time, it uses physical spatial topology as a pre-constraint for cross-camera deduplication, limiting the high-computing-power pedestrian re-identification to pairs of physically connected adjacent cameras. This not only significantly reduces the computational cost of the system, but also logically eliminates spatial jump-type misjudgments caused by similar clothing among pedestrians in non-adjacent areas, achieving a closed-loop statistics of the entire area's passenger flow that combines real-time performance and high robustness in a wide range of scenarios. Attached Figure Description

[0009] One or more embodiments are illustrated by way of example with reference numerals in the accompanying drawings. These illustrations do not constitute a limitation on the embodiments. Elements with the same reference numerals in the drawings are denoted as similar elements. Unless otherwise stated, the figures in the drawings are not to be limited by scale.

[0010] Figure 1 This is a flowchart illustrating the regional pedestrian flow statistics method provided in this embodiment.

[0011] Figure 2 This is a schematic diagram of the architecture of the regional pedestrian flow statistics method provided in this embodiment.

[0012] Figure 3 This is a flowchart of the single-camera processing logic provided in the embodiments of this disclosure.

[0013] Figure 4 This is a flowchart of the cross-camera deduplication logic provided in the embodiments of this disclosure.

[0014] Figure 5 This is a schematic diagram of the CSRNet model provided in an embodiment of this disclosure.

[0015] Figure 6 This is a schematic diagram of the YOLOv7 model provided in an embodiment of this disclosure.

[0016] Figure 7 This is a schematic diagram of the multi-camera monitoring area coverage provided in this embodiment of the present disclosure; wherein, the area enclosed by the red dashed line represents the monitoring range of camera 1; the area enclosed by the orange dashed line represents the monitoring range of camera 2; the area enclosed by the orange solid line represents the monitoring range of camera 3; and the area enclosed by the red solid line represents the monitoring range of camera 4. Detailed Implementation

[0017] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0018] The terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate for the embodiments of this disclosure described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion.

[0019] In this disclosure, the terms "upper," "lower," "inner," "middle," "outer," "front," and "rear," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. These terms are primarily for better description of the embodiments of this disclosure and their implementations, and are not intended to limit the indicated devices, elements, or components to having a specific orientation, or to require them to be constructed and operated in a specific orientation. Furthermore, some of the aforementioned terms may be used to indicate other meanings besides orientation or positional relationship; for example, the term "upper" may in some cases indicate a dependency or connection relationship. Those skilled in the art can understand the specific meaning of these terms in the embodiments of this disclosure according to the specific circumstances.

[0020] Furthermore, the terms "set up," "connect," and "fix" should be interpreted broadly. For example, "connection" can be a fixed connection, a detachable connection, or an integral structure; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium, or it can be an internal connection between two devices, components, or parts. Those skilled in the art can understand the specific meaning of the above terms in the embodiments of this disclosure according to the specific circumstances.

[0021] Unless otherwise stated, the term "multiple" means two or more.

[0022] In this embodiment of the disclosure, the character " / " indicates that the objects before and after it are in an "or" relationship. For example, A / B means: A or B.

[0023] The term "and / or" describes an association between objects, indicating that three relationships can exist. For example, A and / or B means: A or B, or A and B.

[0024] It should be noted that, unless otherwise specified, the embodiments and features described in the present disclosure can be combined with each other.

[0025] Combination Figures 1 to 7 As shown in the figure, this disclosure provides a method for regional pedestrian flow statistics based on multi-camera collaborative perception, which includes the following steps: S1. Acquire real-time video streams from multiple cameras distributed within the monitoring area, sample and process the video streams, and calculate a density index reflecting the crowding level in the current scene. S2. Compare the density index with the preset switching conditions, and adaptively implement the number of people counting strategy based on the comparison results: When the density index meets the sparse scene conditions, the first statistical strategy based on object detection is used to count the number of people in the image. When the density index meets the dense scene conditions, a second statistical strategy based on density estimation is used to count the number of people in the image. S3. Obtain the spatial topology relationship between each camera, and filter out adjacent camera pairs with physical connectivity based on the spatial topology relationship; S4. For adjacent cameras, perform cross-camera duplicate person identification, count the number of duplicate people across cameras; and calculate the overall traffic flow in the monitored area based on the number of people counted at each camera point and the number of duplicate people.

[0026] The regional pedestrian flow statistics method provided in this disclosure effectively solves the accuracy problem of a single algorithm being unable to take into account changes in density in complex monitoring scenarios. By adaptively switching target detection and density estimation strategies, it ensures accurate individual positioning during off-peak hours and guarantees anti-occlusion capability during peak hours. At the same time, it uses physical spatial topology as a pre-constraint for cross-camera deduplication, limiting the high-computing-power pedestrian re-identification to adjacent camera pairs with physical connectivity. This not only significantly reduces the computational overhead of the system, but also logically eliminates spatial jump misjudgments caused by similar clothing among pedestrians in non-adjacent areas, achieving a closed-loop statistics of the entire area's pedestrian flow that combines real-time performance and high robustness in a wide range of scenarios.

[0027] Specifically, steps S1 and S2 construct an adaptive switching mechanism based on density metrics. In existing technologies, simple object detection algorithms (such as YOLO) are accurate in sparse scenes but are susceptible to occlusion and may miss detections, while simple density estimation algorithms (such as CSRNet) are resistant to occlusion but prone to overestimation noise in sparse scenes. This embodiment can automatically and seamlessly switch between a first statistical strategy and a second statistical strategy based on the congestion level of the real-time monitoring screen. That is, it leverages the low false alarm rate advantage of object detection during off-peak hours and the occlusion resistance advantage of density estimation during peak hours. This dynamic complementary strategy significantly improves the system's all-time statistical robustness in tidal passenger flow scenarios such as subway stations.

[0028] For steps S3 and S4, the "spatial topological relationship" of the cameras is used as a pre-constraint for pedestrian re-identification (Re-ID). Compared with the traditional blind search Re-ID method, this embodiment only selects adjacent camera pairs with "physical connectivity" for feature comparison. This logic first reduces the search space for feature matching from the full permutation level to the local neighborhood level, greatly reducing the computational load on edge computing nodes. Second, it directly blocks false matches caused by similar clothing between non-adjacent areas (i.e., areas that cannot be crossed in a short time) from a physical and logical perspective, thereby improving computational efficiency while ensuring the logical correctness of the deduplication results.

[0029] Through the collaborative calculation logic in step S4, this embodiment overcomes the limitation of traditional turnstiles or infrared counters that can only count entrance and exit sections. By using a correction formula that subtracts duplicate statistics from single-point statistics, the system can accurately invert the real-time number of people staying in large open areas such as platforms, transfer passages, and waiting halls. This not only eliminates the duplicate counting errors caused by overlapping fields of view of multiple cameras, but also provides subway operation scheduling with a more granular basis for real-time passenger flow distribution within the station than entry and exit data.

[0030] Optionally, step S1 calculates a density index to characterize the crowding level in the current scene, including: using a trained crowd density estimation model to generate a density map corresponding to the image frame, calculating an average density value based on the density map, and using the average density value as the density index.

[0031] The input image is processed using a model such as CSRNet or SANet as the front end, undergoing full convolution. The model outputs a single-channel density map proportional to the size of the original image, where the value of each pixel represents the probability of a person being present at that location. By averaging the pixel values ​​across the entire density map, a density index reflecting the overall crowding level of the current scene can be obtained.

[0032] Compared to directly counting the number of detection boxes, the density index based on pixel regression can more accurately capture the characteristics of crowds in extremely crowded scenes, avoiding the problem of detection boxes failing due to severe occlusion and thus being unable to determine the degree of crowding.

[0033] Optionally, step S2 compares the density index with preset switching conditions, specifically including: Obtain the preset density threshold; When the average crowd density is less than the density threshold, the sparse scene condition is met; when the average crowd density is greater than or equal to the density threshold, the dense scene condition is met.

[0034] This embodiment uses a preset empirical threshold (such as...) in the system. or after physical calibration When the average density value calculated in real time is below the threshold, the system determines it to be an off-peak period and calls the YOLO model to obtain precise individual locations; conversely, it determines it to be a peak period and calls the density estimation model to ensure the accuracy of the counting trend. In this way, the optimal allocation of computing resources and statistical accuracy is achieved. In sparse conditions, the detection model provides rich location information, while in crowded conditions, the density model resists occlusion interference, ensuring the system stability under the tidal passenger flow changes of the subway.

[0035] Optionally, the first statistical strategy based on object detection includes: using an object detection model to extract candidate objects from image frames, filtering valid objects based on confidence, and using the number of valid objects as the population statistics result.

[0036] In sparse mode, the YOLOv7 model is used to infer the image, outputting bounding boxes containing category and confidence score. The system filters low-quality targets by setting a confidence threshold and uses the NMS algorithm to remove redundant overlapping boxes targeting the same pedestrian, finally counting the number of remaining boxes. This embodiment leverages the advantage of the object detection model's accurate localization in sparse scenes, accurately counting each discrete pedestrian and compensating for the deficiency of high integral noise in density maps in sparse scenes.

[0037] Optionally, a second statistical strategy based on density estimation includes summing or integrating the pixel values ​​of the density map and using the result as the population count.

[0038] In dense mode, the density map generated in step S1 is directly used to perform integration (summation) on all pixels in the image. The integration result is mathematically equivalent to the predicted total number of people in the visible area. This embodiment effectively solves the severe occlusion problem caused by overcrowding during subway rush hours. Because it does not rely on separate detection boxes, even if only the head or part of a pedestrian's limbs are visible, the model can still contribute a corresponding density value, thus ensuring that the total count does not suffer from a collapse-like missed detection.

[0039] Optionally, step S3 involves filtering adjacent camera pairs with physical connectivity based on spatial topological relationships, specifically including: Construct a topology map describing the distribution of cameras within the monitored area. Calculate the topological distance between each camera node; group cameras with a topological distance of 1 and identify them as adjacent camera pairs with physical connectivity.

[0040] The subway station cameras are considered nodes in a topological graph, and the physical pathways (such as stairs and corridors) between cameras are considered edges. A "pair of adjacent cameras" is defined as having a topological distance of 1 only when a person can move directly from the field of view of camera A to the field of view of camera B without passing through other cameras. This embodiment simplifies the complex cross-camera matching problem into a local adjacent matching problem. This not only greatly reduces the number of feature comparisons but also prevents algorithmic misjudgments caused by instantaneous movement through physical constraints.

[0041] Optionally, step S4 involves performing cross-camera duplicate person identification for adjacent camera pairs, specifically including: Extract the appearance feature vectors of pedestrians in the image frames of adjacent camera pairs; Calculate the cosine similarity between the appearance feature vectors of different pedestrians from two cameras; If the cosine similarity is greater than the preset similarity threshold, the two pedestrians are determined to be the same person and are counted in the number of repeated people across cameras.

[0042] By extracting pedestrian image patches detected from the fields of view of two adjacent cameras, normalizing them, and inputting them into a Re-ID model to extract appearance feature vectors, the cosine similarity between the two appearance feature vectors is calculated. If the similarity exceeds a similarity threshold (e.g., 0.7), they are determined to be the same person. This embodiment achieves cross-field identity association, enabling the identification of the same passenger crossing the overlapping or intersecting areas of cameras, which is the basis for calculating the number of duplicates.

[0043] In some embodiments, the formula for calculating cosine similarity is: ; in, Let be the pedestrian feature vector from camera A. This is the pedestrian feature vector from camera B. A similarity threshold is set; if the similarity is greater than the set threshold, the person is considered a duplicate.

[0044] Optionally, in step S4, the overall pedestrian flow in the monitored area is calculated using the following formula: ; in, For the overall foot traffic, This is the sum of the number of people counted from all individual cameras. This is the sum of the number of people repeating across cameras between all adjacent camera pairs.

[0045] The system backend aggregates the independent counts from all individual cameras in real time and subtracts duplicate counts identified between adjacent camera pairs. This integrates isolated monitoring sources into a unified station-level passenger flow data, solving the problem of duplicate counts caused by the simple aggregation of data from multiple cameras in traditional methods.

[0046] Optionally, the crowd density estimation model employs a network structure based on backend dilated convolutions, specifically including: In the backend network of the crowd density estimation model, multiple cascaded dilated convolutional layers are deployed; The dilation rate of the dilated convolutional layer is set to a value greater than 1 to expand the receptive field of feature extraction while maintaining resolution. The feature map output by the dilated convolutional layer is mapped to a single-channel density map through convolution operations.

[0047] In the back end of the crowd density estimation model, convolutional layers with a dilation rate of 2 are stacked; this structure allows the convolutional kernels to expand the receptive field exponentially without reducing the feature map resolution (i.e., without downsampling). This embodiment preserves the spatial details of the image (crucial for recognizing tiny heads in the distance) while acquiring sufficient contextual information (facilitating the differentiation of heads from background clutter), significantly improving the quality of the density map.

[0048] Optionally, the object detection model performs category filtering, retaining detection results belonging to the human category as candidate objects, and selecting valid objects based on confidence.

[0049] After inference in the YOLO model, all detection boxes that are not in the "human" category (such as backpacks, suitcases, etc.) are forcibly discarded. During the training phase, the weights of the loss function are adjusted to reduce the proportion of classification loss, allowing the model to focus on locating "humans". This embodiment is specifically optimized for the subway scenario, reducing false positives for non-humans and slightly improving the model's inference speed and convergence efficiency due to the simplification of the category.

[0050] In some embodiments, before sampling the video stream in step S1, an image normalization step is further included: Obtain the pre-defined standardized parameters, including the mean vector and the standard deviation vector; The pixel values ​​of the image frame are normalized using standardized parameters to make them conform to the distribution requirements of the model input.

[0051] Before the images are fed into the neural network, the mean of the dataset is subtracted and divided by the standard deviation to standardize the pixel value distribution to near zero mean. This embodiment eliminates the influence of different lighting conditions and differences in imaging styles of different camera brands, accelerates the convergence speed of the deep learning model, and improves the generalization ability of the system.

[0052] In some embodiments, the process of extracting pedestrian appearance feature vectors employs a multi-scale feature fusion mechanism, specifically including: Image frames are input into a full-scale network, and local detail features and global semantic features under different receptive fields are extracted through multiple parallel convolutional branches; Channel-level adaptive weighting units are used to assign weight coefficients to branches of different scales, and the weighted features are fused along the channel dimension. Global average pooling is performed on the fused feature maps to generate appearance feature vectors.

[0053] A full-scale network (OSNet) is used as the feature extractor. Its internal modules contain multiple parallel convolutional streams to extract fine-grained texture (such as clothing logos) and coarse-grained body shape (such as body proportions), respectively. Finally, these features are fused through adaptive weighting. In this way, the scale variation problem commonly found in Re-ID is solved (e.g., a person is close in camera A but far in camera B); multi-scale fusion ensures that the feature vector is insensitive to distance changes.

[0054] In some embodiments, the crowd density estimation model is trained on a training set containing real crowd annotation information, and its loss function is formulated as follows: ; in, To predict the mean squared error loss between the density map and the true density map, To predict the loss due to the absolute difference between the total number of people and the actual total number of people, and These are the preset weighting coefficients.

[0055] When training the density model, both the pixel-level mean squared error (MSE) and the total number of people absolute error (L1) are minimized simultaneously. The pixel-level mean squared error (MSE) is used to constrain the shape of the density distribution, while the total number of people absolute error (L1) is used to constrain the accuracy of the total number of people. In this way, the model can avoid situations where the total number is predicted correctly but the location is completely wrong, or the location is correct but the numerical deviation is large; and a weighted combination achieves dual optimization of distribution accuracy and counting accuracy.

[0056] In some optional embodiments, determining that the two corresponding pedestrians are the same person also includes a spatiotemporal consistency verification step: obtaining the physical path length between the monitoring areas of the two cameras, and calculating the minimum and maximum passage time required for a pedestrian to cross the adjacent camera pair based on the preset average pedestrian passage speed; calculating the timestamp difference between the two pedestrian image frames to be matched; only when the timestamp difference is within the effective time window formed by the minimum passage time and the maximum passage time is it confirmed that the two pedestrians are the same person, otherwise even if the cosine similarity is greater than the threshold, they are determined to be different people.

[0057] For example, the system pre-stores the physical distance between cameras A and B (e.g., 10 meters). If a pedestrian's normal walking speed is 1.2 m / s, the theoretical crossing time is approximately 8 seconds. The system sets a time window (e.g., 6-12 seconds). Feature comparison is only allowed when the time difference between the images captured by the two cameras falls within this window.

[0058] In this way, it can effectively eliminate the possibility of two passersby who look extremely similar but are unlikely to cross a long distance in a short period of time being mistaken for the same person, and significantly reduce the false positive rate of Re-ID.

[0059] In some optional embodiments, the density index is compared with preset switching conditions, and a hysteresis comparison strategy is adopted to prevent frequent strategy switching. Specifically, this includes: preset high threshold and low threshold, wherein the high threshold is greater than the low threshold; if the current scene mode is sparse, the switch to dense scene mode is only made when the average population density value continues to rise and exceeds the high threshold; if the current scene mode is dense, the switch to sparse scene mode is only made when the average population density value continues to fall and falls below the low threshold; when the average population density value is between the high threshold and the low threshold, the current population counting strategy remains unchanged.

[0060] Specifically, set a high threshold. and low threshold When the density value changes from low to high, it must exceed... Only then should you switch to dense mode; when the density value changes from high to low, it must be lower than [the specified value]. Only then did I switch back to sparse mode.

[0061] In some alternative embodiments, before extracting candidate targets using the target detection model, a static interference removal step is also included: performing background modeling on consecutive frames of the video stream and extracting a foreground mask containing motion information; performing a logical AND operation between the foreground mask and the image frame to generate an image to be detected containing only the motion region; and inputting the image to be detected into the target detection model to filter out static humanoid interference targets belonging to billboards or posters in the subway station scene.

[0062] Background modeling is performed on the video stream using a Gaussian mixture model (GMM) or frame difference method to extract a mask of the moving foreground region. This mask is then multiplied with the original image to turn the static billboard portrait area in the original image black (zero value), and then fed into the detection model.

[0063] In some optional embodiments, the construction process of the topology map has the ability to be dynamically updated. Specifically, it records pedestrian trajectory data that are successfully matched across cameras in historical time periods; it counts the pedestrian transfer frequency between each camera combination; if the pedestrian transfer frequency between a non-adjacent camera combination exceeds a preset connectivity threshold, it automatically establishes an edge between the two camera nodes in the topology map and updates the topology distance to 1 to adapt to changes in spatial relationships caused by changes in station passage structure or temporary diversion measures.

[0064] The system may have a basic topology configured during the initialization phase. During operation, if the system detects a large number of pedestrians appearing consecutively between two cameras marked as "non-adjacent" with extremely high confidence (and conforming to temporal logic), it automatically determines that there is an unrecorded passage between these two cameras and adds a new edge to the topology graph. When temporary traffic diversions occur in the subway station (such as setting up warning lines to change passages) or internal renovations cause changes in routes, the system can adapt to the new spatial structure without manual reconfiguration, reducing operation and maintenance costs.

[0065] This disclosure also provides a passenger flow statistics system for subway hub stations. The system mainly consists of three parts: a front-end acquisition unit, edge computing nodes, and a central processing server. The front-end acquisition unit includes several high-definition surveillance cameras distributed at the entrance gates, security checkpoints, escalator entrances in the concourse, and platform waiting areas. All cameras are connected to the local area network via fiber optic network, and the video stream resolution is uniformly set to 640×640 pixels to meet the model input requirements.

[0066] In terms of data processing logic, this embodiment adopts a distributed architecture to reduce transmission bandwidth pressure. Each camera terminal is equipped with an edge calculator equipped with a lightweight AI acceleration card. The edge calculator is responsible for executing steps S1 and S2, acquiring video streams in real time and performing frame extraction (e.g., 5 frames per second), and running a preset crowd density estimation model locally to perform initial screening of the images. If the calculated average crowd density value is less than a preset threshold... The edge computing unit automatically calls the YOLOv7 object detection model to output the number of pedestrian bounding boxes; if the number exceeds a threshold, it directly outputs the integral value of the CSRNet density map. The edge calculator only uploads the final number of people, extracted pedestrian appearance feature vectors (for adjacent area cameras), and necessary timestamp metadata to the central processing unit (CPU), instead of transmitting the raw high-definition video stream, thus significantly reducing network load. The CPU then focuses on maintaining a topology map of all cameras in the site and performs cross-camera deduplication logic based on the uploaded feature vectors.

[0067] Furthermore, this embodiment uses the No. 1 camera on a subway platform as an example to specifically illustrate the adaptive counting logic under a single camera. The system first performs ImageNet normalization preprocessing on the input RGB three-channel image. The preprocessed image is then input into a density estimation network with the first 10 layers of VGG-16 as the front end and dilated convolutions as the back end.

[0068] The system calculates the global pixel mean of the output density map in real time. Assuming it's currently rush hour and there are many people waiting at the platform, the calculated... for It is significantly greater than the system's preset switching threshold. (This example is set as) At this point, the system determines that the current frame meets the dense scene conditions, automatically blocks the target detection branch, and directly accumulates and integrates all pixel values ​​in the density map to obtain the predicted number of people in the frame (e.g., 156 people).

[0069] As the peak hours began, only a few passengers were visible in the footage. Down to The value is below the switching threshold. The system then activates the first statistical strategy based on YOLOv7, retaining only the detection boxes for the "person" category and setting the confidence threshold to 0.5 for filtering. At this point, the model accurately outputs the specific number of detection boxes (e.g., 5 people), avoiding the artificially inflated counts that may occur due to background noise in the density map, and achieving accurate adaptation to dense and sparse scenes.

[0070] This embodiment uses a cross-camera deduplication method based on topological constraints. The implementation process of multi-camera collaborative deduplication is as follows: The system pre-constructs a topological map describing the spatial structure of the subway station in the database. Assume that camera 1 covers the area of ​​the descending escalator, and camera 2 covers the area on the east side of the platform connected to it. The two cameras are physically connected, therefore their distance is defined in the topology diagram. These form a pair of adjacent cameras; however, camera 4 is located on the west side of the platform, requiring the entire platform to be traversed to reach the field of view of camera 1, hence the definition... They do not constitute adjacent pairs.

[0071] During operation, the central processing unit (CPU) performs Re-ID matching only for cameras 1 and 2, which are topologically distant by 1. The system calls the OSNet full-scale network to extract the appearance feature vectors of pedestrians from the fields of view of both cameras. If the feature vector of pedestrian A detected by camera 1... The feature vector of pedestrian B detected by camera 2 The cosine similarity between the two passengers is 0.85 (higher than the preset similarity threshold of 0.75), so the system determines that the two passengers are the same person, indicating that the passenger is moving from the escalator area to the platform area.

[0072] When calculating the total pedestrian flow, the system adds the count from camera 1 (e.g., 50 people) to the count from camera 2 (e.g., 80 people) and subtracts the number of duplicates (e.g., 10 people identified moving across areas). The final net pedestrian flow for the combined area is 50 + 80 - 10 = 120 people. For non-adjacent cameras 1 and 4, the system skips the feature matching step, thus completely avoiding the risk of false deduplication caused by similar clothing but large distances.

[0073] Example 1: The specific structure of the crowd density estimation model based on the CSRNet architecture includes an input layer, a front-end feature extraction network, a back-end dilated convolutional network, and an output layer connected in sequence, wherein: Input layer: The subway station surveillance images captured by the camera are input through the input layer. The default size of the input image is set to (480, 640, 3), which means that the image resolution is 480×640 pixels and the RGB three-channel format is used.

[0074] Preferably, a normalization process is added to the input layer, using ImageNet normalized parameters (mean=[0.485,0.456,0.406], std=[0.229,0.224,0.225]) to improve the model's adaptability to images under different lighting and color conditions.

[0075] Front-end Feature Extraction Network: This part uses the first 10 convolutional layers of the VGG-16 network as the feature extractor to extract multi-scale spatial features. Its layer structure is configured as [64, 64, 'M', 128, 128, 'M', 256, 256, 256, 'M', 512, 512, 512], where 'M' represents a max-pooling layer with a kernel size of 3×3, a stride of 1, and padding of 1. Each convolutional layer uses the ReLU activation function to enhance the non-linear representation of features.

[0076] Preferably, the weights of the convolutional layers in this part are initialized by loading the weights of the ImageNet pre-trained model to achieve transfer learning, thereby achieving rapid convergence in the subway scenario and improving the generalization performance of feature extraction.

[0077] The back-end dilated convolutional network consists of six sequentially connected dilated convolutional layers with a channel configuration of [512, 512, 512, 256, 128, 64]. Each layer uses a 3×3 convolutional kernel with a dilation rate of 2, thereby expanding the receptive field to capture a wider range of population distribution information without increasing the number of parameters.

[0078] The ReLU activation function is applied after convolution to preserve the non-linear feature representation.

[0079] Furthermore, each element y(i,j) in the output feature map Y of the dilated convolution can be represented by the following equation: ; Where x(i,j) is the pixel value of the input feature map, k(m,n) is the weight of the convolution kernel, and d is the dilation rate.

[0080] Output layer: A 1×1 convolutional layer is used to map the 64-channel feature map into a single-channel density map. The size of this density map is consistent with the input image, and each pixel value in the density map represents the crowd density of that region. By summing all pixel values ​​in the density map, the predicted number of people in that frame can be obtained. : .

[0081] Weight initialization and regularization: The back-end dilated convolutional layers without pre-trained parameters are initialized with a Gaussian distribution, with a mean of 0, a standard deviation of 0.01, and a bias of 0. To prevent overfitting, the model employs L2 regularization to ensure parameter stability.

[0082] Model Training: The model was trained using a dataset containing labeled information from real-world crowds. Training samples were generated by manually labeling the center points of pedestrian heads and using ground truth density maps based on a geometrically adaptive Gaussian kernel.

[0083] The model's loss function is a weighted combination of mean squared error (MSE) and L1 headcount constraint loss: ; in, For model-predicted density maps, This is the true density map. The predicted and actual number of people are respectively; α and β are weighting coefficients, preferably α=1 and β=0.001.

[0084] The optimization algorithm uses the Adam optimizer with an initial learning rate of 1e-5 and a batch size of 4.

[0085] Model Inference: In actual deployment, the system calls the trained CSRNet model to perform inference on images captured by real-time cameras. The density map output by the model is integrated to obtain the predicted number of people, and the average crowd density is calculated. : ; H and W represent the height and width of the density map, respectively.

[0086] Density determination and threshold setting: This involves setting the average population density... With preset density threshold Comparisons are made to distinguish between sparse and dense scenes.

[0087] Preferred, Set to without pixel physical calibration If pixel area calibration has been completed, then 1.0 person / ㎡ can be used as the physical threshold.

[0088] when ≥ When a region is identified as dense, it is handed over to the CSRNet model for statistical analysis; when < When the area is identified as sparse, the YOLO detection model is used to calculate the number of people.

[0089] Visualization and Results Output: The output density map is displayed on the crowd monitoring interface after pseudo-color rendering. The Jet color mapping scheme is used, with red areas representing high-density crowds and blue areas representing sparse crowds. The system interface simultaneously displays the predicted number of people and the average crowd density value, providing a basis for subsequent crowd monitoring and risk warning.

[0090] The CSRNet (Congested Scene Recognition Network) model in this embodiment can achieve high-precision crowd counting in high-density scenes in subway stations, significantly improving the accuracy of crowd flow statistics and the intelligence level of the system.

[0091] Example 2: This example provides a YOLOv7-based method for people detection in subway station surveillance scenes, used for pedestrian flow statistics in sparsely populated areas, complementing the CSRNet density estimation method in Example 1. This method achieves automatic identification and counting of pedestrians in a single frame image through object detection, featuring fast detection speed, lightweight model, and easy deployment. The overall model processing flow includes an input layer, a front-end feature extraction network, a multi-scale feature fusion module, a detection layer, and an output layer.

[0092] Input Layer: Real-time monitoring images or static monitoring frames of subway stations are input through the input layer. The default input image size is set to (640, 640, 3), which means a resolution of 640×640 pixels and an RGB three-channel format.

[0093] Preferably, the input image is preprocessed by normalization in the input layer to normalize the pixel values ​​to the range of [0,1] and normalize them according to the COCO dataset standard to enhance the robustness of the model under different lighting conditions and camera models.

[0094] Front-end feature extraction network: YOLOv7's front-end feature extraction network is used to extract multi-scale spatial features, including multi-layer convolution, bottleneck structure, and E-ELAN feature enhancement module. Its main structure can be described by the channel configuration from top to bottom according to the network depth as: [64, 128, 256, 512, 1024]; Each convolutional layer uses a 3×3 kernel with a stride of 1 or 2, achieving spatial downsampling through convolutions with a stride of 2. All convolutional operations are accompanied by batch normalization and the SiLU activation function to enhance nonlinear expressiveness.

[0095] The E-ELAN (Extended Efficient Layer Aggregation Network) structure extends the gradient path through multi-branch convolution and cross-layer feature fusion, which is beneficial to improving the feature representation ability of the network in the subway pedestrian detection task.

[0096] Furthermore, the convolutional layer weights of the front-end feature extraction network are initialized using pre-trained model parameters from the COCO dataset to achieve transfer learning, enabling the model to converge faster and achieve higher recognition accuracy in subway scenarios.

[0097] Multi-scale feature fusion module (SPPCSPC): The deepest layer output of the front-end feature extraction network is a multi-scale feature fusion module, which is used to extract multi-scale contextual information.

[0098] This module uses three different pooling kernel sizes (5×5, 9×9, and 13×13) to pool the feature maps, and then concatenates the pooled multi-scale features with the original features to enhance the model's ability to detect large-scale scenes and distant pedestrians in subway stations.

[0099] Preferably, the module adopts a CSP (Cross Stage Partial) substructure to reduce computation while maintaining feature representation capability.

[0100] Detection Layer: YOLOv7 employs a Path Aggregation Network (PANet) structure for multi-scale feature fusion, including upsampling, concatenation, and convolution operations to combine features from different resolutions. The fused features are fed into three detection heads, corresponding to 80×80, 40×40, and 20×20 feature maps respectively. The goal is to detect pedestrians at different scales: the large feature map (80×80) is used to detect small pedestrians, the medium feature map (40×40) is used to detect regular-sized pedestrians, and the small feature map (20×20) is used to detect large pedestrians in close-up.

[0101] Each detection box outputs five parameters: center coordinates (x, y), width w, height h, and confidence score for each detection box, and predicts the target category "person".

[0102] Its core calculation formula is: ; in, These are the prediction parameters, The center position of the anchor frame. This refers to the anchor frame dimensions.

[0103] Preferably, in this embodiment, only class 0 (person category) in COCO is retained for pedestrian detection tasks, thereby improving inference speed and detection stability.

[0104] Output layer: Non-maximum suppression (NMS) algorithm is used to remove duplicate detection boxes and retain only the detection results with the highest confidence.

[0105] The coordinates and confidence scores of all detection boxes in each frame are output, and the number of detection boxes with a confidence score greater than a set threshold is recorded as the number of people in that frame.

[0106] Model training: The model is trained based on publicly available pre-trained weights YOLOv7 and fine-tuned using transfer learning.

[0107] The training samples consist of subway station surveillance footage, and the annotation format conforms to the YOLO standard. Each label includes a category number, target center coordinates, and normalized width and height parameters.

[0108] During training, a batch size of 32, an input resolution of 640×640, the optimization algorithm of Adam, an initial learning rate of 1e-3, and 50 training epochs were used.

[0109] The loss function consists of localization loss. Confidence loss and classification loss The composition, specifically the formula, is as follows: ; in =0.05, =1.0, =0.5.

[0110] Preferably, in this embodiment only the human category is detected, therefore the classification loss is... The proportion of [something] decreases, thereby accelerating the model convergence speed.

[0111] Model Inference and People Counting: During the inference phase, the system calls the trained YOLOv7 model to perform target detection on the subway station surveillance images. After parsing the coordinates of the human detection boxes output by the model, the system counts the number of detection boxes with a confidence level greater than a set threshold, directly counts the number of "human" category targets detected, and obtains the total number of people in that frame of the image.

[0112] The system automatically records and saves the detection results, and the labeled images are output to the specified directory.

[0113] Results Output and Visualization: The detection results are displayed as bounding boxes on the crowd monitoring interface, with human targets highlighted in purple. The result image file is saved to the specified directory with the original image filename, and the detection box information is saved as a text file in the folder.

[0114] The YOLOv7 model in this embodiment enables real-time detection and counting of people in sparse areas within a subway station monitoring system. Combined with the CSRNet density estimation model, it forms an adaptive switching mechanism between sparse and dense scenes, thereby improving the overall intelligence level and response speed of the passenger flow statistics system.

[0115] Example 3: This example provides a deduplication method based on deep learning and cross-camera pedestrian re-identification (Re-ID) fusion with spatial constraints to solve the problem of duplicate counting in multi-camera collaborative statistics. This example is applicable to real-world scenarios in subway stations with multiple monitoring perspectives, strong occlusion, and complex camera distribution. It can be directly interfaced with the single-camera people counting results generated in Example 1 (CSRNet) and Example 2 (YOLOv7).

[0116] The overall process of this embodiment includes camera spatial topology constraint judgment, input layer, pedestrian detection and cropping module, feature extraction network, multi-camera feature matching module, and repeat person counting module.

[0117] Camera spatial topology constraint judgment: In multi-camera collaborative perception scenarios, the spatial location and connectivity of different cameras within a subway station determine the accessibility and travel distance of people between cameras. To avoid mismatches between cameras that do not conform to the actual spatial structure, a spatial constraint judgment mechanism based on camera topology is introduced.

[0118] First, based on the actual spatial layout of the subway station, each camera is abstracted as a node in the topology graph, and the connection relationship between cameras that allows people to pass through is abstracted as an edge, thus constructing a camera topology graph G=(V,E), where V represents the set of camera nodes and E represents the set of connectivity relationships between cameras.

[0119] In the camera topology diagram, the shortest path length between camera A and camera B is defined as d(A,B). The path length represents the minimum number of path segments a person needs to traverse when moving from the monitoring area of ​​camera A to the monitoring area of ​​camera B. For example... Figure 7 As shown, the monitoring areas of camera 1 and camera 2 are adjacent and overlap, and their topological distance d(1,2)=1; while the areas of camera 1 and camera 4 are not directly connected and need to be connected through camera 2, so d(1,4)=2.

[0120] In a preferred embodiment, cross-camera duplicate person determination is performed only for pedestrian targets between adjacent cameras with a topological distance of 1; for camera combinations with a topological distance greater than 1, duplicate person determination is not performed, thereby avoiding unreasonable matching across regions and paths.

[0121] Input Layer: Real-time monitoring images from multiple cameras are input to the cross-camera deduplication module. Input content includes: image frames from each camera, pedestrian bounding box coordinates (x, y, w, h) detected by YOLOv7 in Example 2, camera ID, frame timestamp t, and camera topology (adjacency relationships). The input is an RGB image of variable size. The single pedestrian image after cropping is input at a fixed size of 256×128 and is standardized to provide consistent input for subsequent feature extraction.

[0122] Pedestrian detection and pedestrian image cropping module: In order to achieve pedestrian re-identification, this embodiment relies on the "person" detection box output by YOLOv7 in embodiment 2.

[0123] The steps are as follows: 1) Perform YOLOv7 inference on the image frames from the camera to obtain N pedestrian bounding boxes; 2) Crop each pedestrian image based on the bounding box; 3) Scale the pedestrian images to 256×128 and perform normalization processing; This step outputs the cropped pedestrian image and its corresponding metadata (camera ID, timestamp, original image coordinates).

[0124] Pedestrian Re-identification Feature Extraction Network (ReID Backbone): This embodiment utilizes a deep learning pedestrian re-identification network to extract appearance features from cropped pedestrian images, enabling the identification of the same passenger across different cameras. Optional models include OSNet, ResNet, etc.

[0125] All models take 256×128 pedestrian images as input and output normalized high-dimensional feature vectors for subsequent cross-camera matching.

[0126] Based on the OSNet model structure and features, OSNet (Omni-Scale Network) extracts pedestrian appearance features through multi-scale feature fusion units. OSBlock can simultaneously acquire features from different receptive fields, thereby improving its ability to handle pose changes and occlusion scenes; specifically, it includes the following components: The input is a 256×128 pedestrian image; The full-scale feature extraction module internally sets up multiple parallel convolutional branches, each corresponding to a different effective receptive field, to extract multi-scale feature information from pedestrian images. Each parallel convolutional branch consists of lightweight convolutional units with different numbers of layers to extract local detail features, mesoscale structural features, and global semantic features respectively. The output features of the multi-scale parallel feature extraction unit are weighted by a channel-level adaptive weighting unit to assign corresponding weight coefficients to different scale branches. The weighted multi-scale features are then fused along the channel dimension to form a unified output feature map. Multiple full-scale feature extraction modules are cascaded and stacked in a preset order to form the backbone feature extraction network of OSNet. As the network layers deepen, the output features gradually transition from low-level visual features to high-level semantic features, which are used to characterize the pedestrian's identity information. At the end of the backbone network, OSNet uses global average pooling to compress the spatial features into a fixed-length one-dimensional feature vector, thereby eliminating the influence of changes in pedestrian image size. The global feature vector is then linearly mapped and normalized through the embedding layer to obtain the final feature representation for pedestrian re-identification.

[0127] As another implementation method, this embodiment can use ResNet (such as ResNet-50) to extract pedestrian appearance features. This network alleviates the gradient degradation problem in deep network training through residual structures and is a classic backbone network widely used in pedestrian re-identification tasks; specifically, it includes the following parts: The input is a normalized 256×128 pedestrian image; A pedestrian feature extraction model is constructed using ResNet as the backbone. The ResNet network consists of an input layer, a convolutional feature extraction layer, multiple residual modules, and a feature output layer connected sequentially. The residual modules introduce a shortcut connection structure, allowing the network to add the input and output features simultaneously during convolution operations. This achieves residual learning, avoiding performance degradation caused by increasing the number of network layers and improving the stability and effectiveness of feature extraction.

[0128] The preprocessed pedestrian images are input into a ResNet network, where multi-layer convolutional operations and residual modules are used to extract the appearance features of pedestrians step by step. As the number of network layers increases, the extracted features gradually transition from low-level color and texture information to high-level human body structure and semantic features, thereby obtaining a deep feature representation of pedestrians with strong discriminative ability.

[0129] Global average pooling is performed on the pedestrian feature map output by the ResNet network to aggregate the feature information in the spatial dimension and generate a pedestrian feature vector of fixed dimension.

[0130] Feature matching module: After obtaining pedestrian features extracted from all cameras, this embodiment uses cosine similarity for matching.

[0131] For the pedestrian feature vector in camera A , and the pedestrian feature vector in camera B : ; The higher the similarity, the more likely the two pedestrian images are from the same person.

[0132] In this embodiment, the threshold is ,like ≥ If they are the same person, then they are considered to be the same person.

[0133] Duplicate People Count Module: For each pair of cameras, the matching pairs generated (A i B j That is, if the i-th person at camera A and the j-th person at camera B are determined to be the same person, they are counted as duplicates. ; Final total number of people calculation formula: .

[0134] Model training: Pre-training was performed using a public pedestrian re-identification dataset, followed by fine-tuning using samples collected from subway scenes.

[0135] The foregoing description and accompanying drawings fully illustrate embodiments of the present disclosure to enable those skilled in the art to practice them. Other embodiments may include structural and other changes. The embodiments represent only possible variations. Individual components and functions are optional unless explicitly required, and the order of operation may vary. Parts and features of some embodiments may be included or substituted for parts and features of other embodiments. Embodiments of the present disclosure are not limited to the structures described above and shown in the accompanying drawings, and various modifications and changes may be made without departing from its scope. The scope of the present disclosure is limited only by the appended claims, and the foregoing embodiments should be considered exemplary and non-limiting.

Claims

1. A method for regional pedestrian flow statistics based on multi-camera collaborative perception, characterized in that, Includes the following steps: S1. Acquire real-time video streams from multiple cameras distributed within the monitoring area, and sample the video streams to calculate a density index reflecting the crowding level in the current scene. S2. Compare the density index with the preset switching conditions, and adaptively execute the number of people counting strategy based on the comparison result; When the density index meets the sparse scene conditions, the first statistical strategy based on object detection is used to count the number of people in the image. When the density index meets the dense scene conditions, a second statistical strategy based on density estimation is used to count the number of people in the image. S3. Obtain the spatial topology relationship between each camera, and filter out adjacent camera pairs with physical connectivity based on the spatial topology relationship; S4. For the adjacent cameras, perform cross-camera duplicate person identification, count the number of duplicate people across cameras; and combine the single-point count of people from each camera with the number of duplicate people to calculate the overall traffic flow in the monitored area.

2. The method according to claim 1, characterized in that, Step S1 involves calculating a density index to characterize the crowding level in the current scene, including: Using a trained crowd density estimation model, a density map corresponding to the image frame is generated, and an average density value is calculated based on the density map. The average density value is then used as the density index.

3. The method according to claim 2, characterized in that, Step S2 compares the density index with preset switching conditions, specifically including: Obtain the preset density threshold; When the average population density value is less than the density threshold, it is determined that the sparse scene condition is met. When the average population density value is greater than or equal to the density threshold, it is determined that the dense scene condition is met.

4. The method according to claim 1, characterized in that, The first statistical strategy based on object detection includes: using an object detection model to extract candidate objects from image frames, filtering valid objects based on confidence, and using the number of valid objects as the result of the population statistics.

5. The method according to claim 2, characterized in that, The second statistical strategy based on density estimation includes: summing or integrating the pixel values ​​of the density map and using the result as the population statistics.

6. The method according to claim 1, characterized in that, Step S3, which involves filtering out adjacent camera pairs with physical connectivity based on the spatial topology, specifically includes: Construct a topology map describing the distribution of cameras within the monitored area, and calculate the topology distance between each camera node; Camera pairs with a topological distance of 1 are identified as adjacent camera pairs with physical connectivity.

7. The method according to claim 1, characterized in that, Step S4, which involves performing cross-camera duplicate person identification for adjacent cameras, specifically includes: Extract the appearance feature vectors of pedestrians in each image frame of the adjacent camera pairs; Calculate the cosine similarity between the appearance feature vectors of different pedestrians from two cameras; If the cosine similarity is greater than the preset similarity threshold, the two pedestrians are determined to be the same person and are counted in the number of repeated people across cameras.

8. The method according to claim 1, characterized in that, In step S4, the overall pedestrian flow in the monitored area is calculated using the following formula: in, For the overall foot traffic, This is the sum of the number of people counted from all individual cameras. This is the sum of the number of people repeating across cameras between all adjacent camera pairs.

9. The method according to claim 2, characterized in that, The crowd density estimation model employs a network structure based on backend dilated convolutions, specifically including: In the back-end network of the crowd density estimation model, multiple cascaded dilated convolutional layers are deployed; The dilation rate of the dilated convolutional layer is set to a value greater than 1 in order to expand the receptive field of feature extraction while maintaining resolution. The feature map output by the dilated convolutional layer is mapped to the density map of a single channel through a convolution operation.

10. The method according to claim 4, characterized in that, The target detection model retains the detection results for human targets and filters and counts the number of valid targets based on confidence levels.

Images