Pedestrian flow monitoring method and system for dense sitting posture scene

Abstract

The invention relates to a human traffic monitoring method and system for a dense sitting posture scene, and the method comprises the steps: S1, obtaining the video stream data of the dense sitting posture scene, constructing an original data set, and optimizing the size distribution of an anchor frame; s2, constructing a human traffic monitoring model which is improved and realized on the basis of a YOLOv7 model: in a backbone network, replacing the four starting CBS modules with ODSConv modules, and replacing the two subsequent ELAN modules with E-ELAN-Sim modules; in the head network, a coordinate attention mechanism is embedded in an SPPCSPC module to obtain a CA-SPPCSPC module, the CA module is added in front of a CBS module connected with an ELAN-H module and a Concat module, and meanwhile, a four-scale detection framework is realized in the head network to enhance the small target detection capability; training the constructed people flow supervision model through the training data set; and S3, inputting real-time video stream data into the trained people flow supervision model, and outputting a people flow supervision quantity. According to the method and the system, accurate detection and real-time statistics of the sitting posture target person in the dense sitting posture scene can be realized.

Description

Technical Field

[0001] This invention belongs to the field of target detection technology, specifically relating to a method and system for monitoring pedestrian traffic in densely seated scenarios. Background Technology

[0002] Deep learning-based object detection technologies (such as YOLOv7) have made significant progress in general scenarios; however, their practical application in extremely dense crowd environments such as transportation hubs and large venues still faces severe challenges. Traditional methods suffer from insufficient spatial sensitivity in multi-scale feature fusion. For example, when the standard SPP module extracts features using a fixed pooling kernel, it ignores the spatial correlation of overlapping areas of human limbs, resulting in a feature confusion rate of 42.3% for small targets smaller than 30×40 pixels (test results on the COCO-Person dataset). Furthermore, the feature extraction backbone network consumes approximately 35% of its computational resources for background noise processing due to the lack of a channel-space joint attention mechanism, severely limiting real-time performance. In addition, convolutional kernels with fixed geometric structures struggle to adapt to complex human posture changes such as sitting posture and backpack occlusion, leading to a 18.6% increase in false negatives compared to upright pedestrians in posture deformation scenarios. Although some studies have attempted local optimization by increasing the detection head scale or introducing attention modules, single improvement schemes often result in a surge in computational complexity or a decrease in generalization ability. For instance, while a four-scale detection head improves the recall rate for small targets, it increases inference latency. Existing technologies lack systematic and coordinated optimization of feature selection, deformation modeling, and multi-objective exclusion constraints, making it difficult to meet the dual requirements of high detection accuracy and real-time processing in high-density seated human scenarios. This severely restricts the practical application of human traffic supervision in densely seated scenarios. Summary of the Invention

[0003] The purpose of this invention is to provide a method and system for monitoring pedestrian traffic in densely seated scenarios. This method and system can achieve accurate detection and real-time statistics of seated target people in densely seated scenarios.

[0004] To achieve the above objectives, the technical solution adopted by the present invention is: a method for monitoring pedestrian traffic in densely seated scenarios, comprising:

[0005] S1: Obtain video stream data of dense seated scenes, construct the original dataset and optimize the anchor box size distribution to obtain the training dataset;

[0006] S2: Construct a pedestrian flow supervision model, which is an improved version of the YOLOv7 model. In the backbone network, the first four CBS modules are replaced with ODSConv modules, and the first two of the subsequent four ELAN modules are replaced with E-ELAN-Sim modules to enhance geometric deformation adaptation and dynamic feature selection capabilities. The ODSConv module is implemented by combining the ODSConv module with the SReLU activation function. The E-ELAN-Sim module is implemented by concatenating the E-ELAN module with the SIMAM module. In the head network, a coordinate attention mechanism is embedded in the SPCSPC module to obtain the CA-SPPCSPC module, and a CA module is added before the CBS module connecting the ELAN-H module and the Concat module to enhance the representation capability of multi-scale features. At the same time, a four-scale detection framework consisting of four detection heads is implemented in the head network to detect ultra-small, small, medium, and large targets to enhance the small target detection capability. The constructed pedestrian flow supervision model is trained using a training dataset to obtain a trained pedestrian flow supervision model.

[0007] S3: Input real-time video stream data into the trained pedestrian flow supervision model and output the number of pedestrian flow supervisions.

[0008] Further, in step S1, video stream data of densely seated scenes including waiting halls and restaurants is acquired, image frames are extracted from the video stream data, and the targets in the image frames are labeled with real target boxes to construct the original dataset.

[0009] For the ground truth target boxes labeled in the original dataset, k-means++ clustering algorithm is used for cluster analysis to obtain the optimal anchor box size distribution; specifically, the following steps are included:

[0010] S101: Obtain the width and height information of all labeled actual target bounding boxes;

[0011] S102: The k-means++ clustering algorithm is used to select the initial centers as cluster centers from all labeled ground truth bounding boxes;

[0012] S103: Calculate the distance between all labeled ground truth bounding boxes and each cluster center using the following formula:

[0013]

[0014] Here, `box` represents the labeled ground truth bounding box, and `centroid` represents the cluster center. This represents the distance between the labeled actual bounding box and the cluster center. This represents the IoU distance between the labeled true bounding boxes and the cluster centers;

[0015] S104: Select k=12 for clustering to obtain 12 anchor frames;

[0016] S105: Sort the obtained anchor frames according to their area and assign them to four detection heads: ultra-small, small, medium, and large. Each detection head uses three anchor frames to cover human targets of different scales.

[0017] Furthermore, in step S2, the implementation method of the ODSConv module is as follows:

[0018] S2101: Insert a Conv2d module before the ODConv module. Perform two-dimensional convolution on the input feature map through the Conv2d module, and then input it into the ODConv module for feature extraction.

[0019] S2102: The output of the ODConv module is batch-normalized through a bn layer and then processed by the SReLU activation function to obtain the output of the ODSConv module; the expression of the SReLU activation function is as follows:

[0020]

[0021] Where SReLU(x) represents the result obtained after processing the input feature map x through the SReLU activation function. , These are the left and right thresholds used to dynamically adjust the activation interval, respectively. , These are the slope coefficients for the negative and positive half-axis, respectively, used to control the nonlinear intensity; , , , All of these are learnable parameters that are dynamically updated during model training.

[0022] Further, in step S2, the implementation method of the E-ELAN-Sim module is as follows:

[0023] S2201: Input feature map It is divided into two paths. The first path obtains the feature map F through the first CBS module. c1 The second path sequentially performs phantom convolutions through five GhostConv modules to obtain five feature maps; the process is as follows:

[0024]

[0025] Where GhostConv1(), GhostConv2(), GhostConv3(), GhostConv4(), and GhostConv5() represent the five GhostConv modules that perform phantom convolution, and F1, F2, F3, F4, and F5 represent the feature maps output by the five GhostConv modules, respectively; X represents the feature map input to the E-ELAN-Sim module, and C, H, and W are the number of channels, height, and width of the feature map, respectively.

[0026] S2202: Convert the feature map F output from the first path... c1 The feature maps F1, F3, and F5 from the second output are concatenated along the channel dimension, as shown below:

[0027]

[0028] Among them, F e This indicates the output of the E-ELAN module, and Concat() represents the concatenation operation;

[0029] S2203: Convert the output F of the E-ELAN module e As input to the SIMAM module, the weighted feature map F is obtained. s Then, after a 1×1 convolution, the output of the E-ELAN-Sim module is obtained:

[0030]

[0031] Among them, F es This represents the feature map output by the E-ELAN-Sim module, Conv 1×1 ( ) represents a 1×1 convolution operation.

[0032] Further, in step S2, the implementation method of the CA-SPPCSPC module is as follows:

[0033] S2301: Transmit the feature map X from the backbone network b The process is divided into two paths. The first path obtains the feature map F through the second CBS module. c2 The second path obtains the feature map F through the third CBS module. c3 Then enter the CA module;

[0034] S2302: For the input feature map F c3 The CA module performs adaptive average pooling along both the horizontal and vertical directions, generating feature vectors after horizontal average pooling. and the feature vector after vertical average pooling To achieve spatial-channel union, feature vectors and Global context information for the row and column dimensions of multiple channels is encoded separately;

[0035] To generate spatial attention weights, the feature vectors are... and After concatenation, a spatial attention weight matrix λ is generated through 1×1 convolution and nonlinear activation:

[0036]

[0037] Here, Concat represents the concatenation operation. This represents a 1×1 convolution to compress the number of channels from 2C to C. This represents the Sigmoid function, ensuring that the weight range is within the range [0,1].

[0038] Multiply the spatial attention weight matrix element-wise with the feature map of the input CA module:

[0039]

[0040] in, This represents the feature map after being weighted by the spatial attention weight matrix λ, i.e., the output of the CA module, which is used to enhance the response of spatially sensitive regions.

[0041] S2303: Feature map Perform multi-scale pooling to generate multi-scale feature maps and concatenate them along the channel dimensions to obtain the output features. :

[0042]

[0043] In the formula, Indicates the feature map Perform max pooling operations with a receptive field of k×k, k=5,9,13, to extract contextual information at different scales;

[0044] S2304: Output features The fourth and fifth CBS modules output feature maps F sequentially. c5 ;

[0045] S2305: Convert the feature map F output from the first path... c2 Feature map F of the second output c5 Channel-dimensional concatenation is performed, followed by feature integration and dimensionality compression through a sixth CBS module based on 1×1 convolution, ultimately yielding the output of the CA-SPPCSPC module to enhance spatial localization capabilities while preserving detailed information.

[0046]

[0047] Among them, F cs This represents the feature map output by the CA-SPPCSPC module.

[0048] Furthermore, in step S2, the implementation method of the four-scale detection framework is as follows:

[0049] To achieve ultra-small target detection, an additional upsampling and downsampling process is added to the feature pyramid of the PAFPN module in the head network. Specifically, after the second upsampling, a first ELAN-H module, a seventh CBS module, a third upsampling module, a first Concat module, a second ELAN-H module, an MP2 module, and a second Concat module are added sequentially between the Concat module and the ELAN-H module at the P3 level. The first Concat module is connected to the first E-ELAN-Sim module of the backbone network through an eighth CBS module, and the first ELAN-H module and the second Concat module are connected through a first CA module and a ninth CBS module. The third upsampling module and the first CBS module are connected through the first CBS module. The oncat module upsamples the 80×80×128 feature map to 160×160×128, and then inputs the resulting 160×160×128 feature map into the second ELAN-H module. The second ELAN-H module outputs ultra-small targets through the RepConv module, thus realizing the first detection head, namely the P2 level detection head. The output of the second ELAN-H module is then used as the input for downsampling and input into the MP2 module. The output of the MP2 module is concatenated with the output of the ninth CBS module by channel dimension, and the result is used as the input of the P3 level ELAN-H module. Thus, a P2 level detection head is added to the P3, P4, and P5 level detection heads of the PAFPN module feature pyramid, realizing a four-scale detection framework.

[0050] The spatial dimensions and channel dimensions of the feature maps output by each level of the detection head are as follows:

[0051] P2: 160×160×128

[0052] P3: 80×80×256

[0053] P4: 40×40×512

[0054] P5: 20×20×1024.

[0055] Furthermore, the loss function of the pedestrian flow supervision model is based on the YOLOv7 model loss function. It improves the CIoU loss function by introducing a dynamic aspect ratio penalty term and a spatiotemporal repulsion loss, and its expression is as follows:

[0056]

[0057] Where Loss represents the loss function of the pedestrian flow supervision model, Loss cls Cross-entropy classification loss, Loss CIoU This represents the improved positioning loss, Loss rep Indicates spacetime repulsion loss; , , Loss cls Loss CIoU Loss rep The weights;

[0058] The improved positioning loss CIoU Represented as:

[0059]

[0060] Wherein, IoU represents the alternation ratio between the predicted bounding box and the ground truth bounding box. This indicates the relationship between the center point b of the predicted bounding box and the center point b of the actual target bounding box. gt The square of the Euclidean distance between them, c 2 β represents the square of the diagonal length of the smallest bounding rectangle covering both the predicted bounding box and the ground truth bounding box; β represents the weighting coefficient of the aspect ratio dynamic penalty term; w and h represent the width and height of the predicted bounding box, respectively. gt h gt These represent the width and height of the actual target bounding box, respectively.

[0061] The rejection loss Loss rep Represented as:

[0062]

[0063] Where Ω represents the total number of predicted boxes with the repulsion loss applied, and ReLU() represents the activation function. Let δ represent the IoU between the i-th and j-th predicted bounding boxes, where i,j=1,2,...,N, N represents the number of predicted bounding boxes participating in the spatiotemporal repulsion loss, and δ represents the maximum allowed overlap threshold. This represents the mask between the i-th and j-th predicted bounding boxes.

[0064] The present invention also provides a crowd monitoring system for densely seated scenarios, including a memory, a processor, and computer program instructions stored in the memory and executable by the processor. When the processor executes the computer program instructions, it can implement the above-mentioned method.

[0065] Compared with existing technologies, this invention has the following advantages: This invention provides a method and system for monitoring pedestrian traffic in densely seated scenarios. This method combines multi-scale feature fusion and dynamic attention optimization strategies to achieve accurate detection and real-time statistics of seated targets in high-density scenes. Based on multi-source surveillance video stream data, this invention features multi-scale perception, deformation adaptation, and occlusion robustness. To address the problem of blurred target boundaries in dense scenes, this invention optimizes the feature pyramid network by embedding a coordinate attention mechanism and reconstructs the detection head using deformable convolution and a lightweight Ghost module. This effectively distinguishes overlapping human targets and suppresses background interference, significantly improving detection accuracy and computational efficiency in dense scenes. Furthermore, by introducing a spatiotemporal repulsion loss function, the spatial distribution and trajectory continuity of the predicted bounding boxes can be dynamically constrained, reducing false detection and false negative rates. This invention can be widely applied to intelligent monitoring systems for canteen waiting halls, subway seats, and large venues, providing real-time data support for crowd density analysis, traffic peak warning, and emergency channel scheduling. Its real-time data supports dynamic seat allocation, improves the capacity utilization of canteen waiting halls, and enables timely detection and response to sudden gathering events in subway waiting halls, which has significant practical value for improving the level of intelligent public transportation management. Attached Figure Description

[0066] Figure 1 This is an architecture diagram of the pedestrian flow monitoring model in an embodiment of the present invention;

[0067] Figure 2 This is an architecture diagram of the YOLOv7 model in existing technology;

[0068] Figure 3 This is a structural diagram of the ODSConv module in an embodiment of the present invention;

[0069] Figure 4 This is a structural diagram of the E-ELAN-Sim module in an embodiment of the present invention;

[0070] Figure 5 This is a structural diagram of the CA-SPPCSPC module in an embodiment of the present invention. Detailed Implementation

[0071] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0072] It should be noted that the following detailed descriptions are exemplary and intended to provide further explanation of this application. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains.

[0073] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the exemplary embodiments according to this application. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof.

[0074] This embodiment provides a method for monitoring pedestrian traffic in densely seated scenarios, including:

[0075] S1: Obtain video stream data of dense seated scenes, construct the original dataset and optimize the anchor box size distribution to obtain the training dataset;

[0076] S2: Construct as follows Figure 1 The pedestrian flow monitoring model shown is based on the YOLOv7 model (e.g., ...). Figure 3 Improvements are implemented as follows: In the backbone network, the first four CBS modules are replaced with ODSConv modules, and the first two of the subsequent four ELAN modules are replaced with E-ELAN-Sim modules to enhance geometric deformation adaptation and dynamic feature selection capabilities; the ODSConv module is implemented by combining the ODSConv module with the SReLU activation function; the E-ELAN-Sim module is implemented by concatenating the E-ELAN module with the SIMAM module; in the head network, a coordinate attention mechanism is embedded in the SPPCSPC module to obtain the CA-SPPCSPC module, and a CA module is added before the CBS module connecting the ELAN-H module and the Concat module to enhance the representation capability of multi-scale features. At the same time, a four-scale detection framework consisting of four detection heads is implemented in the head network to detect ultra-small, small, medium, and large targets to enhance the small target detection capability; the constructed pedestrian flow supervision model is trained using the training dataset to obtain the trained pedestrian flow supervision model.

[0077] S3: Input real-time video stream data into the trained pedestrian flow supervision model and output the number of pedestrian flow supervisions.

[0078] In step S1, video stream data of densely seated scenes such as waiting rooms and restaurants are acquired, image frames are extracted from the video stream data, and the target bounding boxes in the image frames are labeled with labels such as labelme to construct the original dataset.

[0079] For the ground truth bounding boxes labeled in the original dataset, k-means++ clustering algorithm is used for cluster analysis to obtain the optimal anchor box size distribution. The specific implementation method is as follows.

[0080] S101: Obtain the width and height information of all labeled actual target bounding boxes.

[0081] S102: The k-means++ clustering algorithm is used to select the initial center as the cluster center from all labeled real target boxes to strengthen the distribution diversity of the initial centroids and prevent local optima.

[0082] S103: Using IoU distance as the clustering metric, the distance between all labeled ground truth bounding boxes and each cluster center is calculated using the following formula:

[0083]

[0084] Here, `box` represents the labeled ground truth bounding box, and `centroid` represents the cluster center. This represents the distance between the labeled actual bounding box and the cluster center. This represents the IoU distance between the actual target bounding box and the cluster center.

[0085] S104: Select k=12 for clustering to obtain 12 anchor frames.

[0086] S105: Sort the obtained anchor frames according to their area and assign them to four detection heads: ultra-small, small, medium, and large. Each detection head uses three anchor frames to cover human targets of different scales, especially optimizing the fitting ability for small-sized, horizontally seated human figures.

[0087] In step S2, such as Figure 3 As shown, the specific implementation method of the ODSConv module is as follows.

[0088] S2101: Insert the Conv2d module before the ODConv module. The Conv2d module performs omnidirectional feature convolution on the input feature map, and then inputs it into the ODConv module for feature extraction.

[0089] S2102: The output of the ODConv module is batch-normalized through a bn layer and then processed by the SReLU activation function to enhance nonlinear modeling capabilities, especially providing finer control and response to outliers and boundary regions in the input activation distribution, resulting in the output of the ODSConv module; the expression for the SReLU activation function is as follows:

[0090]

[0091] Where SReLU(x) represents the result obtained after processing the input feature map x through the SReLU activation function. , These are the left and right thresholds used to dynamically adjust the activation interval, respectively. , These are the slope coefficients for the negative and positive half-axis, respectively, used to control the nonlinear intensity; , , , All parameters are learnable and dynamically updated during model training. Parameterized SReLU allows for adjustment of the nonlinear response curve, better fitting the distribution characteristics of different channels and tasks. Compared to traditional ReLU, it avoids information clipping or zero gradient problems, maintains feature continuity, and is suitable for tasks such as object detection in dense scenes and pedestrian flow estimation.

[0092] In step S2, such as Figure 4 As shown, the specific implementation method of the E-ELAN-Sim module is as follows.

[0093] S2201: Input feature map It is divided into two paths. The first path obtains the feature map F through the first CBS module. c1 The second path sequentially performs phantom convolutions through five GhostConv modules to obtain five feature maps; the process is as follows:

[0094]

[0095] Where GhostConv1(), GhostConv2(), GhostConv3(), GhostConv4(), and GhostConv5() represent the five GhostConv modules that perform phantom convolution, and F1, F2, F3, F4, and F5 represent the feature maps output by the five GhostConv modules, respectively; X represents the feature map input to the E-ELAN-Sim module, and C, H, and W are the number of channels, height, and width of the feature map, respectively.

[0096] S2202: Convert the feature map F output from the first path... c1 The feature maps F1, F3, and F5 from the second output are concatenated along the channel dimension, as shown below:

[0097]

[0098] Among them, F e This indicates the output of the E-ELAN module, and Concat() represents the concatenation operation.

[0099] S2203: Convert the output F of the E-ELAN module e As input to the SIMAM module, the weighted feature map F is obtained. s Then, after a 1×1 convolution, the output of the E-ELAN-Sim module is obtained:

[0100]

[0101] Among them, F es This represents the feature map output by the E-ELAN-Sim module, Conv 1×1 ( ) represents a 1×1 convolution operation.

[0102] In step S2, such as Figure 5 As shown, the specific implementation method of the CA-SPPCSPC module is as follows.

[0103] S2301: Transmit the feature map X from the backbone network b The process is divided into two paths. The first path obtains the feature map F through the second CBS module. c2 The second path obtains the feature map F through the third CBS module. c3 Then enter the CA module.

[0104] S2302: For the input feature map F c3 The CA module performs adaptive average pooling along both the horizontal and vertical directions, generating feature vectors after horizontal average pooling. and the feature vector after vertical average pooling To achieve spatial-channel union, feature vectors and Global context information for the row and column dimensions of multiple channels is encoded separately.

[0105] To generate spatial attention weights, the feature vectors are... and After concatenation, a spatial attention weight matrix λ is generated through 1×1 convolution and nonlinear activation:

[0106]

[0107] Here, Concat represents the concatenation operation. This represents a 1×1 convolution to compress the number of channels from 2C to C. This represents the Sigmoid function, which makes the weight range [0,1].

[0108] Multiply the spatial attention weight matrix element-wise with the feature map of the input CA module:

[0109]

[0110] in, This represents the feature map after being weighted by the spatial attention weight matrix λ, i.e., the output of the CA module, which is used to enhance the response of spatially sensitive regions.

[0111] S2303: Feature map Perform multi-scale pooling to generate multi-scale feature maps and concatenate them along the channel dimensions to obtain the output features. :

[0112]

[0113] In the formula, Indicates the feature map Perform max pooling operations with a receptive field of k×k, k=5,9,13, to extract contextual information at different scales.

[0114] S2304: Output features The fourth and fifth CBS modules output feature maps F sequentially. c5 .

[0115] S2305: Convert the feature map F output from the first path... c2 Feature map F of the second output c5 Channel-dimensional concatenation is performed, followed by feature integration and dimensionality compression through a sixth CBS module based on 1×1 convolution, ultimately yielding the output of the CA-SPPCSPC module to enhance spatial localization capabilities while preserving detailed information.

[0116]

[0117] Among them, F cs This represents the feature map output by the CA-SPPCSPC module.

[0118] In step S2, the specific implementation method of the four-scale detection framework is as follows.

[0119] To achieve ultra-small target detection, an additional upsampling and downsampling process is added to the feature pyramid of the PAFPN module in the head network. Specifically, after the second upsampling, a first ELAN-H module, a seventh CBS module, a third upsampling module, a first Concat module, a second ELAN-H module, an MP2 module, and a second Concat module are added sequentially between the Concat module and the ELAN-H module at the P3 level. The first Concat module is connected to the first E-ELAN-Sim module of the backbone network through an eighth CBS module, and the first ELAN-H module and the second Concat module are connected through a first CA module and a ninth CBS module. The third upsampling module and the first CBS module are connected through the first CBS module. The oncat module upsamples the 80×80×128 feature map to 160×160×128, and then inputs the resulting 160×160×128 feature map into the second ELAN-H module. The second ELAN-H module outputs ultra-small targets through the RepConv module, thus realizing the first detection head, namely the P2 level detection head. The output of the second ELAN-H module is then used as the input for downsampling and input into the MP2 module. The output of the MP2 module is concatenated with the output of the ninth CBS module by channel dimension, and the result is used as the input of the P3 level ELAN-H module. Thus, a P2 level detection head is added to the P3, P4, and P5 level detection heads of the PAFPN module feature pyramid, realizing a four-scale detection framework.

[0120] The spatial dimensions and channel dimensions of the feature maps output by each level of the detection head are as follows:

[0121] P2: 160×160×128

[0122] P3: 80×80×256

[0123] P4: 40×40×512

[0124] P5: 20×20×1024.

[0125] The above method realizes the update from the original three-scale detection of the YOLOv7 model to the current four-scale detection of the pedestrian flow supervision model. The newly added upsampling path extends the feature pyramid to a shallower layer (P2), which significantly improves the detection capability of small-sized targets (such as the heads of people sitting in the distance). The top-down fusion of multi-scale features and the introduction of the CA module improve the response capability in spatial and channel directions. The overall structure is particularly suitable for scenarios with high requirements for scale adaptability and boundary sensitivity, such as dense targets and pedestrian flow statistics.

[0126] In this embodiment, the loss function of the pedestrian flow supervision model is based on the YOLOv7 model loss function. The CIoU loss function is improved by introducing a dynamic aspect ratio penalty term, and a spatio-repulsion loss is introduced. Based on the spatio-repulsion loss according to the repulsion relationship between predicted boxes, as a supplement to the object detection loss function, the network is guided to actively penalize unnecessary overlap between predicted boxes during training, especially when they correspond to different targets.

[0127] The expression for the loss function of the pedestrian flow monitoring model is as follows:

[0128]

[0129] Where Loss represents the loss function of the pedestrian flow supervision model, Loss cls Cross-entropy classification loss, Loss CIoU This represents the improved positioning loss, Loss rep Indicates spacetime repulsion loss; , , Loss cls Loss CIoU Loss rep The weight.

[0130] The improved positioning loss CIoU Represented as:

[0131]

[0132] Wherein, IoU represents the alternation ratio between the predicted bounding box and the ground truth bounding box. This indicates the relationship between the center point b of the predicted bounding box and the center point b of the actual target bounding box. gt The square of the Euclidean distance between them, c 2 β represents the square of the diagonal length of the smallest bounding rectangle covering both the predicted bounding box and the ground truth bounding box; β represents the weighting coefficient of the aspect ratio dynamic penalty term; w and h represent the width and height of the predicted bounding box, respectively. gt h gt These represent the width and height of the actual target bounding box, respectively.

[0133] The rejection loss Loss rep Represented as:

[0134]

[0135] Where Ω represents the total number of predicted boxes with the repulsion loss applied, and ReLU() represents the activation function. Let δ represent the IoU between the i-th and j-th predicted bounding boxes, where i,j=1,2,...,N, N represents the number of predicted bounding boxes participating in the spatiotemporal repulsion loss, and δ represents the maximum allowed overlap threshold. This represents the mask between the i-th and j-th predicted bounding boxes. This indicates that the two predicted boxes are likely two independent and different targets (e.g., two people who are far apart). We want their corresponding predicted boxes to also be far apart, so the mask is set to 1 and a repulsion loss needs to be applied. , which means that no repulsion loss is applied, thus allowing the predicted boxes of overlapping targets to be close together.

[0136] The introduction of the spatiotemporal repulsion loss term, while maintaining the accuracy of conventional localization loss, has the following advantages: enhanced discriminability between detection boxes: avoids multiple boxes being predicted in the same target or nearby area; suitable for dense detection tasks; can be seamlessly integrated into the training process of any target detection network; low computational cost, without introducing additional network structure or learnable parameters.

[0137] This embodiment also provides a crowd monitoring system for densely seated scenarios, including a memory, a processor, and computer program instructions stored in the memory and executable by the processor. When the processor executes the computer program instructions, it can implement the above-described method.

[0138] In this embodiment, the method proposed in this invention is experimentally verified.

[0139] System deployment and hardware adaptation were carried out. The first step was the selection of edge computing devices. The relevant hardware configuration was: deploying NVIDIA Jetson AGX Xavier edge devices, equipped with a 512-core Volta GPU and 32GB LPDDR4x memory, supporting parallel processing of 16 video streams. Its real-time performance is demonstrated by achieving a processing speed of 25 FPS at 4K resolution, a single-frame latency of ≤40ms, and a power consumption of ≤20W, meeting the all-weather operational requirements of densely populated seating scenarios such as cafeterias and waiting areas.

[0140] Performance was validated in dense seated scene scenarios using a self-built dataset. Regarding detection accuracy, on the cafeteria waiting hall dataset, mAP@0.5 improved to 94.5%, map@0.95 improved to 43.1%, precision reached 93.8%, and recall was 93.1%, meeting the requirement for high-density scene detection accuracy (mAP@0.5≥90%). In edge computing deployment, the Jetson AGX Xavier model had FLOPs≤4.2G, supporting real-time processing of 16 1080P video streams (latency≤40ms); its power consumption was controlled to ≤20W at full load, adapting to 24 / 7 operation requirements.

[0141] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0142] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0143] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0144] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0145] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention in any other way. Any person skilled in the art may make changes or modifications to the above-disclosed technical content to create equivalent embodiments. However, any simple modifications, equivalent changes, and modifications made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the protection scope of the present invention.

Claims

1. A method for monitoring pedestrian flow in densely seated scenarios, characterized in that, include: S1: Obtain video stream data of dense seated scenes, construct the original dataset and optimize the anchor box size distribution to obtain the training dataset; S2: Construct a pedestrian flow supervision model, which is an improved version of the YOLOv7 model. In the backbone network, the first four CBS modules are replaced with ODSConv modules, and the first two of the subsequent four ELAN modules are replaced with E-ELAN-Sim modules to enhance geometric deformation adaptation and dynamic feature selection capabilities. The ODSConv module is implemented by combining the ODSConv module with the SReLU activation function. The E-ELAN-Sim module is implemented by concatenating the E-ELAN module with the SIMAM module. In the head network, a coordinate attention mechanism is embedded in the SPCSPC module to obtain the CA-SPPCSPC module, and a CA module is added before the CBS module connecting the ELAN-H module and the Concat module to enhance the representation capability of multi-scale features. At the same time, a four-scale detection framework consisting of four detection heads is implemented in the head network to detect ultra-small, small, medium, and large targets to enhance the small target detection capability. The constructed pedestrian flow supervision model is trained using a training dataset to obtain a trained pedestrian flow supervision model. S3: Input real-time video stream data into the trained pedestrian flow supervision model and output the number of pedestrian flow supervisions.

2. The method for monitoring pedestrian flow in densely seated scenarios according to claim 1, characterized in that, In step S1, video stream data of densely seated scenes including waiting halls and restaurants is acquired, image frames are extracted from the video stream data, and the targets in the image frames are labeled with real target boxes to construct the original dataset. For the ground truth target boxes labeled in the original dataset, k-means++ clustering algorithm is used for cluster analysis to obtain the optimal anchor box size distribution; specifically, the following steps are included: S101: Obtain the width and height information of all labeled actual target bounding boxes; S102: The k-means++ clustering algorithm is used to select the initial centers as cluster centers from all labeled ground truth bounding boxes; S103: Calculate the distance between all labeled ground truth bounding boxes and each cluster center using the following formula: Here, `box` represents the labeled ground truth bounding box, and `centroid` represents the cluster center. This represents the distance between the labeled actual bounding box and the cluster center. This represents the IoU distance between the labeled true bounding boxes and the cluster centers; S104: Select k=12 for clustering to obtain 12 anchor frames; S105: Sort the obtained anchor frames according to their area and assign them to four detection heads: ultra-small, small, medium, and large. Each detection head uses three anchor frames to cover human targets of different scales.

3. The method for monitoring pedestrian flow in densely seated scenarios according to claim 1, characterized in that, In step S2, the implementation method of the ODSConv module is as follows: S2101: Insert a Conv2d module before the ODConv module. Perform two-dimensional convolution on the input feature map through the Conv2d module, and then input it into the ODConv module for feature extraction. S2102: The output of the ODConv module is batch-normalized through a bn layer and then processed by the SReLU activation function to obtain the output of the ODSConv module; the expression of the SReLU activation function is as follows: Where SReLU(x) represents the result obtained after processing the input feature map x through the SReLU activation function. , These are the left and right thresholds used to dynamically adjust the activation interval, respectively. , These are the slope coefficients for the negative and positive half-axis, respectively, used to control the nonlinear intensity; , , , All of these are learnable parameters that are dynamically updated during model training.

4. The method for monitoring pedestrian flow in densely seated scenarios according to claim 1, characterized in that, In step S2, the implementation method of the E-ELAN-Sim module is as follows: S2201: Input feature map It is divided into two paths. The first path obtains the feature map F through the first CBS module. c1 The second path sequentially performs phantom convolution through 5 GhostConv modules to obtain 5 feature maps; The process is as follows: Where GhostConv1(), GhostConv2(), GhostConv3(), GhostConv4(), and GhostConv5() represent the five GhostConv modules that perform phantom convolution, and F1, F2, F3, F4, and F5 represent the feature maps output by the five GhostConv modules, respectively; X represents the feature map input to the E-ELAN-Sim module, and C, H, and W are the number of channels, height, and width of the feature map, respectively. S2202: Convert the feature map F output from the first path... c1 The feature maps F1, F3, and F5 from the second output are concatenated along the channel dimension, as shown below: Among them, F e This indicates the output of the E-ELAN module, and Concat() represents the concatenation operation; S2203: Convert the output F of the E-ELAN module e As input to the SIMAM module, the weighted feature map F is obtained. s Then, after a 1×1 convolution, the output of the E-ELAN-Sim module is obtained: Among them, F es This represents the feature map output by the E-ELAN-Sim module, Conv 1×1 ( ) represents a 1×1 convolution operation.

5. A method for monitoring pedestrian flow in densely seated scenarios according to claim 1, characterized in that, In step S2, the implementation method of the CA-SPPCSPC module is as follows: S2301: Transmit the feature map X from the backbone network b The process is divided into two paths. The first path obtains the feature map F through the second CBS module. c2 The second path obtains the feature map F through the third CBS module. c3 Then enter the CA module; S2302: For the input feature map F c3 The CA module performs adaptive average pooling along both the horizontal and vertical directions, generating feature vectors after horizontal average pooling. and the feature vector after vertical average pooling To achieve spatial-channel union, feature vectors and Global context information for the row and column dimensions of multiple channels is encoded separately; To generate spatial attention weights, the feature vectors are... and After concatenation, a spatial attention weight matrix λ is generated through 1×1 convolution and nonlinear activation: Here, Concat represents the concatenation operation. This represents a 1×1 convolution to compress the number of channels from 2C to C. This represents the Sigmoid function, ensuring that the weight range is within the range [0,1]. Multiply the spatial attention weight matrix element-wise with the feature map of the input CA module: in, This represents the feature map after being weighted by the spatial attention weight matrix λ, i.e., the output of the CA module, which is used to enhance the response of spatially sensitive regions. S2303: Feature map Perform multi-scale pooling to generate multi-scale feature maps and concatenate them along the channel dimensions to obtain the output features. : In the formula, Indicates the feature map Perform max pooling operations with a receptive field of k×k, k=5,9,13, to extract contextual information at different scales; S2304: Output features The fourth and fifth CBS modules output feature maps F sequentially. c5 ; S2305: Convert the feature map F output from the first path... c2 Feature map F of the second output c5 Channel-dimensional concatenation is performed, followed by feature integration and dimensionality compression through a sixth CBS module based on 1×1 convolution, ultimately yielding the output of the CA-SPPCSPC module to enhance spatial localization capabilities while preserving detailed information. Among them, F cs This represents the feature map output by the CA-SPPCSPC module.

6. The method for monitoring pedestrian flow in densely seated scenarios according to claim 1, characterized in that, In step S2, the implementation method of the four-scale detection framework is as follows: To achieve ultra-small target detection, an additional upsampling and downsampling process is added to the feature pyramid of the PAFPN module in the head network. Specifically, after the second upsampling, a first ELAN-H module, a seventh CBS module, a third upsampling module, a first Concat module, a second ELAN-H module, an MP2 module, and a second Concat module are added sequentially between the Concat module after the second upsampling and the ELAN-H module at the P3 level. The first Concat module is connected to the first E-ELAN-Sim module of the backbone network through an eighth CBS module, and the first ELAN-H module and the second Concat module are connected through a first CA module and a ninth CBS module. The third upsampling module and the first CBS module are connected through the first CBS module. The oncat module upsamples the 80×80×128 feature map to 160×160×128, and then inputs the resulting 160×160×128 feature map into the second ELAN-H module. The second ELAN-H module outputs ultra-small targets through the RepConv module, thus realizing the first detection head, namely the P2 level detection head. The output of the second ELAN-H module is then used as the input for downsampling and input into the MP2 module. The output of the MP2 module is concatenated with the output of the ninth CBS module by channel dimension, and the result is used as the input of the P3 level ELAN-H module. Thus, a P2 level detection head is added to the P3, P4, and P5 level detection heads of the PAFPN module feature pyramid, realizing a four-scale detection framework. The spatial dimensions and channel dimensions of the feature maps output by each level of the detection head are as follows: P2：160×160×128 P3：80×80×256 P4：40×40×512 P5：20×20×1024。 7. A method for monitoring pedestrian flow in densely seated scenarios according to claim 1, characterized in that, The loss function of the pedestrian flow supervision model is based on the YOLOv7 model loss function. It improves the CIoU loss function by introducing a dynamic aspect ratio penalty term and a spatiotemporal repulsion loss. Its expression is as follows: Where Loss represents the loss function of the pedestrian flow supervision model, Loss cls Cross-entropy classification loss, Loss CIoU This represents the improved positioning loss, Loss rep Indicates spacetime repulsion loss; , , Loss cls Loss CIoU Loss rep The weights; The improved positioning loss CIoU Represented as: Wherein, IoU represents the alternation ratio between the predicted bounding box and the ground truth bounding box. This indicates the relationship between the center point b of the predicted bounding box and the center point b of the actual target bounding box. gt The square of the Euclidean distance between them, c 2 β represents the square of the diagonal length of the smallest bounding rectangle covering both the predicted bounding box and the ground truth bounding box; β represents the weighting coefficient of the aspect ratio dynamic penalty term; w and h represent the width and height of the predicted bounding box, respectively. gt h gt These represent the width and height of the actual target bounding box, respectively. The rejection loss Loss rep Represented as: Where Ω represents the total number of predicted boxes with the repulsion loss applied, and ReLU() represents the activation function. Let δ represent the IoU between the i-th and j-th predicted bounding boxes, where i,j=1,2,...,N, N represents the number of predicted bounding boxes participating in the spatiotemporal repulsion loss, and δ represents the maximum allowed overlap threshold. This represents the mask between the i-th and j-th predicted bounding boxes.

8. A crowd monitoring system for densely seated scenarios, characterized in that, It includes a memory, a processor, and computer program instructions stored in the memory and executable by the processor, which, when executed by the processor, enable the implementation of the method as described in any one of claims 1-7.

Images