A method and system for crowd supervision in dense sitting scenarios

Abstract

The present application relates to a kind of crowd supervision method and system for dense sitting posture scene, the method comprises the following steps: S1: obtaining the video stream data of dense sitting posture scene, constructs original data set and optimizes anchor frame size distribution;S2: construct crowd supervision model, it is improved and realized on YOLOv7 model: in backbone network, the first 4 CBS modules are replaced with ODSConv module, and subsequent 2 ELAN modules are replaced with E-ELAN-Sim module;In head network, in SPPCSPC module, coordinate attention mechanism is embedded to obtain CA-SPPCSPC module, and CA module is added before the CBS module connecting ELAN-H module and Concat module, and four scale detection frame is realized in head network, to enhance small target detection capability;The training data set is used to train the crowd supervision model constructed;S3: real-time video stream data is input into the crowd supervision model trained, and the output is crowd supervision quantity.The method and system can realize the accurate detection and real-time statistics of sitting posture target person under dense sitting posture scene.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of target detection, and particularly relates to a people flow supervision method and system for a dense sitting posture scene. BACKGROUND

[0002] Target detection technology based on deep learning (such as YOLOv7) has made significant progress in general scenarios, but its practical application in extreme dense flow environments such as transportation hubs and large venues still faces severe challenges. Traditional methods have insufficient spatial position sensitivity in multi-scale feature fusion. For example, when the standard SPP module extracts features by using a fixed pooling kernel, it ignores the spatial correlation in the interlaced area of the human body limbs, resulting in a feature confusion rate of 42.3% for small targets below 30x40 pixels (COCO-Person dataset test results). The feature extraction backbone network does not introduce a channel-space joint attention mechanism, and about 35% of the computing resources are consumed in background noise processing, which seriously restricts real-time performance. In addition, the fixed geometric structure of the convolution kernel is difficult to adapt to complex human sitting posture changes such as sitting posture and backpack shielding, and the miss detection rate in the posture deformation scene increases by 18.6% compared with standing people. Although existing research attempts to optimize locally by increasing the scale of the detection head or introducing an attention module, a single improvement scheme often leads to a sharp increase in computational complexity or a decline in generalization ability. For example, a four-scale detection head improves the small target recall rate, but increases the inference delay. The existing technology lacks systematic optimization of feature selection, deformation modeling, and multi-target repulsion constraints, and is difficult to meet the dual demands of high detection accuracy and real-time processing in the high-density sitting posture scene, which seriously restricts the landing application of people flow supervision in dense sitting posture scenes. SUMMARY

[0003] The purpose of the present application is to provide a people flow supervision method and system for a dense sitting posture scene, which can realize accurate detection and real-time statistics of sitting posture targets in a dense sitting posture scene.

[0004] To achieve the above purpose, the technical scheme adopted by the present application is: a people flow supervision method for a dense sitting posture scene, comprising:

[0005] S1: acquiring video stream data of a dense sitting posture scene, constructing an original data set and optimizing anchor box size distribution to obtain a training data set;

[0006] S2: Construct a pedestrian flow supervision model, which is an improved version of the YOLOv7 model. In the backbone network, the first four CBS modules are replaced with ODSConv modules, and the first two of the subsequent four ELAN modules are replaced with E-ELAN-Sim modules to enhance geometric deformation adaptation and dynamic feature selection capabilities. The ODSConv module is implemented by combining the ODSConv module with the SReLU activation function. The E-ELAN-Sim module is implemented by concatenating the E-ELAN module with the SIMAM module. In the head network, a coordinate attention mechanism is embedded in the SPCSPC module to obtain the CA-SPPCSPC module, and a CA module is added before the CBS module connecting the ELAN-H module and the Concat module to enhance the representation capability of multi-scale features. At the same time, a four-scale detection framework consisting of four detection heads is implemented in the head network to detect ultra-small, small, medium, and large targets to enhance the small target detection capability. The constructed pedestrian flow supervision model is trained using a training dataset to obtain a trained pedestrian flow supervision model.

[0007] S3: Input real-time video stream data into the trained pedestrian flow supervision model and output the number of pedestrian flow supervisions.

[0008] Further, in step S1, video stream data of densely seated scenes including waiting halls and restaurants is acquired, image frames are extracted from the video stream data, and the targets in the image frames are labeled with real target boxes to construct the original dataset.

[0009] For the ground truth target boxes labeled in the original dataset, k-means++ clustering algorithm is used for cluster analysis to obtain the optimal anchor box size distribution; specifically, the following steps are included:

[0010] S101: Obtain the width and height information of all labeled actual target bounding boxes;

[0011] S102: The k-means++ clustering algorithm is used to select the initial centers as cluster centers from all labeled ground truth bounding boxes;

[0012] S103: Calculate the distance between all labeled ground truth bounding boxes and each cluster center using the following formula:

[0013]

[0014] Here, `box` represents the labeled ground truth bounding box, and `centroid` represents the cluster center. This represents the distance between the labeled actual bounding box and the cluster center. This represents the IoU distance between the labeled true bounding boxes and the cluster centers;

[0015] S104: Select k=12 for clustering, and get 12 anchor boxes;

[0016] S105: Sort the obtained anchor boxes according to their areas, and respectively assign them to four detection heads of super-small, small, medium and large, and each detection head uses three anchor boxes to cover human targets of different scales.

[0017] Further, in step S2, the implementation method of the ODSConv module is:

[0018] S2101: Insert a Conv2d module before the ODConv module, perform two-dimensional convolution on the input feature map through the Conv2d module, and then input the ODConv module for feature extraction;

[0019] S2102: The result output by the ODConv module is input into the SReLU activation function after batch normalization processing by the bn layer, and the output of the ODSConv module is obtained; the expression of the SReLU activation function is as follows:

[0020]

[0021] Wherein, SReLU(x) represents the result obtained by processing the input feature map x through the SReLU activation function, 、 respectively, are the left and right threshold values for dynamically adjusting the activation interval, 、 respectively, are the slope coefficients of the negative half-axis and the positive half-axis, used to control the nonlinearity strength; 、 、 、 All are learnable parameters, which are dynamically updated during model training.

[0022] Further, in step S2, the implementation method of the E-ELAN-Sim module is:

[0023] S2201: Divide the input feature map F into two paths, the first path obtains a feature map F c1 through a first CBS module, and the second path sequentially passes through five GhostConv modules for ghost convolution to obtain five feature maps; the process is as follows:

[0024]

[0025] Wherein, GhostConv1(), GhostConv2(), GhostConv3(), GhostConv4(), GhostConv5() respectively represent 5 GhostConv modules performing ghost convolution, F1, F2, F3, F4, F5 respectively represent feature maps output by the 5 GhostConv modules; X represents a feature map input to the E-ELAN-Sim module, C, H, W respectively represent the channel number, height and width of the feature map;

[0026] S2202: The feature map F c1 output by the first path is concatenated with the feature maps F1, F3 and F5 output by the second path in the channel dimension, as shown below:

[0027]

[0028] Wherein, F e represents the output of the E-ELAN module, and Concat() represents a concatenation operation.

[0029] S2203: The output F e of the E-ELAN module is input to the SIMAM module to obtain a weighted feature map F s ; then, after a 1x1 convolution, the output of the E-ELAN-Sim module is obtained:

[0030]

[0031] Wherein, F es represents a feature map output by the E-ELAN-Sim module, and Conv 1×1 () represents a 1x1 convolution operation.

[0032] Further, in step S2, the implementation method of the CA-SPPCSPC module is as follows:

[0033] S2301: The feature map X b transmitted by the backbone network is divided into two paths, the first path passes through the second CBS module to obtain a feature map F c2 , and the second path passes through the third CBS module to obtain a feature map F c3 , and then inputs the CA module;

[0034] S2302: For the input feature map F c3 , the CA module performs adaptive average pooling along the horizontal direction and the vertical direction respectively to generate a horizontal direction average pooled feature vector and a vertical direction average pooled feature vector , realizes spatial-channel joint, and the feature vectors and global context information of row and column dimensions of multiple channels are encoded respectively;

[0035] To generate the spatial attention weight, the feature vectors and are concatenated, and then a 1x1 convolution and a nonlinear activation are used to generate a spatial attention weight matrix λ:

[0036]

[0037] where Concat denotes the concatenation operation, denotes a 1x1 convolution to compress the channel number from 2C to C, denotes a Sigmoid function to make the weight value range in [0, 1];

[0038] The spatial attention weight matrix is multiplied element-wise with the feature map input to the CA module:

[0039]

[0040] where denotes the feature map weighted by the spatial attention weight matrix λ, which is the output of the CA module, used to strengthen the response of the spatial sensitive region;

[0041] S2303: Perform multi-scale pooling on the feature map to generate multi-scale feature maps and concatenate along the channel dimension to obtain the output feature :

[0042]

[0043] where denotes performing a max-pooling operation with a receptive field of kxk on the feature map , k = 5, 9, 13, used to extract context information of different scales;

[0044] S2304: Concatenate the output feature in turn through the fourth CBS module, the fifth CBS module, and output the feature map F c5 ;

[0045] S2305: Concatenate the feature map F c2 output by the first path with the feature map F c5 output by the second path along the channel dimension, and then pass through the sixth CBS module based on 1x1 convolution to realize feature integration and dimension compression, and finally obtain the output of the CA-SPPCSPC module to strengthen the spatial positioning ability while preserving the detail information:

[0046]

[0047] wherein F cs represents the feature map output by the CA-SPPCSPC module.

[0048] Further, in step S2, the implementation method of the four-scale detection framework is:

[0049] On the basis of the PAFPN module feature pyramid of the head network, an upsampling and downsampling process is added again to realize the detection of super-small targets, specifically: a first ELAN-H module, a seventh CBS module, a third upsampling module, a first Concat module, a second ELAN-H module, an MP2 module and a second Concat module are sequentially connected between the Concat module after the second upsampling and the ELAN-H module at the P3 level, and the first Concat module and the first E-ELAN-Sim module of the backbone network are connected through an eighth CBS module, and the first ELAN-H module and the second Concat module are connected through a first CA module and a ninth CBS module; the feature map of 80x80x128 is upsampled to 160x160x128 through the third upsampling module and the first Concat module, and then the obtained feature map of 160x160x128 is input into the second ELAN-H module, the second ELAN-H module outputs super-small targets through the RepConv module, thereby realizing the first detection head, i.e., the detection head at the P2 level; the output of the second ELAN-H module is input into the MP2 module as the input of downsampling; the output of the MP2 module and the output of the ninth CBS module are spliced in the channel dimension, and then the obtained result is input into the ELAN-H module at the P3 level; thereby, the P2 level detection head is added on the basis of the P3, P4 and P5 level detection heads of the PAFPN module feature pyramid, and the four-scale detection framework is realized.

[0050] The spatial size and channel dimension of the feature map output by each level detection head are respectively:

[0051] P2: 160x160x128

[0052] P3: 80x80x256

[0053] P4: 40x40x512

[0054] P5: 20x20x1024.

[0055] Further, the loss function of the crowd supervision model is improved on the basis of the YOLOv7 model loss function by introducing a width-height ratio dynamic penalty term to improve the CIoU loss function and introducing a space-time repulsion loss, and its expression is:

[0056]

[0057] wherein, Loss represents a loss function of the crowd supervision model, Loss cls represents a cross-entropy classification loss, Loss CIoU represents an improved positioning loss, Loss rep represents a spatiotemporal repulsion loss; 、 、 are weights of Loss cls , Loss CIoU , and Loss rep , respectively.

[0058] The improved positioning loss Loss CIoU is represented as:

[0059]

[0060] wherein, IoU represents an intersection over union of a predicted bounding box and a real target bounding box, represents a Euclidean distance square between a center point b of the predicted bounding box and a center point b gt of the real target bounding box, c 2 represents a square of a diagonal length of a minimum circumscribed rectangle covering the predicted bounding box and the real target bounding box, β represents a weight coefficient of a width-height ratio dynamic penalty term, w and h represent a width and a height of the predicted bounding box, respectively, gt gt w gt , h gt represent a width and a height of the real target bounding box, respectively.

[0061] The repulsion loss Loss rep is represented as:

[0062]

[0063] wherein, Ω represents a total number of predicted bounding boxes to which the repulsion loss is applied, ReLU( ) represents an activation function, represents an IoU between an i-th predicted bounding box and a j-th predicted bounding box, i, j = 1, 2,..., N, N represents a number of predicted bounding boxes participating in the spatiotemporal repulsion loss, and δ represents a maximum allowed overlap threshold, represents a mask between the i-th predicted bounding box and the j-th predicted bounding box.

[0064] The application further provides a crowd supervision system for a dense sitting position scene, comprising a memory, a processor, and computer program instructions stored on the memory and capable of being executed by the processor, when the processor executes the computer program instructions, the above-mentioned method can be implemented.

[0065] Compared with the prior art, the present application has the following beneficial effects: the present application provides a people flow supervision method and system for dense sitting posture scenes, which combines multi-scale feature fusion and dynamic attention optimization strategy to realize accurate detection and real-time statistics of sitting posture target people in high-density scenes. The present application is based on multi-source monitoring video stream data and has the characteristics of multi-scale perception, deformation adaptation and occlusion robustness. To solve the problem of fuzzy target boundary in dense scenes, the present application optimizes the feature pyramid network by embedding the coordinate attention mechanism, combines the deformable convolution and lightweight Ghost module to reconstruct the detection head, which can effectively distinguish overlapping human targets and suppress background interference, significantly improving the detection accuracy and computational efficiency in dense scenes. Further, by introducing the spatiotemporal repulsion loss function, the spatial distribution and motion trajectory continuity of the prediction box can be dynamically constrained, reducing the false detection and missed detection rate. The present application can be widely applied to intelligent monitoring systems of canteen waiting halls, subway seats and large venues, providing real-time data support for crowd density analysis, flow peak warning and emergency passage scheduling, real-time data support for seat dynamic allocation, improving the utilization rate of canteen waiting hall capacity, and timely detecting and responding to subway waiting hall emergency gathering events, which has important practical value for improving the intelligent level of public transportation management. BRIEF DESCRIPTION OF DRAWINGS

[0066] Figure 1 is the architecture diagram of the people flow supervision model in the embodiment of the present application;

[0067] Figure 2 is the architecture diagram of the YOLOv7 model in the prior art;

[0068] Figure 3 is the structure diagram of the ODSConv module in the embodiment of the present application;

[0069] Figure 4 is the structure diagram of the E-ELAN-Sim module in the embodiment of the present application;

[0070] Figure 5 is the structure diagram of the CA-SPPCSPC module in the embodiment of the present application. DETAILED DESCRIPTION

[0071] The present application will be further described below in conjunction with the drawings and embodiments.

[0072] It should be pointed out that the following detailed description is exemplary and is intended to provide further description of the present application. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as generally understood by those skilled in the art to which the present application belongs.

[0073] It is to be noted that the terms used herein are only intended to describe specific embodiments and are not intended to limit the example embodiments according to the present application. As used herein, the singular form is intended to include the plural form as well, unless the context clearly indicates otherwise, and it should also be understood that the terms "comprise" and / or "include" when used in this specification, specify the presence of stated features, steps, operations, devices, components and / or combinations thereof.

[0074] The embodiment provides a people flow supervision method for a dense sitting posture scene, comprising:

[0075] S1: acquiring video stream data of a dense sitting posture scene, constructing an original data set and optimizing anchor box size distribution to obtain a training data set;

[0076] S2: constructing a people flow supervision model as shown in Figure 1 The people flow supervision model is improved and implemented on a YOLOv7 model as shown in Figure 3 The first four CBS modules in the backbone network are replaced with ODSConv modules, and the first two of the subsequent four ELAN modules are replaced with E-ELAN-Sim modules to enhance the geometric deformation adaptation and dynamic feature screening capability; the ODSConv module is realized by combining the ODConv module with the SReLU activation function; the E-ELAN-Sim module is realized by splicing the E-ELAN module and the SIMAM module; in the head network, the coordinate attention mechanism is embedded in the SPPCSPC module to obtain the CA-SPPCSPC module, and the CA module is added before the CBS module connecting the ELAN-H module and the Concat module to enhance the representation ability of multi-scale features, and a four-scale detection framework composed of four detection heads is realized in the head network to detect super-small, small, medium and large targets to enhance the small target detection capability; the training data set is used to train the constructed people flow supervision model to obtain the trained people flow supervision model;

[0077] S3: inputting real-time video stream data into the trained people flow supervision model to output the people flow supervision quantity.

[0078] In step S1, video stream data of a dense sitting posture scene such as a waiting hall and a restaurant is acquired, image frames are extracted from the video stream data, and a labelme or the like is used to label the real target frame in the image frame, and then an original data set is constructed.

[0079] For the real target frame labeled in the original data set, k-means++ clustering algorithm is used for clustering analysis to obtain the optimal anchor box size distribution. The specific implementation method is as follows.

[0080] S101: Obtain the width and height information of all labeled real target boxes.

[0081] S102: Use the k-means++ clustering algorithm to select the initial center as the clustering center from all labeled real target boxes, to prevent local optimization by increasing the diversity of the strong initial centroid distribution.

[0082] S103: Use IoU distance measurement as the clustering basis, and calculate the distance between all labeled real target boxes and each clustering center by the following formula:

[0083]

[0084] wherein box represents the labeled real target box, centroid represents the clustering center, represents the distance between the labeled real target box and the clustering center, represents the IoU distance between the labeled real target box and the clustering center.

[0085] S104: Select k=12 for clustering, and obtain 12 anchor boxes.

[0086] S105: Sort the obtained anchor boxes according to their areas, and assign them to four detection heads of super-small, small, medium and large, respectively, and use three anchor boxes for each detection head to cover human targets of different sizes, especially to optimize the fitting ability for small size and horizontal posture human figures.

[0087] In step S2, as shown in Figure 3 , the specific implementation method of the ODSConv module is as follows.

[0088] S2101: Insert a Conv2d module before the ODConv module, perform omnidirectional feature convolution on the input feature map through the Conv2d module, and then input the ODConv module for feature extraction.

[0089] S2102: After the result output by the ODConv module is processed by the bn layer for batch normalization, input it into the SReLU activation function for processing to enhance the non-linear modeling capability, especially to provide more fine control and response to abnormal values and boundary areas in the input activation distribution, and obtain the output of the ODSConv module; the expression of the SReLU activation function is as follows:

[0090]

[0091] wherein SReLU(x) represents the result obtained by processing the input feature map x through the SReLU activation function, , are the left and right threshold values for dynamically adjusting the activation interval, respectively, , These are the slope coefficients for the negative and positive half-axis, respectively, used to control the nonlinear intensity; , , , All parameters are learnable and dynamically updated during model training. Parameterized SReLU allows for adjustment of the nonlinear response curve, better fitting the distribution characteristics of different channels and tasks. Compared to traditional ReLU, it avoids information clipping or zero gradient problems, maintains feature continuity, and is suitable for tasks such as object detection in dense scenes and pedestrian flow estimation.

[0092] In step S2, such as Figure 4 As shown, the specific implementation method of the E-ELAN-Sim module is as follows.

[0093] S2201: Input feature map It is divided into two paths. The first path obtains the feature map F through the first CBS module. c1 The second path sequentially performs phantom convolutions through five GhostConv modules to obtain five feature maps; the process is as follows:

[0094]

[0095] Where GhostConv1(), GhostConv2(), GhostConv3(), GhostConv4(), and GhostConv5() represent the five GhostConv modules that perform phantom convolution, and F1, F2, F3, F4, and F5 represent the feature maps output by the five GhostConv modules, respectively; X represents the feature map input to the E-ELAN-Sim module, and C, H, and W are the number of channels, height, and width of the feature map, respectively.

[0096] S2202: Convert the feature map F output from the first path... c1 The feature maps F1, F3, and F5 from the second output are concatenated along the channel dimension, as shown below:

[0097]

[0098] Among them, F e This indicates the output of the E-ELAN module, and Concat() represents the concatenation operation.

[0099] S2203: Convert the output F of the E-ELAN module e As input to the SIMAM module, the weighted feature map F is obtained. s Then, after a 1×1 convolution, the output of the E-ELAN-Sim module is obtained:

[0100]

[0101] where F es denotes the feature map output by the E-ELAN-Sim module, Conv 1×1 denotes a 1x1 convolution operation.

[0102] In step S2, as shown in Figure 5 , the specific implementation method of the CA-SPPCSPC module is as follows.

[0103] S2301: The feature map X b transmitted by the backbone network is divided into two paths, the first path passes through the second CBS module to obtain the feature map F c2 , and the second path passes through the third CBS module to obtain the feature map F c3 , and then inputs the CA module.

[0104] S2302: For the input feature map F c3 , the CA module respectively performs adaptive average pooling along the horizontal direction and the vertical direction to generate the horizontal direction average pooled feature vector and the vertical direction average pooled feature vector , realizes spatial-channel joint, and the feature vectors and respectively encode the global context information of the row and column dimensions of multiple channels.

[0105] To generate the spatial attention weight, the feature vectors and are spliced, and then a 1x1 convolution and a nonlinear activation are performed to generate a spatial attention weight matrix λ:

[0106]

[0107] where Concat denotes a splicing operation, denotes a 1x1 convolution to compress the number of channels from 2C to C, denotes a Sigmoid function to make the weight value range in the range [0, 1].

[0108] The spatial attention weight matrix is multiplied element by element with the feature map input into the CA module:

[0109]

[0110] where denotes the feature map weighted by the spatial attention weight matrix λ, that is, the output of the CA module, which is used to strengthen the response of the spatial sensitive region.

[0111] S2303: The feature map Multi-scale pooling is performed to generate multi-scale feature maps and concatenate along the channel dimension to obtain output features

[0112]

[0113] In the formula, represents the feature map The receptive field of the maximum pooling operation is k x k, k = 5, 9, 13, which is used to extract context information of different scales.

[0114] S2304: Concatenate the output features F in turn, the fourth CBS module, the fifth CBS module, and output the feature map F c5 .

[0115] S2305: Concatenate the feature map F c2 output by the first path with the feature map F c5 output by the second path along the channel dimension, and then pass through the sixth CBS module based on 1 x 1 convolution to realize feature integration and dimension compression, and finally obtain the output of the CA-SPPCSPC module to strengthen the spatial positioning ability while preserving the detailed information:

[0116]

[0117] wherein F cs represents the feature map output by the CA-SPPCSPC module.

[0118] In step S2, the specific implementation method of the four-scale detection framework is as follows.

[0119] ​On the basis of the PAFPN module feature pyramid of the head network, a second upsampling and downsampling process is added to realize super-small target detection. Specifically, a first ELAN-H module, a seventh CBS module, a third upsampling module, a first Concat module, a second ELAN-H module, an MP2 module and a second Concat module are sequentially connected between the second upsampling Concat module and the P3 level ELAN-H module, and the first Concat module is connected with the first E-ELAN-Sim module of the backbone network through an eighth CBS module, and the first ELAN-H module is connected with the second Concat module through a first CA module and a ninth CBS module; the 80x80x128 feature map is upsampled to 160x160x128 through the third upsampling module and the first Concat module, and then the obtained 160x160x128 feature map is input into the second ELAN-H module, and the second ELAN-H module outputs a super-small target through a RepConv module, thereby realizing the first detection head, i.e., the P2 level detection head; the output of the second ELAN-H module is input into the MP2 module as the input of downsampling; the output of the MP2 module is spliced with the output of the ninth CBS module in the channel dimension, and then the obtained result is input into the P3 level ELAN-H module; thereby, the P2 level detection head is added on the basis of the P3, P4 and P5 level detection heads of the PAFPN module feature pyramid, and a four-scale detection framework is realized.

[0120] The spatial size and channel dimension of the feature map output by each level detection head are as follows:

[0121] P2: 160x160x128

[0122] P3: 80x80x256

[0123] P4: 40x40x512

[0124] P5: 20x20x1024.

[0125] The above method realizes the update from the three-scale detection of the original YOLOv7 model to the four-scale detection of the present crowd supervision model, the newly added upsampling path expands the feature pyramid to a shallower layer (P2), and significantly improves the detection capability of small size targets (such as distant sitting crowd heads, etc.); the top-down fusion of multi-scale features and the introduction of the CA module improve the response capability in the spatial and channel directions; the overall structure is especially suitable for dense targets, crowd counting and other scenes with high requirements for scale adaptability and boundary sensitivity.

[0126] In the embodiment, the loss function of the crowd supervision model is improved on the basis of the loss function of the YOLOv7 model by introducing a width-height ratio dynamic penalty term to improve the CIoU loss function and introducing a spatio-repulsion loss. According to the spatio-repulsion loss based on the repulsion relationship between the prediction boxes, as a complementary term of the target detection loss function, the network is guided to actively punish unnecessary overlap between the prediction boxes during the training process, especially when they correspond to different targets.

[0127] The expression of the loss function of the crowd supervision model is:

[0128]

[0129] wherein Loss represents the loss function of the crowd supervision model, Loss cls represents the cross-entropy classification loss, Loss CIoU represents the improved positioning loss, Loss rep represents the spatio-repulsion loss. , , are the weights of Loss cls , Loss CIoU , and Loss rep respectively.

[0130] The improved positioning loss Loss CIoU is represented as:

[0131]

[0132] wherein IoU represents the intersection ratio of the prediction box and the real target box, represents the Euclidean distance square between the center point b of the prediction box and the center point b gt of the real target box, c 2 represents the square of the diagonal length of the minimum circumscribed rectangle covering the prediction box and the real target box, β represents the weight coefficient of the width-height ratio dynamic penalty term, w and h respectively represent the width and height of the prediction box, w gt , h gt respectively represent the width and height of the real target box.

[0133] The repulsion loss Loss rep is represented as:

[0134]

[0135] wherein Ω represents the total number of prediction boxes to which the repulsion loss is applied, ReLU( ) represents an activation function, denotes the IoU between the ith prediction box and the jth prediction box, i, j = 1, 2, …, N, N denotes the number of prediction boxes participating in the spatiotemporal repulsion loss, and δ denotes the maximum overlap threshold allowed, denotes the mask between the ith prediction box and the jth prediction box. , it is explained that two prediction boxes are probably two independent and different targets (for example, two people far apart), and we hope that the prediction boxes corresponding to them are also far apart, so the mask is set to 1, and the repulsion loss needs to be applied. , it is explained that the repulsion loss is not applied, so that the prediction boxes of overlapping targets are allowed to be close.

[0136] The introduction of the spatiotemporal repulsion loss term has the following advantages while maintaining the accuracy of the conventional assignment loss: enhancing the distinguishability between detection boxes; avoiding multiple boxes from being concentrated in the same target or adjacent area; being applicable to dense detection tasks; being seamlessly integrated into the training process of any target detection network; having small computational overhead and not introducing additional network structures or learnable parameters.

[0137] The embodiment also provides a people flow supervision system for a dense sitting posture scene, including a memory, a processor, and computer program instructions stored in the memory and capable of being executed by the processor, and when the processor executes the computer program instructions, the above-mentioned method can be implemented.

[0138] In the embodiment, the method of the application is experimentally verified.

[0139] System deployment and hardware adaptation are performed. First, edge computing device selection is performed. The related hardware configuration is as follows: an NVIDIA Jetson AGX Xavier edge device is deployed, equipped with a 512-core Volta GPU and a 32 GB LPDDR4x memory, and supports 16-way video stream parallel processing. The real-time performance is as follows: 25 FPS processing speed is achieved at 4K resolution, single-frame delay ≤40 ms, and power consumption ≤20 W, meeting the all-weather operation requirements of the dense sitting posture scene such as the dining room waiting hall.

[0140] The performance of the dense sitting posture scene is verified through a self-built dataset. For detection accuracy, on the dining room waiting hall dataset, mAP@0.5 is improved to 94.5%, map@0.95 is improved to 43.1%, accuracy is 93.8%, recall rate is 93.1%, and the requirement for detection accuracy (mAP@0.5 ≥ 90%) of a high-density scene is met. On the edge computing deployment, for Jetson AGX Xavier, the model FLOPs ≤4.2G, supports 16-way 1080P video stream real-time processing (delay ≤40 ms), and the power consumption control full-load power consumption ≤20 W, which adapts to the 7×24-hour operation requirement.

[0141] Those skilled in the art will appreciate that embodiments of the application can be readily used as software, hardware, or a combination of software and hardware. In one

[0142] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 one or more flowcharts and / or blocks can represent code, circuits, hardware components, or executable computer program instructions on a machine-readable or computer-readable storage medium which, when executed by a computer or other Figure 1 an apparatus with a processor-based platform adapted to perform the functions described in a flowchart block(s) or block(s).

[0143] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 one or more flowcharts and / or blocks can represent code, circuits, hardware components, or executable computer program instructions on a machine-readable or computer-readable storage medium which, when executed by a computer or other Figure 1 an apparatus with a processor-based platform adapted to perform the functions described in a flowchart block(s) or block(s).

[0144] The computer program instructions can also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart block or blocks. Figure 1 one or more flowcharts and / or blocks can represent code, circuits, hardware components, or executable computer program instructions on a machine-readable or computer-readable storage medium which, when executed by a computer or other Figure 1 an apparatus with a processor-based platform adapted to perform the functions described in a flowchart block(s) or block(s).

[0145] The above description is only preferred embodiments of the application, not intended to limit other forms of the application. Any person familiar with the art can make changes or modifications to the above-mentioned technical content as equivalent embodiments. However, any simple modification, equivalent change and modification of the above-mentioned embodiments without departing from the technical solution of the application, according to the technical essence of the application, still belongs to the protection scope of the technical solution of the application.

Claims

1. A method for crowd supervision in dense sitting scenarios, characterized in that, The application relates to a people flow supervision method and device. S1: acquiring video stream data of a dense sitting posture scene, constructing an original data set and optimizing anchor box size distribution to obtain a training data set; S2: constructing a people flow supervision model, wherein the people flow supervision model is improved on a YOLOv7 model, in the backbone network, the first four CBS modules are replaced by ODSConv modules, and the first two of the subsequent four ELAN modules are replaced by E-ELAN-Sim modules to enhance the geometric deformation adaptation and dynamic feature screening capability; the ODSConv module is realized by combining an ODConv module and an SReLU activation function; the E-ELAN-Sim module is realized by splicing an E-ELAN module and a SIMAM module; in the head network, a coordinate attention mechanism is embedded in the SPPCSPC module to obtain a CA-SPPCSPC module, and a CA module is added before the CBS module connecting the ELAN-H module and the Concat module to enhance the representation ability of multi-scale features, and a four-scale detection framework composed of four detection heads is realized in the head network to detect super-small, small, medium and large targets to enhance the small target detection capability; the people flow supervision model is trained by the training data set to obtain a trained people flow supervision model; In step S1, video stream data of a dense sitting posture scene including a waiting hall and a restaurant is acquired, image frames are extracted from the video stream data, and real target boxes in the image frames are labeled, and then an original data set is constructed; 2. The method of claim 1, wherein, For the real target boxes labeled in the original data set, k-means++ clustering algorithm is used for clustering analysis to obtain the optimal anchor box size distribution; the specific steps include the following steps: S101: acquiring the width and height information of all labeled real target boxes; S102: using k-means++ clustering algorithm to select initial centers as clustering centers from all labeled real target boxes; S103: calculating the distance between all labeled real target boxes and each clustering center by the following formula: S104: selecting k=12 for clustering to obtain 12 anchor boxes; wherein, box denotes a labeled real target frame, centroid denotes a clustering center, denotes the distance between the labeled real target frame and the clustering center, denotes the IoU distance between the labeled real target frame and the clustering center; S105: sorting the obtained anchor boxes according to their areas, and respectively assigning them to four detection heads of super-small, small, medium and large, and using three anchor boxes for each detection head to cover human targets of different scales. In step S2, the implementation method of the ODSConv module is as follows:

3. The method of claim 1, wherein, S2101: inserting a Conv2d module before the ODConv module, performing two-dimensional convolution on the input feature map through the Conv2d module, and then inputting the ODConv module for feature extraction; S2102: inputting the result output by the ODConv module into the SReLU activation function after batch normalization processing by the bn layer to obtain the output of the ODSConv module; the expression of the SReLU activation function is as follows: In step S2, the implementation method of the E-ELAN-Sim module is as follows: wherein SReLU(x) represents a result obtained by processing an input feature map x through an SReLU activation function, 、 are respectively left and right thresholds for dynamically adjusting the activation interval, 、 are respectively slope coefficients of the negative and positive half-axes, used to control the nonlinearity strength; 、 、 、 are all learnable parameters, which are dynamically updated in the model training process.

4. The method of claim 1, wherein, The process is as follows: S2201: divide the input feature map into two paths, the first path passes through a first CBS module to obtain a feature map F c1 , and the second path sequentially passes through five GhostConv modules to perform ghost convolution to obtain five feature maps;​ ​ Wherein, GhostConv1(), GhostConv2(), GhostConv3(), GhostConv4(), GhostConv5() respectively represent five GhostConv modules performing ghost convolution, F1, F2, F3, F4, F5 respectively represent feature maps output by the five GhostConv modules; X represents a feature map input to the E-ELAN-Sim module, C, H, and W respectively represent the channel number, height, and width of the feature map; S2202: Convert the feature map F output from the first path... c1 The feature maps F1, F3, and F5 from the second output are concatenated along the channel dimension, as shown below: where F e represents the output of the E-ELAN module, and Concat() represents a concatenation operation. S2203: obtaining the output F of the E-ELAN module e As input to the SIMAM module, the weighted feature maps F s are then passed through a 1x1 convolution to obtain the output of the E-ELAN-Sim module: where F es represents the feature map output by the E-ELAN-Sim module, Conv 1×1 ( ) represents a 1x1 convolution operation.

5. The method of claim 1, wherein, In step S2, the implementation method of the CA-SPPCSPC module is as follows: S2301: Transmit the feature map X from the backbone network b The process is divided into two paths. The first path obtains the feature map F through the second CBS module. c2 The second path obtains the feature map F through the third CBS module. c3 Then enter the CA module; S2302: For the input feature map F c3 , the CA module respectively performs adaptive average pooling in the horizontal direction and the vertical direction to generate the horizontal direction average pooled feature vector and the vertical direction average pooled feature vector , realizes space-channel joint, and the feature vectors and respectively encode the global context information of the row and column dimensions of multiple channels; To generate the spatial attention weights, the concatenated feature vectors and are passed through a 1x1 convolution with a non-linear activation to generate a matrix of spatial attention weights, l: where Concat denotes a concatenation operation, denotes a 1x1 convolution to compress the channel number from 2C to C, denotes a Sigmoid function to make the weight value range in [0, 1]; The spatial attention weight matrix is multiplied element by element with the feature map input to the CA module: wherein, represents the feature map weighted by the spatial attention weight matrix λ, i.e., the output of the CA module, for enhancing the response of the spatial sensitive region; S2303: to the feature map perform multi-scale pooling, generate multi-scale feature maps and concatenate along the channel dimension to obtain output features : In the formula, representing the feature map The receptive field of the max-pooling operation is k x k, k = 5, 9, 13, for extracting context information of different scales; S2304: output the feature The fourth CBS module and the fifth CBS module are sequentially outputted. c5 ; S2305: The feature map F c2 with the feature map F c5 is spliced in the channel dimension, and then is input into a sixth CBS module implemented based on a 1x1 convolution to realize feature integration and dimension compression, and finally an output of the CA-SPPCSPC module is obtained to reserve the detail information while strengthening the spatial positioning capability. wherein F cs represents the feature map output by the CA-SPPCSPC module.

6. The method of claim 1, wherein, In step S2, the implementation method of the four-scale detection framework is as follows: On the basis of the PAFPN module feature pyramid of the head network, an upsampling and downsampling process is additionally added to realize super-small target detection, specifically: a first ELAN-H module, a seventh CBS module, a third upsampling module, a first Concat module, a second ELAN-H module, an MP2 module, and a second Concat module are sequentially connected between the Concat module after the second upsampling and the ELAN-H module at the P3 level, and the first ELAN-H module and the second Concat module are connected through a first CA module and a ninth CBS module; the feature map of 80x80x128 is upsampled to 160x160x128 through the third upsampling module and the first Concat module, and then the obtained feature map of 160x160x128 is input to the second ELAN-H module, the second ELAN-H module outputs a super-small target through a RepConv module, thereby realizing the first detection head, i.e., the detection head at the P2 level; the output of the second ELAN-H module is input to the MP2 module as the input of downsampling; the output of the MP2 module is spliced with the output of the ninth CBS module in the channel dimension, and then the obtained result is input to the ELAN-H module at the P3 level; thereby, the detection head at the P2 level is added on the basis of the detection heads at the P3, P4, and P5 levels of the PAFPN module feature pyramid, and the four-scale detection framework is realized; The spatial size and channel dimension of the feature map output by each level detection head are as follows: P2：160×160×128 P3：80×80×256 P4：40×40×512 P5：20×20×1024。 7. The method of claim 1, wherein, The loss function of the passenger flow supervision model is improved on the basis of the loss function of the YOLOv7 model by introducing a width-height ratio dynamic penalty term to improve the CIoU loss function and introducing a space-time repulsion loss, and its expression is as follows: wherein Loss represents a loss function of the crowd flow supervision model, Loss cls represents a cross-entropy classification loss, Loss CIoU represents an improved positioning loss, Loss rep represents a spatio-temporal repulsion loss; 、 、 are weights of Loss cls , Loss CIoU , and Loss rep , respectively. The improved positioning loss Loss CIoU is represented as: wherein, IoU represents the intersection over union of the prediction box and the real target box, represents the Euclidean distance square between the prediction box center point b and the real target box center point b gt 2 represents the square of the diagonal length of the minimum circumscribed rectangle covering the prediction box and the real target box, β represents the weight coefficient of the width-height ratio dynamic penalty term, w, h respectively represent the width and height of the prediction box, gt gt respectively represent the width and height of the real target box;​​ The repulsion loss Loss rep is represented as: wherein Ω represents the total log of the prediction boxes to which the repulsion loss is applied, ReLU( ) represents an activation function, represents the IoU between the i-th prediction box and the j-th prediction box, i, j = 1, 2, …, N, N represents the number of prediction boxes participating in the spatiotemporal repulsion loss, and δ represents the maximum overlap threshold allowed, represents the mask between the i-th prediction box and the j-th prediction box.

8. A people flow supervision system for dense sitting scenarios, characterized by, The computer program instructions stored in the memory and capable of being executed by the processor can implement the method of any one of claims 1-7 when the processor executes the computer program instructions. The computer program instructions stored in the memory and capable of being executed by the processor can implement the method of any one of claims 1-7 when the processor executes the computer program instructions.

Images