A crowd counting method of MFFBSNet network

Abstract

The application discloses a kind of MFFBSNet network crowd counting methods based on multi-scale feature fusion and background suppression, it is characterized by comprising: the original image is input VGG-16 backbone network, the output feature of conv3_3, conv4_3, conv5_3 is extracted respectively 3×3 convolution operation, uniform channel number input is constructed by PSA module and ASPP module Multi-scale feature fusion module, obtain multi-scale feature map;Multi-scale feature map is input channel space attention module, obtain the multi-scale feature map of fusion depth feature, input hole convolution module, obtain density feature map, and carry out convolution dimension reduction, input sigmoid function, obtain background segmentation attention map, and high-quality density map is obtained by fusing with density feature map, and crowd counting is carried out.The application can obtain more abundant multi-scale information, improve the background noise suppression capability, and improve the counting network performance.

Description

Technical Field

[0001] This invention relates to the field of dense crowd counting technology, and in particular to a crowd counting method based on MFFBSNet network using multi-scale feature fusion and background suppression. Background Technology

[0002] Crowd counting primarily aims to estimate the total number of people in complex scenarios, finding wide application in public safety management and urban traffic management. With the increasing urban population, large-scale gatherings are becoming more frequent, particularly at stations, tourist attractions, and large-scale sporting or musical performances. If not controlled promptly, these gatherings can easily lead to stampedes and other incidents endangering public safety. Crowd counting networks can estimate the number of people in these scenarios, providing timely warnings and preventing dangerous events. However, in real-world scenarios, significant density differences between images, diverse target scales, and background occlusion limit the accuracy of crowd counting, leaving considerable room for improvement.

[0003] Traditional crowd counting methods are mainly divided into detection-based and regression-based methods. Detection-based methods detect heads in an image and then sum them up to obtain the total number of people. These methods perform well in low-density scenes, but their performance drops sharply in high-density scenes or scenes with occluded objects. Regression-based methods learn the mapping relationship between image features and crowd numbers, and then train a regression model to count people based on this learned mapping. However, these methods heavily rely on manually extracted features and cannot provide detailed information such as the spatial distribution of the crowd. They also perform poorly in complex scenes with significant variations in lighting, viewing angle, and crowd distribution. In summary, traditional methods mainly rely on manually extracted features to identify crowds, which is easily affected by problems such as viewing angle distortion, scale changes, and background interference. They cannot accurately predict the true number of people in an image and therefore perform poorly in real-world scenarios.

[0004] In recent years, crowd counting algorithms based on deep learning have become increasingly popular, with more and more CNN-based methods being applied to the field of crowd counting and achieving good results. To address the problem of varying head sizes and drastic scale changes in crowd scenes, Zhang et al. proposed a multi-column convolutional neural network (MCNN), which uses branches with convolutional kernels of different sizes to construct a crowd counting model to extract low-level and high-level feature information. However, its model structure is redundant, and its generalization ability needs improvement. Therefore, Li et al. combined the first ten layers of VGG-16 with dilated convolutions to propose a dense scene recognition network (CSRNet) based on extended convolutions. This network better aggregates multi-scale features in crowded scenes while reducing network redundancy, avoiding the loss of spatial information. Similar to CSRNet, Sindagi et al. proposed a counting network, MBTTBF, which bidirectionally fuses low-level and high-level features as the final output, resulting in better fusion of features at different scales. Although MBTTBF and CSRNet have addressed the negative impact of scale changes on counting results to some extent, the problem of background noise such as trees and buildings in images severely affecting the accuracy of crowd counting persists.

[0005] To address the challenge of background noise affecting the performance of counting networks, ZHU et al. proposed the symmetric dual-path multi-scale fusion network SFANet. It utilizes the spatial attention map generated by path one to guide the density feature map generated by path two, reducing noise interference and generating a high-quality density map. However, the quality of the generated spatial attention map is low, and the counting performance remains poor in cluttered scenes. Xu Xiaochen et al. added spatial attention and channel attention mechanisms to the CSRNet network, enabling it to better learn key information in the image and reduce the impact of complex background environments and occlusions. However, the model structure is complex, and training is difficult.

[0006] The above methods only perform simple feature extraction on the input image, ignoring large-scale pixel-level contextual information and failing to effectively solve the problem of background occlusion affecting the counting effect.

[0007] Therefore, how to provide a crowd counting method based on multi-scale feature fusion and background suppression of the MFFBSNet network that can acquire richer multi-scale information and improve background noise suppression capabilities to enhance the performance of counting networks is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0008] In view of this, the present invention proposes a crowd counting method for MFFBSNet network based on multi-scale feature fusion and background suppression.

[0009] To achieve the above objectives, the present invention adopts the following technical solution:

[0010] A crowd counting method for MFFBSNet network based on multi-scale feature fusion and background suppression includes:

[0011] Step (1): Input the original image into the VGG-16 backbone network and extract the output features of conv3_3, conv4_3 and conv5_3;

[0012] Step (2): Multi-scale feature fusion: The output features of conv5_3, conv4_3, and conv3_3 are used as F1, F2, and F3, and 3×3 convolution operations are performed on them respectively. After unifying the number of channels, F1', F2', and F3' are obtained. They are then input into the multi-scale feature fusion module constructed by the PSA module and the ASPP module to obtain the multi-scale feature map.

[0013] Step (3): Background noise suppression: Input the multi-scale feature map into the channel spatial attention module, including: the channel attention module and the spatial attention module connected in sequence, to obtain the multi-scale feature map fused with deep features;

[0014] Step (4): Dilated convolution and background segmentation attention fusion: Input the multi-scale feature map with fused depth features into the dilated convolution module to obtain the density feature map, and perform 1×1×1 convolution dimensionality reduction. Input the sigmoid function to obtain the background segmentation attention map, and fuse it with the density feature map to obtain a high-quality density map for crowd counting.

[0015] Optionally, in step (2), the multi-scale feature fusion module specifically includes:

[0016] Input F1' into the PSA module to obtain feature map P1, and perform upsampling. Then concatenate it with F2' and input it into the ASPP module and PSA module connected in sequence to obtain feature map P2. Perform upsampling on P2, and concatenate it with F3' into the PSA module to obtain feature map P3. Perform upsampling on P3 to obtain multi-scale feature maps.

[0017] Optionally, in step (2), the PSA module specifically includes:

[0018] The input vector X is divided into S groups according to the channels. Each group is convolved with convolution kernels of different sizes, and the processing results are fused in a cascade manner to obtain feature maps F of different scales.

[0019] The attention weight module is used to extract channel attention from feature maps F at different scales to obtain multi-scale channel attention vectors;

[0020] The attention weights Z' after multi-scale channel interaction are obtained by recalibrating the multi-scale channel attention vector using the SoftMax function.

[0021] The attention weight Z' is element-wise multiplied with feature maps F at different scales to obtain a refined feature map.

[0022] Optionally, in step (2), the ASPP module specifically includes:

[0023] Perform 1×1 convolution on the input feature map;

[0024] Construct a pooling pyramid by stacking corresponding dilated convolutional layers for a given inflation factor;

[0025] Add a dilated pooling layer to compress the feature maps of each channel to 1×1 and perform 1×1 convolution processing;

[0026] The output of the ASPP layers is convolved to obtain the final result.

[0027] Optionally, in step (3), the channel attention module is as follows:

[0028]

[0029] Where F' is the output feature map of the channel attention module; M c For channel attention operations; F is the input feature map; This is for element-wise multiplication.

[0030] Optionally, in step (3), the spatial attention module is as follows:

[0031]

[0032] Where F” is the output feature map of the spatial attention module; M s For spatial attention operations; F' is the output feature map of the channel attention module; This is for element-wise multiplication.

[0033] Optionally, in step (4), the dilated convolution module includes: 3*Conv3-512-2, Conv3-256-2, Conv3-128-2, and Conv3-64-2 connected in sequence.

[0034] Optionally, in step (4), based on the density feature map, a 1×1×1 convolution dimensionality reduction is performed, and the sigmoid function is input to obtain the background segmentation attention map, as follows:

[0035] M att =sigmoid(W⊙f den +b);

[0036] Among them, M att The attention map is used for background segmentation; W and b are the weights and biases of the one-dimensional convolution, respectively; ⊙ represents the convolution operation; fden This is a density feature map.

[0037] Optionally, in step (4), the background segmentation attention map and the density feature map are fused to obtain a high-quality density map, specifically as follows:

[0038] Expand the number of channels in the background segmentation attention map, and combine it with density features. Figure 1 The corresponding element is then multiplied with the corresponding element of the density feature map to obtain a high-quality density map, as follows:

[0039]

[0040] Among them, F refine For high-quality density maps; f den Density feature map; Multiply the elements at corresponding positions; M att Segment the attention map for the background.

[0041] Optionally, in step (4), after fusing the background segmentation attention map and the density feature map to obtain a high-quality density map, the following steps are also included: performing a 1×1 convolution on the high-quality density map to generate the final high-quality density map, as follows:

[0042] F den =Conv 1×1 (F refine );

[0043] Among them, F den For the final high-quality density map; Conv 1×1 Processed as a 1×1 convolution; F refine This is a high-quality density map.

[0044] As can be seen from the above technical solution, compared with the prior art, this invention proposes a crowd counting method based on multi-scale feature fusion and background suppression using the MFFBSNet network. The network structure mainly consists of a VGG-16 network, a multi-scale feature fusion module (MFFM), a convolutional block attention module (CBAM), and a dilated convolution module (DCM). The network first inputs the primary features extracted by the VGG-16 network into the MFFM for processing, obtaining multi-scale features rich in semantic information; secondly, the obtained features are further input into the CBAM for feature map calibration, effectively suppressing background noise; then, the output features are processed by the DCM to generate a background segmentation attention map and a density feature map; finally, the corresponding elements of the two are multiplied to obtain a high-quality predicted density map, thereby achieving more accurate counting for the entire network. Attached Figure Description

[0045] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0046] Figure 1 This is a schematic diagram of the method flow of the present invention.

[0047] Figure 2 This is a schematic diagram of the MFFBSNet network structure based on multi-scale feature fusion and background suppression according to the present invention.

[0048] Figure 3 This is a schematic diagram of the PSA module structure of the present invention.

[0049] Figure 4 This is a schematic diagram of the ASPP module structure of the present invention.

[0050] Figure 5 This is a schematic diagram of the channel space attention module of the present invention.

[0051] Figure 6(a) is a schematic diagram of the original image of the present invention for a scene with a relatively dense population distribution.

[0052] Figure 6(b) is a schematic diagram of the actual density map of the present invention for a densely populated scene.

[0053] Figure 6(c) is a schematic diagram of the background segmentation attention map for densely populated scenes according to the present invention.

[0054] Figure 6(d) is a schematic diagram of the predicted density map for densely populated scenes according to the present invention.

[0055] Figure 7(a) is a schematic diagram of the original images of the present invention for different head sizes.

[0056] Figure 7(b) is a schematic diagram of the actual density map of the present invention for different human head sizes.

[0057] Figure 7(c) is a schematic diagram of the attention map for background segmentation with different head sizes according to the present invention.

[0058] Figure 7(d) is a schematic diagram of the predicted density map for different head sizes according to the present invention.

[0059] Figure 8(a) is a schematic diagram of the original image of the present invention for a crowded scene with existing trees, etc.

[0060] Figure 8(b) is a schematic diagram of the actual density of existing crowded scenarios such as trees, according to the present invention.

[0061] Figure 8(c) is a schematic diagram of background segmentation attention map for crowded scenes such as trees in this invention.

[0062] Figure 8(d) is a schematic diagram of the predicted density map for crowded scenarios such as existing trees, according to the present invention.

[0063] Figure 9(a) is a schematic diagram of the original image of the present invention for crowded scenarios such as existing buildings.

[0064] Figure 9(b) is a schematic diagram of the actual density of existing buildings and other crowded scenarios according to the present invention.

[0065] Figure 9(c) is a schematic diagram of background segmentation attention map for existing crowded scenes such as buildings according to the present invention.

[0066] Figure 9(d) is a schematic diagram of the predicted density map for existing crowded scenarios such as buildings, according to the present invention. Detailed Implementation

[0067] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0068] Example 1:

[0069] Embodiment 1 of this invention discloses a crowd counting method for MFFBSNet network based on multi-scale feature fusion and background suppression, such as... Figure 1 As shown, it includes:

[0070] The MFFBSNet network structure based on multi-scale feature fusion and background suppression, such as Figure 2 As shown.

[0071] Given VGG-16's powerful feature extraction capabilities and standardized network structure, this invention employs VGG-16 as the backbone network, with four layers (conv2_2, conv3_3, conv4_3, and conv5_3) serving as output features. Considering the significant noise in shallow features, only conv3_3, conv4_3, and conv5_3 are retained as input features for the next stage. Based on this, a multi-scale feature fusion module is proposed, fusing multi-level features through top-down and lateral connections to address the issue of continuous scale variations. Furthermore, to enhance the network's ability to suppress background noise, this invention designs a channel-space attention module, which performs channel-space attention calibration on the input feature map, emphasizing the head region in the image. Finally, to extract image details, the calibrated feature map is input into a dilated convolution module to extract deeper features, resulting in a background segmentation attention map and a density feature map. Multiplying corresponding elements of these two maps yields a high-quality density map. Wherein: Convolution operation is represented by Conv kernel × convolution kernel (number of channels) and Conv kernel - number of channels - dilation rate, and setting 3*Conv means that the convolution of this layer is performed three times.

[0072] Step (1): Input the original image into the VGG-16 backbone network and extract the output features of conv3_3, conv4_3 and conv5_3.

[0073] Step (2): Multi-scale feature fusion: The output features of conv5_3, conv4_3, and conv3_3 are used as F1, F2, and F3, respectively, and 3×3 convolution operations are performed on them. After unifying the number of channels to 256, F1', F2', and F3' are obtained. They are then input into the multi-scale feature fusion module constructed by the PSA module and the ASPP module to obtain the multi-scale feature map.

[0074] CNN networks are highly adaptable to positional changes, but their performance is somewhat lacking in scale-dependent applications. Therefore, Lin et al. proposed the Feature Pyramid Network (FPN), which can simultaneously utilize the strong semantic information of high-level features and the high resolution of low-level features. Building upon this, Zhang et al. proposed a lightweight pyramid split attention module (PSA). The PSA module can fully extract multi-scale spatial information and important cross-dimensional features from the channel attention vectors, improving multi-scale representation capabilities at a finer granular level. Chen et al. proposed an atrous spatial pyramid pooling (ASPP) module, which uses convolutional kernels with multiple sampling rates to detect incoming convolutional features, thereby obtaining richer multi-scale information. This invention uses ASPP and PSA modules to construct a feature pyramid and build a multi-scale feature fusion module.

[0075] The multi-scale feature fusion module is as follows:

[0076] Inputting F1' into the PSA module yields a feature map P1 with richer multi-scale information representation capabilities. This P1 is then upsampled to twice its original size and concatenated with F2'. The input to the ASPP and PSA modules, which are connected sequentially, obtains receptive fields at different scales, extracting more detailed spatial information to obtain feature map P2. P2 is then upsampled and concatenated with F3' into the PSA module to obtain feature map P3. P3 is then upsampled to obtain a multi-scale feature map containing rich semantic information.

[0077] PSA module, such as Figure 3 As shown, specifically:

[0078] The input vector X is divided into S groups according to the channels. Each group is convolved with convolution kernels of different sizes to extract information at different scales. The processing results are then fused in a cascade manner to obtain feature maps F at different scales.

[0079] The attention weight module is used to extract channel attention from feature maps F at different scales to obtain multi-scale channel attention vectors;

[0080] The attention weights Z' after multi-scale channel interaction are obtained by recalibrating the multi-scale channel attention vector using the SoftMax function.

[0081] By performing element-wise dot products on the attention weight Z' and feature maps F at different scales, we obtain a refined feature map with richer multi-scale feature information.

[0082] ASPP modules, such as Figure 4As shown, this includes dilated convolutions with various sampling rates. By using dilated convolutions with different sampling rates, multi-scale information can be captured. Different padding and dilation techniques can obtain receptive fields of different scales, thereby achieving free multi-scale feature extraction. Specifically:

[0083] ASPP essentially consists of a 1×1 convolutional layer, a pooling pyramid, and dilated pooling layers. The dilation factor of each layer in the pooling pyramid can be customized, thus enabling flexible multi-scale feature extraction.

[0084] The input feature map is first subjected to 1×1 convolution to reduce its dimensionality.

[0085] Construct a pooling pyramid, and for a given dilation factor, stack corresponding dilated convolutional layers to extract features at different scales;

[0086] Add a dilated pooling layer to compress the feature maps of each channel to 1×1, thereby extracting the features of each channel and obtaining the global features. Then, perform 1×1 convolution processing to further extract and reduce the dimensionality of the features obtained in the previous step.

[0087] The output of the ASPP layers is convolved to obtain the final result.

[0088] Step (3): Background noise suppression: Input the multi-scale feature map into the channel spatial attention module, such as... Figure 5 As shown, it includes a channel attention module and a spatial attention module connected in sequence to obtain a multi-scale feature map that fuses deep features.

[0089] By combining VGG-16 with a multi-scale feature fusion structure, a refined feature map with richer multi-scale feature information is obtained, effectively solving the problem of crowd scale variations. Furthermore, the introduction of an attention mechanism enhances the weight of spatial information in the image, making the counting model focus more on the head region and discarding environmental interference information such as houses and trees. This invention inserts a channel spatial attention module (CBAM) after the multi-scale feature fusion module. This module not only focuses on channel domain information but also on spatial domain information, learning from both dimensions. Given an intermediate feature map, channel attention maps and spatial attention maps are generated by learning the channel and spatial relationships between features, and the features are adaptively adjusted accordingly, thereby increasing the weight of the crowd region in the feature map.

[0090] The channel attention module is as follows:

[0091]

[0092] Where F' is the output feature map of the channel attention module; M c For channel attention operations; F is the input feature map; This is for element-wise multiplication.

[0093] Spatial attention module, as follows:

[0094]

[0095] Where F” is the output feature map of the spatial attention module; M s For spatial attention operations; F' is the output feature map of the channel attention module; This is for element-wise multiplication.

[0096] The output of MFFBSNet in the multi-scale feature fusion stage is a multi-scale feature map that integrates features of different depths. These feature maps of different depths have different perceptual capabilities when dealing with heads of different sizes. CBAM increases the dimensional correlation by adjusting the channel dimensions, making the model more adaptable to scenes with crowds of varying sizes and uneven distribution. In addition, CBAM recalibrates the feature maps in the spatial dimension, allowing the network to learn key information in crowd images, enhancing the model's background suppression ability and effectively solving common problems such as occlusion and complex environments in images.

[0097] Step (4): Dilated convolution and background segmentation attention fusion: Input the multi-scale feature map with fused depth features into the dilated convolution module to obtain the density feature map, and perform 1×1×1 convolution dimensionality reduction. Input the sigmoid function and output the probability representation between 0 and 1 to distinguish the head and background regions, obtain the background segmentation attention map, and fuse it with the density feature map to obtain a high-quality density map for crowd counting.

[0098] Considering the presence of numerous small targets in densely populated images, a dilated convolutional module with a small dilation rate is designed after the channel spatial attention module to extract image details and enhance feature extraction. The DCM module outputs two paths: path one outputs a background segmentation attention map, and path two outputs a density feature map. The background segmentation attention map can distinguish the head region from other regions in the image, effectively reducing the interference of background noise on counting. Finally, the background segmentation attention map and the density feature map are fused to generate a high-quality density map, which is then subjected to a 1×1 convolution to produce the final high-quality density map.

[0099] The dilated convolution module includes: 3*Conv3-512-2, Conv3-256-2, Conv3-128-2, and Conv3-64-2 connected in sequence.

[0100] Based on the density feature map, a 1×1×1 convolutional dimensionality reduction is performed. The input is a sigmoid function, and the output is a probability representation between 0 and 1, which is used to distinguish the head and background regions, resulting in a background segmentation attention map, as shown below:

[0101] M att =sigmoid(W⊙f den +b);

[0102] Among them, M att The attention map is used for background segmentation; W and b are the weights and biases of the one-dimensional convolution, respectively; ⊙ represents the convolution operation; f den This is a density feature map.

[0103] The high-quality density map is obtained by fusing the background segmentation attention map and the density feature map, specifically:

[0104] Expand the number of channels in the background segmentation attention map, and combine it with density features. Figure 1 The corresponding element is then multiplied with the corresponding element of the density feature map to obtain a high-quality density map, as follows:

[0105]

[0106] Among them, F refine For high-quality density maps; f den Density feature map; Multiply the elements at corresponding positions; M att Segment the attention map for the background.

[0107] The high-quality density map is processed by a 1×1 convolution to generate the final high-quality density map, as follows:

[0108] F den =Conv 1×1 (F refine );

[0109] Among them, F den For the final high-quality density map; Conv 1×1 Processed as a 1×1 convolution; F refine This is a high-quality density map.

[0110] Experiment and Results Analysis:

[0111] I. Training Details:

[0112] This invention is based on the PyTorch framework and was tested under Windows 10 operating system and NVIDIA GeForce RTX 3090 GPU conditions. Furthermore, this invention uses a pre-trained VGG-16 as the backbone network, trained with an Adam optimizer with a learning rate of 1e-4 and a momentum of 0.9. Each batch of samples contains 8 images, and the number of iterations is set to 500. To ensure sufficient model training, random flipping and cropping operations are performed at different positions in the images to improve model robustness. All experimental results obtained in this invention were achieved under the above environment and parameters.

[0113] Real density map generation: This invention uses a geometrically adaptive Gaussian kernel-based method to generate a real density map of a crowd scene, as follows:

[0114]

[0115] Where x is any pixel in the image; x i Let δ(xx) be the position of the i-th head in the crowd scene; i ) is x i Location information of people at the location; For Gaussian kernel filters; σ i The standard deviation is Gaussian. For x i The average distance between the k people adjacent to it; β is set to 0.3.

[0116] Based on existing annotation information and density maps, a background segmentation attention map can be further generated. This invention uses a threshold to adjust the obtained F... GT (x) is binarized as follows:

[0117]

[0118]

[0119] Where x is F GT The value of the pixel in (x) is given, and th is the threshold value, which is set to 0.001 in this invention. By comparing the value of pixel x with the threshold th, the 0, 1 mapping on the background segmentation attention map is obtained, and finally, the crowd background segmentation attention map A is generated. GT (x).

[0120] Loss Function: To generate high-quality prediction density maps, this invention uses a multi-task joint loss function, optimized with the Euclidean geometric distance loss function, as follows:

[0121]

[0122] Where N is the batch size of training images; θ is the parameter in the network; X i Let X be the i-th image in the dataset; F(X) i ;θ) and F GT These are the predicted density map and the actual density map of the i-th image output by the network, respectively.

[0123] Furthermore, this invention introduces cross-entropy loss as a loss function to obtain an effective background segmentation attention map, as follows:

[0124]

[0125] Among them, P i This represents the probability value of each pixel in the predicted background segmentation attention map after processing with the sigmoid function.

[0126] During training, the two losses are weighted and summed to obtain the final joint loss function, as follows:

[0127] L = L den +αL att ;

[0128] Where α is a hyperparameter, set to 0.1.

[0129] Evaluation Criteria: Two main evaluation metrics are used in crowd counting tasks: Mean Absolute Error (MAE) and Mean Squared Error (MSE). MAE reflects the accuracy of the network; a smaller value indicates higher accuracy. MSE reflects the robustness of the network; a smaller value indicates better robustness. As follows:

[0130]

[0131]

[0132] Among them, C i and Let be the predicted number of people and the actual number of people in the i-th image, respectively.

[0133] II. Results Analysis:

[0134] Dataset: The Shanghai Tech dataset is one of the earliest and most commonly used datasets, divided into two parts based on crowd density. Part A contains 482 images with relatively dense crowds (300 for training and 182 for testing), with a minimum of 33 people and a maximum of 3139 people per image. Part B contains 716 images taken on busy streets in Shanghai (400 for training and 316 for testing), with a minimum of 9 people and a maximum of 578 people per image.

[0135] The UCF-CC-50 dataset is currently the most densely distributed and extremely challenging dataset, consisting of 50 grayscale images with a wide range of population sizes and different resolutions. The number of people in each image ranges from 94 to 4543, with an average of 1280 people per image. The experiments on this dataset in this invention use 5-fold cross-validation to verify the model performance.

[0136] The NWPU-Crowd dataset is a large-scale dataset released by Northwestern Polytechnical University in 2020. It contains 5109 high-resolution images: 3109 training images, 500 validation images, and 1500 test images, including 351 negative samples. The number of people per image ranges from 0 to 20033, with an average of 418 people. There are a total of 2,133,375 head annotations, making it the largest and most challenging crowd counting dataset to date.

[0137] Details of the three datasets are shown in Table 1.

[0138] Table 1 Dataset Information

[0139]

[0140] Experimental Results Analysis: To verify the effectiveness of the network of this invention, it was trained on three challenging datasets and compared with some excellent methods in recent years in terms of counting accuracy. The results are shown in Tables 2 and 3.

[0141] Table 2 Comparison results of each network on the Shanghai Tech and UCF_CC_50 datasets.

[0142]

[0143] Table 3 Comparison results of each network on the NWPU-Crowd dataset.

[0144]

[0145] As shown in Table 2, MFFBSNet achieved MAE and MSE of 63.6 and 100.2 respectively on the Shanghai Tech dataset Part A. Compared to the second-best FusionCount dataset, although the MAE is slightly lower, the MSE is reduced by 0.98%, indicating that MFFBSNet has good stability. On the Shanghaitech dataset Part B, the MAE and MSE reached the current best of 6.9 and 11.3, indicating that counting networks also perform well in relatively sparse crowd scenarios. On the UCF_CC_50 dataset with small samples and a large density range, compared with URC, the MAE is reduced by 33.2% and the MSE by 38.0%, indicating that counting networks also have good counting performance with a small number of samples. As shown in Table 3, on the latest large-scale dataset NWPU-Crowd, although negative samples increase the training difficulty, they also improve the model's generalization ability. Compared with other methods, MAE and MSE reach the current best scores of 102.3 and 416.2, respectively, proving that MFFBSNet still has better counting accuracy even with a large number of samples and significant differences in image resolution.

[0146] The experimental data in Tables 2 and 3 demonstrate that the network of this invention exhibits high counting accuracy and good generalization ability regardless of whether the crowd is dense or sparse. Even in complex crowd scenarios with large variations in crowd density, blurred heads, and background occlusion, it still maintains impressive counting accuracy, reflecting the network's strong robustness.

[0147] To better illustrate the prediction performance of the proposed model, some visualized prediction results of MFFBSNet on the Shanghai Tech dataset are shown in Figures 6, 7, 8, and 9. Here, GT Count represents the actual number of people in the scene, and Est Count represents the predicted number of people. Figure 6(a) , 6(b) Figures 6(c) and 6(d) show the original image, the actual density map, the background segmentation attention map, and the predicted density map for a scene with a relatively dense crowd, respectively. As can be seen from Figure 6, for images of scenes with a relatively dense crowd, the number of people predicted by the network is close to the actual number of people. Figure 7(a) , 7(b) Figures 7(c) and 7(d) show the original image, the true density map, the background segmentation attention map, and the predicted density map for images with different head scales, respectively. As can be seen from Figure 7, MFFBSNet can maintain a small relative error for images with different head scales. Figure 8(a) , 8(b) Images 8(c) and 8(d) represent the original image, the ground truth density map, the background segmentation attention map, and the predicted density map, respectively, for crowded scenes with trees and other elements present. Figure 9(a) ,9(b) Figures 8, 9(c), and 9(d) show the original image, the true density map, the background segmentation attention map, and the predicted density map for crowded scenes with existing buildings, etc., respectively. As can be seen from Figures 8 and 9, for crowded scene images with existing buildings, trees, etc., the background segmentation attention map generated by the network of this invention can better distinguish between the foreground and background regions and reduce the interference of occlusions on the prediction results.

[0148] In summary, the network of this invention can adapt well to dense crowd scenes with large scale variations and severe background interference, and the generated predicted density map is closer to the real density map, demonstrating the superior performance of this network in the field of crowd counting.

[0149] III. Ablation Experiment:

[0150] The model proposed in this invention mainly consists of three modules: MFFM, CBAM, and DCM. In order to further verify the rationality and effectiveness of each part of the structure, ablation experiments were conducted on MFFM, DCM, MFFM+CBAM, MFFM+DCM, CBAM+DCM, and MFFM+CBAM+DCM on the Shanghai Tech dataset Part_A. The results are shown in Table 4.

[0151] Table 4 Ablation Experiment Results of Modular Structure

[0152]

[0153] Table 4 shows that both MFFM and DCM alone can acquire crowd information to some extent, but their counting accuracy is low. After fusing MFFM and DCM, the counting accuracy increases, indicating that MFFM can improve the expressive power and counting performance of multi-scale features. When MFFM and DCM are combined with CBAM respectively, the MAE decreases by 3.4% and 3.8% compared to using MFFM and DCM alone, indicating that CBAM has a good suppression effect on background noise. These experimental results demonstrate that the multi-scale feature fusion module and the channel spatial attention module have a strong effect on improving network performance.

[0154] This invention discloses a crowd counting method based on MFFBSNet network using multi-scale feature fusion and background suppression. The multi-scale feature fusion module uses a feature pyramid structure to effectively fuse features at different levels, improving the continuity of multi-scale information. The channel spatial attention module effectively suppresses background noise while enhancing the ability to retain head information, effectively solving common problems such as background occlusion and complex environments in images. The model was tested on three benchmark datasets, and its evaluation metrics outperformed other comparative models, demonstrating good counting accuracy and robustness for various challenging dense crowd images.

[0155] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to the method section.

[0156] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A crowd counting method based on MFFBSNet network with multi-scale feature fusion and background suppression, characterized in that, include: Step (1): Input the original image into the VGG-16 backbone network and extract the output features of conv3_3, conv4_3 and conv5_3; Step (2): Multi-scale feature fusion: The output features of conv5_3, conv4_3, and conv3_3 are used as F1, F2, and F3, and 3×3 convolution operations are performed on them respectively. After unifying the number of channels, F1', F2', and F3' are obtained. They are then input into the multi-scale feature fusion module constructed by the PSA module and the ASPP module to obtain the multi-scale feature map. Step (3): Background noise suppression: Input the multi-scale feature map into the channel spatial attention module, including: the channel attention module and the spatial attention module connected in sequence, to obtain the multi-scale feature map fused with deep features; Step (4): Dilated convolution and background segmentation attention fusion: Input the multi-scale feature map of the fused depth features into the dilated convolution module to obtain the density feature map, and perform 1×1×1 convolution dimensionality reduction. Input the sigmoid function to obtain the background segmentation attention map, and fuse it with the density feature map to obtain a high-quality density map for crowd counting.

2. The crowd counting method for MFFBSNet network based on multi-scale feature fusion and background suppression according to claim 1, characterized in that, In step (2), the multi-scale feature fusion module specifically includes: Input F1' into the PSA module to obtain feature map P1, and perform upsampling. Then concatenate it with F2' and input it into the ASPP module and PSA module connected in sequence to obtain feature map P2. Perform upsampling on P2, and concatenate it with F3' into the PSA module to obtain feature map P3. Perform upsampling on P3 to obtain multi-scale feature maps.

3. The crowd counting method for MFFBSNet network based on multi-scale feature fusion and background suppression according to claim 1, characterized in that, In step (2), the PSA module specifically includes: The input vector X is divided into S groups according to the channels. Each group is convolved with convolution kernels of different sizes, and the processing results are fused in a cascade manner to obtain feature maps F of different scales. The channel attention of the feature maps F at different scales is extracted using the attention weight module to obtain a multi-scale channel attention vector; The attention weights Z' after multi-scale channel interaction are obtained by recalibrating the multi-scale channel attention vector using the SoftMax function. The attention weight Z' is element-wise multiplied with the feature maps F at different scales to obtain a refined feature map.

4. The crowd counting method for MFFBSNet network based on multi-scale feature fusion and background suppression according to claim 1, characterized in that, In step (2), the ASPP module specifically includes: Perform 1×1 convolution on the input feature map; Construct a pooling pyramid by stacking corresponding dilated convolutional layers for a given inflation factor; Add a dilated pooling layer to compress the feature maps of each channel to 1×1 and perform 1×1 convolution processing; The output of the ASPP layers is convolved to obtain the final result.

5. The crowd counting method for MFFBSNet network based on multi-scale feature fusion and background suppression according to claim 1, characterized in that, In step (3), the channel attention module is as follows: Where F' is the output feature map of the channel attention module; M c For channel attention operations; F is the input feature map; This is for element-wise multiplication.

6. The crowd counting method for MFFBSNet network based on multi-scale feature fusion and background suppression according to claim 1, characterized in that, In step (3), the spatial attention module is as follows: Where F” is the output feature map of the spatial attention module; M s For spatial attention operations; F' is the output feature map of the channel attention module; This is for element-wise multiplication.

7. The crowd counting method for MFFBSNet network based on multi-scale feature fusion and background suppression according to claim 1, characterized in that, In step (4), the dilated convolution module includes: 3*Conv3-512-2, Conv3-256-2, Conv3-128-2, and Conv3-64-2 connected in sequence.

8. The crowd counting method for MFFBSNet network based on multi-scale feature fusion and background suppression according to claim 1, characterized in that, In step (4), based on the density feature map, a 1×1×1 convolution dimensionality reduction is performed, and the sigmoid function is input to obtain the background segmentation attention map, as follows: M att =sigmoid(W⊙f den +b); Among them, M att The attention map is used for background segmentation; W and b are the weights and biases of the one-dimensional convolution, respectively; ⊙ represents the convolution operation; f den This is a density feature map.

9. The crowd counting method for MFFBSNet network based on multi-scale feature fusion and background suppression according to claim 1, characterized in that, In step (4), the background segmentation attention map and the density feature map are fused to obtain a high-quality density map, specifically as follows: The number of channels in the background segmentation attention map is expanded to match the density feature map, and then multiplied with the corresponding elements of the density feature map to obtain a high-quality density map, as follows: Among them, F refine For high-quality density maps; f den Density feature map; Multiply the elements at corresponding positions; M att Segment the attention map for the background.

10. The crowd counting method for MFFBSNet network based on multi-scale feature fusion and background suppression according to claim 1, characterized in that, In step (4), after the background segmentation attention map and the density feature map are fused to obtain a high-quality density map, the method further includes: performing a 1×1 convolution on the high-quality density map to generate the final high-quality density map, as follows: F den =Conv 1×1 (F refine ); Among them, F den For the final high-quality density map; Conv 1×1 Processed as a 1×1 convolution; F refine This is a high-quality density map.

Images