Crowd density estimation method and system based on feature-aware weighted contrastive learning

Abstract

The application provides a crowd density estimation method and system based on feature-aware weighted contrast learning, relates to the technical field of crowd density estimation methods, and comprises the following steps: inputting a basic feature map into a constructed multi-level parallel hole convolution layer respectively, normalizing an importance score to obtain a fusion weight coefficient of each branch, and obtaining a final fusion feature map; constructing a regression network to generate a low-resolution density map, inputting the density map into a lightweight convolution network to generate a scaling mask, multiplying the density map after bilinear upsampling with the scaling mask element by element, and generating a predicted density map. The multi-level parallel hole convolution layer enables the network to simultaneously capture local details and wide-range context information, the channel attention automatically strengthens the feature channels related to the crowd, the spatial attention accurately locates the high-density area, the density map is calibrated to the actual number of people for each pixel through a loss function, and the number of people in the whole image can be directly obtained by summation.

Description

Technical Field

[0001] This invention relates to the field of crowd density estimation methods, specifically to a crowd density estimation method and system based on feature-aware weighted contrastive learning. Background Technology

[0002] Existing crowd density estimation methods typically rely on a single convolutional neural network to extract features, which struggles to effectively address the multi-scale characteristics and spatial non-uniformity of crowd distribution in complex scenarios. Traditional methods often employ simple concatenation or addition operations during feature fusion, failing to fully consider the differences in the importance of features at different scales. Furthermore, the application of attention mechanisms is often limited to a single dimension, lacking modeling of the synergistic effect between the two, resulting in insufficient ability to perceive details in dense regions and limited adaptability to scenes where sparse and dense areas coexist. Ultimately, this affects the accuracy of density maps and the precision of crowd counting.

[0003] In the existing technology, the document with publication number CN117351414A describes a method for constructing a crowd density estimation model by building a difference texture module and a multi-channel threshold permutation attention module, and selecting the first 10 convolutional layers of the VGG-16 network as the front-end network. However, this method does not effectively capture the multi-scale spatial features of crowd images through multi-level parallel dilated convolutional structures, nor does it achieve dual enhancement of key information by combining spatial-channel joint attention mechanisms. It does not adaptively integrate the contributions of different scale branches through an importance score weighted fusion strategy to significantly improve the model's perception and representation ability of complex crowd distributions, nor does it generate a high-resolution density map with richer details and clearer boundaries. It also does not significantly improve the accuracy and robustness of crowd counting in sparse, dense, and mixed scenes while maintaining low computational overhead. Therefore, there is an urgent need for a crowd density estimation method and system based on feature-aware weighted contrastive learning.

[0004] The information disclosed in the background section is only intended to enhance the understanding of the background of this disclosure, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0005] The purpose of this invention is to provide a method and system for estimating crowd density based on feature-aware weighted contrastive learning, so as to solve the problems mentioned in the background art.

[0006] To achieve the above objectives, the present invention provides the following technical solution:

[0007] The crowd density estimation method based on feature-aware weighted contrastive learning includes the following steps:

[0008] S1: The image to be analyzed is standardized and then input into a pre-trained feature extraction network. The feature extraction network generates a basic feature map corresponding to the image to be analyzed by analyzing spatial features and channel features.

[0009] S2: Input the basic feature map into a multi-level parallel dilated convolutional layer. Each parallel branch uses dilated convolutional kernels of different scales to perform convolution operations to generate branch feature maps. Pooling operations are performed on the branch feature maps output by each branch.

[0010] S3: After processing the pooled branch feature maps through the attention mechanism, enhanced feature maps are generated, and the corresponding importance scores are calculated based on the response intensity of each enhanced feature map; the importance scores of all branches are normalized to obtain the fusion weight coefficients, and the fusion weight coefficients are weighted and summed with the corresponding enhanced feature maps to obtain the fusion feature map corresponding to the image to be analyzed;

[0011] S4: Input the fused feature map into the pre-trained regression network to generate a low-resolution prediction density map. Upsample the low-resolution prediction density map to obtain a high-resolution density map. Input the low-resolution prediction density map into a lightweight network to generate a scaling mask. Multiply the high-resolution density map and the scaling mask element-wise to obtain the prediction density map.

[0012] S5: Apply nonnegativity constraints to the generated predicted density map and sum the population density values ​​of all pixels to obtain the population density estimate output of the image to be analyzed.

[0013] Further, the basic feature map is obtained, and the specific steps are as follows:

[0014] The original image of the crowd is obtained, and the pixel values ​​of the original image are decomposed according to the three color channels of red, green and blue. The pixel value of each channel is obtained. The pixel value of each channel is subtracted from the preset target standard average value and then divided by the preset target standard deviation value to obtain the standardized image. The standardized image is input into the feature extraction network, which is a pre-trained ResNet-50 network. The feature map output of the last convolutional layer of the ResNet-50 network is extracted as the base feature map.

[0015] Further, the branch feature map of the output of each branch is obtained, and the specific steps are as follows:

[0016] The basic feature map is subjected to a hole rate of 2. The convolution kernel operation involves standardizing the features of each channel to zero mean and unit variance through a batch normalization layer, and then scaling the batch normalized pixels using the ReLU activation function to obtain the first branch feature map.

[0017] The basic feature map is subjected to a hole rate of 4. The convolution kernel operation involves standardizing the features of each channel to zero mean and unit variance through a batch normalization layer. The pixels after batch normalization are then scaled using the ReLU activation function to obtain the second branch feature map.

[0018] The basic feature map is subjected to a hole rate of 6. The convolution kernel operation involves standardizing the features of each channel to zero mean and unit variance through a batch normalization layer. The batch normalized pixels are then scaled using the ReLU activation function to obtain the third branch feature map.

[0019] Furthermore, the branch feature maps are processed using an attention mechanism, specifically through the following steps:

[0020] Global average pooling and global max pooling are performed on the feature maps output by each branch to obtain the corresponding feature maps. The two feature maps are then sequentially input into a multilayer perceptron with shared weights for processing. The multilayer perceptron consists of two fully connected layers with a non-linear transformation through the ReLU activation function in between. The feature maps processed by the multilayer perceptron are summed element by element, and finally the sum is normalized to between 0 and 1 by the Sigmoid function to generate a channel attention weight vector representing the feature map of each branch.

[0021] Simultaneously, the pooled feature maps are concatenated along the channel dimension, and the concatenated feature maps are then processed using dilatants with a dilatancy of 1 and a dilatancy of 3, respectively. Convolutional layers are used, and batch normalization is performed on each layer to obtain local detail feature maps and context detail feature maps, respectively.

[0022] The local detail feature map and the context detail feature map are weighted and fused using a preset learnable scalar parameter of 0.5. Specifically, the learnable scalar parameter of 0.5 is multiplied by each pixel of the local detail feature map, and then 1 is subtracted from the learnable scalar parameter of 0.5 multiplied by each pixel of the context detail feature map. The two are then added together to obtain the weighted fused feature map.

[0023] The weighted fused feature map is input into a feature channel that is compressed to 1. Convolutional layers are used, and the results are normalized using the Sigmoid function to obtain the spatial attention weight matrix for each branch feature map.

[0024] Further, an enhanced feature map is generated, and the specific steps are as follows:

[0025] Based on the feature map output from each branch, attention enhancement is achieved by expanding the channel attention weight vector and the spatial attention weight matrix through an outer product and then multiplying them element-wise.

[0026]

[0027] in, Indicates the first Enhanced feature maps of each branch; Indicates the first Each branch feature map; This indicates the outer product expansion operation; This represents the element-wise multiplication method; Indicates the first Channel attention weight vectors of each branch feature map; Represents the branch number index, and ; Indicates the first Spatial attention weight matrix for each branch feature map.

[0028] Further, the importance score of the enhanced feature map for each branch is calculated, specifically through the following steps:

[0029] Calculate the Pearson correlation coefficient between the spatial attention weights and channel attention weights of each branch feature map:

[0030]

[0031] in, Indicates the first Pearson correlation coefficients of branch feature maps; Indicates vectorization operation; Indicates the calculation of covariance; This represents the standard deviation of the channel attention weights for all branches; The standard deviation of the spatial attention mean across all branches; This represents the mean of the spatial attention weights;

[0032] Calculate the energy focusing index for each branch feature map:

[0033]

[0034] in, Represents the norm; Indicates the first Energy focusing index of branch feature maps; Indicates the height of the enhanced feature map; Indicates the width of the enhanced feature map; Represents the pixel-level index of the enhanced feature map; The vertical index of the pixel in the enhanced feature map; Indicates the first The enhanced feature maps of each branch at each pixel The value at;

[0035] Calculate the channel activation index for each branch feature map:

[0036]

[0037] in,

[0038]

[0039] in, Indicates the first The entropy of the channel attention weights of each branch feature map; Indicates the total number of channels; Indicates the number of channel indices; Indicates the first The branch feature map has a channel number of 1 Channel attention weights; Indicates the first Channel activation index of each branch feature map;

[0040] The importance score of the enhanced feature map for each branch is generated by geometrically averaging the Pearson correlation coefficient, energy focusing index, and channel activation index of each branch feature map.

[0041]

[0042] in, Indicates the first The importance score of each branch enhances the feature map.

[0043] Further, the fused feature map integrating information from each branch is obtained, and the specific steps are as follows:

[0044] The importance scores of all branches are normalized to obtain the fusion weight coefficient:

[0045]

[0046] in, Indicates the first The fusion weight coefficients of each branch;

[0047] The fusion weight coefficients are summed with the corresponding enhanced feature maps to obtain a fusion feature map that integrates information from each branch:

[0048]

[0049] in, Represents the fused feature map; express The number of channels is uniform for convolution operations.

[0050] Further, a predicted density map is generated, and the specific steps are as follows:

[0051] The fused feature map is input into a pre-trained regression network to generate a low-resolution prediction density map. This low-resolution prediction density map is then bilinearly upsampled to obtain a high-resolution density map. Specifically:

[0052] The low-resolution predicted density map is upsampled by 32 times using bilinear interpolation to restore its spatial size to be consistent with the original input image, thus obtaining a preliminary high-resolution density map.

[0053] Meanwhile, the low-resolution predicted density map is input into a lightweight convolutional network to generate a scaling mask. Specifically, the same low-resolution predicted density map is input into a lightweight convolutional network, which consists of a single 3×3 convolutional layer and a Sigmoid activation function. The local context information of the density distribution is extracted through the convolutional layer, and a spatially adaptive scaling mask with values ​​between 0 and 1 is generated.

[0054] Finally, the high-resolution density map is multiplied element-wise with the scaling mask to generate the predicted density map.

[0055] Furthermore, to obtain the total number of people in the entire image, the specific steps are as follows:

[0056] Apply a non-negativity constraint to the predicted density map to ensure that the population density value of each pixel is not less than zero:

[0057]

[0058] in, Indicates the non-negative constraint coefficient; Represents pixels The predicted density value at that location; Represents pixels Population density value at the location; Represents the predicted density map; This represents the number of horizontal pixel indices in the predicted density map; This indicates the number of vertically flat pixel indices in the predicted density map;

[0059] The total number of people in the entire image is obtained by summing the crowd density values ​​of all pixels:

[0060]

[0061] in, Indicates the width of the predicted density map; Indicates the height of the predicted density map; Indicates the total number of people.

[0062] The present invention also provides a crowd density estimation system based on feature-aware weighted contrastive learning, the estimation system being used to perform the above-described estimation method, comprising:

[0063] The image acquisition module is used to input the image to be analyzed into a pre-trained feature extraction network after standardization. The feature extraction network generates a basic feature map corresponding to the image to be analyzed by analyzing spatial features and channel features.

[0064] The pooling module is used to input the basic feature map into a multi-level parallel dilated convolutional layer. Each parallel branch uses dilated convolutional kernels of different scales to perform convolution operations to generate branch feature maps, and pooling operations are performed on the branch feature maps output by each branch.

[0065] The fusion feature acquisition module is used to process the pooled branch feature maps through an attention mechanism to generate enhanced feature maps, and calculate the corresponding importance score based on the response intensity of each enhanced feature map; normalize the importance scores of all branches to obtain fusion weight coefficients, and then perform a weighted summation of the fusion weight coefficients and the corresponding enhanced feature maps to obtain the fusion feature map corresponding to the image to be analyzed;

[0066] The prediction density acquisition module is used to input the fused feature map into the pre-trained regression network to generate a low-resolution prediction density map, perform upsampling on the low-resolution prediction density map to obtain a high-resolution density map, input the low-resolution prediction density map into a lightweight network to generate a scaling mask, and multiply the high-resolution density map and the scaling mask element-wise to obtain the prediction density map.

[0067] The population statistics module is used to apply non-negativity constraints to the generated predicted density map and sum the population density values ​​of all pixels to obtain the population density estimate output of the image to be analyzed.

[0068] Compared with the prior art, the beneficial effects of the present invention are:

[0069] This method effectively captures multi-scale spatial features of crowd images through a multi-level parallel dilated convolutional structure, and achieves dual enhancement of key information by combining a spatial-channel joint attention mechanism. Through an importance score weighted fusion strategy, it adaptively integrates the contributions of branches at different scales, significantly improving the model's ability to perceive and represent complex crowd distributions. This method can not only generate high-resolution density maps with richer details and clearer boundaries, but also significantly improve the accuracy and robustness of crowd counting in sparse, dense, and mixed scenarios while maintaining low computational overhead, providing more reliable technical support for practical applications such as public security and intelligent transportation. Attached Figure Description

[0070] Figure 1This is a schematic diagram of the overall method flow of the present invention;

[0071] Figure 2 A graph showing the relationship between the Pearson correlation coefficient and the importance score;

[0072] Figure 3 This is a block diagram of the overall system structure of the present invention. Detailed Implementation

[0073] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to specific embodiments.

[0074] It should be noted that, unless otherwise defined, the technical or scientific terms used in this invention should have the ordinary meaning understood by one of ordinary skill in the art to which this invention pertains. The terms "first," "second," and similar terms used in this invention do not indicate any order, quantity, or importance, but are merely used to distinguish different components. Terms such as "comprising" or "including" mean that the element or object preceding the word encompasses the elements or objects listed following the word and their equivalents, without excluding other elements or objects. Terms such as "connected" or "linked" are not limited to physical or mechanical connections, but can include electrical connections, whether direct or indirect. Terms such as "upper," "lower," "left," and "right" are used only to indicate relative positional relationships; when the absolute position of the described object changes, the relative positional relationship may also change accordingly.

[0075] Example:

[0076] Please see Figures 1-2 The present invention provides a technical solution:

[0077] The crowd density estimation method based on feature-aware weighted contrastive learning includes the following steps:

[0078] S1: The image to be analyzed is standardized and then input into a pre-trained feature extraction network. The feature extraction network generates a basic feature map corresponding to the image to be analyzed by analyzing spatial features and channel features.

[0079] The specific steps to obtain the basic feature map are as follows:

[0080] The original image of the crowd is obtained, and the pixel values ​​of the original image are decomposed according to the three color channels of red, green and blue. The pixel value of each channel is obtained. The pixel value of each channel is subtracted from the preset target standard average value and then divided by the preset target standard deviation value to obtain the standardized image. The standardized image is input into the feature extraction network, which is a pre-trained ResNet-50 network. The feature map output of the last convolutional layer of the ResNet-50 network is extracted as the base feature map.

[0081] The process of training the ResNet-50 network is as follows: Obtain the ImageNet dataset, set the training epochs, use each image in the ImageNet dataset as the input to the ResNet-50 network, and use the corresponding category of each image as the output label to train the ResNet-50 network. During training, the cross-entropy loss function is used to optimize the network parameters through backpropagation. When the cross-entropy loss function value fluctuates less than a preset threshold over several consecutive training epochs, it indicates that the initial training is complete. Remove the classification output layer of the initially trained ResNet-50 network, and retain the convolutional feature extraction layer as the backbone network to obtain the trained ResNet-50 network.

[0082] In the above process, ResNet-50 was chosen as the feature extraction network because its deep convolutional layers have powerful high-level feature extraction capabilities. The feature maps output by the final convolutional layer retain the structural information of the original image in the spatial dimension, i.e., "where it is," while encoding rich semantic information in the channel dimension, i.e., "what it is." For the crowd density estimation task, this semantic information includes abstract features highly correlated with the human body, head, pose, and clustering patterns. Compared with low-level features such as edges and textures extracted by the shallow layers of the network, deep features are more robust to changes in scale, illumination, and occlusion, and are better able to capture the essential attributes of the crowd.

[0083] The ResNet-50 network architecture consists of an initial convolutional layer, four consecutive residual stages, and a final classification layer. The network begins with a... Convolutional and max-pooling layers are used for fast downsampling and extraction of primary features; then comes the main body, which consists of stacked residual blocks, each of which is processed by... Convolution dimensionality reduction, Convolution extracts features, then... The sequence of convolutions is increased in dimensionality and then supplemented with skip connections to directly add the input to the output. The four residual stages contain 3, 4, 6 and 3 bottleneck blocks respectively, gradually increasing the number of channels and reducing the spatial resolution. Finally, the class probability is output through a global average pooling layer and a fully connected classification layer.

[0084] The ImageNet dataset is a massive visual database containing over 14 million images covering 1,000 object categories. The core purpose of pre-training with it is to enable the network to learn general, transferable visual feature representations. By learning to recognize various objects on such diverse data, ResNet-50 can master hierarchical feature extraction capabilities, ranging from simple edges and textures to complex object parts. This capability provides a powerful feature prior for specific tasks such as crowd density estimation, essentially allowing the network to grasp general visual rules. This is far more efficient and avoids overfitting than training from scratch on limited crowd data, significantly improving the model's generalization ability.

[0085] During training, the cross-entropy loss function is used to optimize the network parameters through backpropagation, specifically as follows:

[0086] For a containing The batch of training samples is denoted as ResNet-50 for the th training sample. The predicted class probability distribution for each sample is as follows: ,in This represents the total number of categories in ImageNet, and The true class labels of these samples are typically represented as a one-hot encoded vector. Among them, only the real category index is 1, and the rest are 0;

[0087] The standardized image is input into the feature extraction network, and after convolution, batch normalization, ReLU, and residual connection operations, the probability distribution is finally output through the softmax function.

[0088]

[0089] Among them, This represents the output of the last fully connected layer; Indicates the first The probability of samples of each category;

[0090] Define the cross-entropy loss function as:

[0091]

[0092] in, This represents the value of the cross-entropy loss function; Indicates the number of sample indices; Indicates the number of category indexes;

[0093] Calculate the loss pair The gradient is used to calculate the error signal of the output layer.

[0094]

[0095] Indicates loss to The gradient value, i.e., the error signal of the output layer;

[0096] The error signal starts from the top layer and passes through fully connected layers, residual blocks, batch normalization (BN) layers, convolutional layers, etc. Each layer calculates its gradient with respect to the input according to the chain rule, while simultaneously updating the scaling and offset parameters. The optimizer Adam aggregates the gradients of all parameters.

[0097] The optimizer Adam aggregates the gradients of all parameters through an iterative update mechanism based on adaptive moment estimation. After obtaining the gradient of each parameter calculated by backpropagation, the Adam algorithm maintains two state variables for each parameter: a first-order moment estimate and a second-order moment estimate. The algorithm first uses a decay coefficient to perform a moving average on the current gradient and updates these two moment estimates respectively. Then, it corrects the bias caused by zero initialization in the initial training phase. Finally, it calculates the adaptive learning step size for each parameter based on the corrected moment estimates and updates the parameter values ​​with this step size. This process allows different parameters to obtain differentiated update magnitudes based on their gradient history information. Parameters with small and stable gradients receive larger updates, while updates for parameters with large and oscillating gradients are suppressed, thereby achieving efficient and stable network optimization as a whole.

[0098] Repeat the above steps until the loss no longer decreases significantly over multiple consecutive validation periods on the validation set, indicating that training is complete.

[0099] The initial training of the ResNet-50 network involves removing the classification output layer and retaining the convolutional feature extraction layer as the backbone. This results in a fully trained ResNet-50 network because the original ResNet-50's terminal structure, including global average pooling layers and fully connected classification layers, was specifically designed for the 1000-class classification task on ImageNet. When the network is adapted for a regression task like crowd density estimation, its output target changes from discrete class labels to a continuous density map, rendering the original classification output layer inapplicable. Removing these classification-specific top layers is equivalent to stripping away the task-specific "shell," while retaining the convolutional feature extraction layer—the backbone—preserves the general feature extraction "skeleton" learned on ImageNet.

[0100] S2: Input the basic feature map into a multi-level parallel dilated convolutional layer. Each parallel branch uses dilated convolutional kernels of different scales to perform convolution operations to generate branch feature maps. Pooling operations are performed on the branch feature maps output by each branch.

[0101] The specific steps to obtain the branch feature map of the output of each branch are as follows:

[0102] The basic feature map is subjected to a hole rate of 2. The convolution kernel operation involves standardizing the features of each channel to zero mean and unit variance through a batch normalization layer, and then scaling the batch normalized pixels using the ReLU activation function to obtain the first branch feature map.

[0103] The basic feature map is subjected to a hole rate of 4. The convolution kernel operation involves standardizing the features of each channel to zero mean and unit variance through a batch normalization layer. The pixels after batch normalization are then scaled using the ReLU activation function to obtain the second branch feature map.

[0104] The basic feature map is subjected to a hole rate of 6. The convolution kernel operation involves standardizing the features of each channel to zero mean and unit variance through a batch normalization layer. The batch normalized pixels are then scaled using the ReLU activation function to obtain the third branch feature map.

[0105] In the above process, the multi-level parallel dilated convolution contains at least three branches. The first branch has a relatively small effective receptive field and is mainly responsible for capturing local details and fine-grained features. This branch is more sensitive to local structures such as sparsely distributed individuals and edge contours in the image and can provide key information for clearly distinguishing neighboring individuals. The second branch increases the dilation rate. This branch obtains a medium-range receptive field without increasing parameters or downsampling, which enables it to perceive local crowd clusters or medium-sized aggregation patterns and understand medium-distance relationships between individuals. The third branch combines a larger convolutional kernel and a high dilation rate to obtain the largest effective receptive field. This branch focuses on extracting global context and macro-layout information and can perceive large-scale crowd distribution trends, background scene structures, and long-distance relationships between different dense areas in the image.

[0106] The advantage of this multi-level parallel dilated convolutional layer is that, through parallel specialized processing paths, the network can simultaneously "see" local, mesoscopic, and global information, thus adapting to various scenes from sparse to extremely dense without pre-setting or assuming the scale of the crowd. The features extracted by different branches are naturally complementary: local details help with accurate localization and segmentation, while global context helps understand the overall distribution and suppress background interference. The subsequent attention mechanism and weighted fusion steps are precisely for intelligently integrating this complementary information. The introduction of dilated convolution allows expanding the receptive field without spatial downsampling or stacking too many layers, achieving multi-scale feature extraction at a low computational cost while maintaining the spatial resolution of the feature map.

[0107] The first and second branches use 3×3 convolutional kernels. 3×3 is a standard and efficient module in deep convolutional networks, such as VGG and ResNet. The first two branches focus on local and mesoscale, and 3×3 kernels are sufficient to capture the required patterns with low computational burden. The third branch uses 5×5 convolutional kernels to maintain sufficient spatial sampling density under high dilatation rate. 5×5 kernels provide more sampling points under the same dilatation rate, making feature extraction more robust, especially beneficial for capturing large-scale but structurally complex population distributions.

[0108] The first branch porosity is set to 2. For a 3×3 convolution kernel, the effective receptive field size is... This is equivalent to the receptive field of a standard 5×5 convolution. This setting can capture local details such as the head and shoulders of an individual, as well as the tiny gaps between adjacent individuals; the second branch's void ratio is set to 4, and the actual effective receptive field size is... This is equivalent to a receptive field of a 9×9 convolution, which can cover the area of ​​a small population cluster; the third branch porosity is set to 6, resulting in an actual effective receptive field size of [size missing]. This is equivalent to a receptive field of a 25×25 convolution, which can perceive the macroscopic distribution trend of crowds, background structure, and the relationship between different dense areas in the entire image. It makes a significant contribution to the overall estimation of highly dense or distant crowds.

[0109] S3: After processing the pooled branch feature maps through the attention mechanism, enhanced feature maps are generated, and the corresponding importance scores are calculated based on the response intensity of each enhanced feature map; the importance scores of all branches are normalized to obtain the fusion weight coefficients, and the fusion weight coefficients are weighted and summed with the corresponding enhanced feature maps to obtain the fusion feature map corresponding to the image to be analyzed;

[0110] The branch feature maps are processed using an attention mechanism, and the specific steps are as follows:

[0111] Global average pooling and global max pooling are performed on the feature maps output by each branch to obtain the corresponding feature maps. The two feature maps are then sequentially input into a multilayer perceptron with shared weights for processing. The multilayer perceptron consists of two fully connected layers with a non-linear transformation through the ReLU activation function in between. The feature maps processed by the multilayer perceptron are summed element by element, and the sum is normalized to between 0 and 1 by the Sigmoid function to generate a channel attention weight vector representing the feature map of each branch.

[0112] Simultaneously, the pooled feature maps are concatenated along the channel dimension, and the concatenated feature maps are then processed using dilatation rates of 1 and 3, respectively. Convolutional layers are used, and batch normalization is performed on each layer to obtain local detail feature maps and context detail feature maps, respectively.

[0113] The local detail feature map and the context detail feature map are weighted and fused using a preset learnable scalar parameter of 0.5. Specifically, the learnable scalar parameter of 0.5 is multiplied by each pixel of the local detail feature map, and then 1 is subtracted from the learnable scalar parameter of 0.5 multiplied by each pixel of the context detail feature map. The two are then added together to obtain the weighted fused feature map.

[0114] The weighted fused feature map is input into a feature channel that is compressed to 1. Convolutional layers are used, and the results are normalized using the Sigmoid function to obtain the spatial attention weight matrix for each branch feature map.

[0115] In the above process, the generation of channel attention weight vector aims to achieve intelligent filtering and enhancement at the feature channel level. Its technical effect is that by integrating the comprehensive information obtained from global average pooling and global max pooling, the network can dynamically evaluate the importance of each feature channel for the crowd density estimation task. The vector assigns a weight between 0 and 1 to each channel, thereby automatically suppressing redundant or interfering channels, while enhancing the channel response related to key semantics such as human contour, head features, and aggregation patterns, guiding the network to focus on the most discriminative feature dimensions, and improving the semantic purity and expression efficiency of features.

[0116] The generation of the spatial attention weight matrix focuses on adaptive focusing in the spatial dimension of the feature map. Its technical effect is that by fusing local details and contextual information and compressing them to generate a single-channel weight map, the matrix can accurately identify densely populated areas, individual boundaries, and key spatial locations in the image. It guides the network to allocate differentiated computing resources in different spatial locations, strengthens feature responses in areas where people gather, and suppresses irrelevant activations in background or non-person areas. This enables the modeling of the spatial non-uniformity of the crowd distribution, enhances the model's spatial perception and localization capabilities in complex scenes, and provides key spatial guidance information for generating density maps with clear boundaries and rich details.

[0117] The initial value of the learnable scalar parameter is preset to 0.5, which is based on the design idea of ​​balanced initialization. This setting avoids the problem of unstable optimization path or slow convergence that may be caused by parameter randomization in the early stage of training. 0.5 means that in the initial state, local detail features and context detail features are given completely equal contribution weights. This provides the model with a neutral and unbiased starting point, ensuring that the two types of information can participate equally in forward propagation and gradient calculation in the early stage of training.

[0118] The technical advantage of weighted fusion feature maps lies in achieving adaptive and flexible fusion of local detail information and global contextual information. By dynamically adjusting the contribution of the two through a single learnable scalar parameter, the model can automatically adjust the fusion strategy for different population distribution patterns, such as relying more on local details to distinguish individuals in sparse scenes and relying more on context to understand the overall distribution in dense scenes. This design not only enhances the flexibility and scene adaptability of feature expression, but also achieves information complementarity with extremely low parameter cost, ultimately improving the discrimination accuracy and robustness of the spatial attention weight matrix, enabling it to more accurately focus on the spatial regions that are crucial for density estimation.

[0119] The weighted fused feature map is input into a feature channel that is compressed to 1. A convolutional layer is used, and the result is normalized using the Sigmoid function to obtain the spatial attention weight matrix for each branch feature map. The core purpose of this is to generate a single-channel attention weight map that can accurately indicate the importance of each spatial location in the image and has spatial adaptability. By using a 1×1 convolutional layer to compress the weighted and fused multi-channel feature map into a single channel, the spatial activation information of all channels can be effectively aggregated and fused into a comprehensive spatial saliency representation. Then, it is normalized to the 0 to 1 interval by the Sigmoid function to form the final spatial attention weight matrix. The effect of this technique is that it can dynamically evaluate and highlight spatial regions that are crucial to crowd density estimation, such as crowd gathering centers and individual boundaries, based on the fused local and contextual features, while suppressing background or irrelevant regions, thereby providing accurate spatial guidance for subsequent feature enhancement. A 1×1 convolutional layer with the number of channels compressed to 1 is used because it can achieve the fusion and dimensionality reduction of cross-channel information with minimal parameter cost. While maintaining computational efficiency, it ensures that the output attention map contains only one scalar weight at each spatial location, which is convenient for element-wise attention weighting operations with the original feature map.

[0120] The specific steps for generating enhanced feature maps are as follows:

[0121] Based on the feature map output from each branch, attention enhancement is achieved by expanding the channel attention weight vector and the spatial attention weight matrix through an outer product and then multiplying them element-wise.

[0122]

[0123] in, Indicates the first Enhanced feature maps of each branch; Indicates the first Each branch feature map; This indicates the outer product expansion operation; This represents the element-wise multiplication method; Indicates the first Channel attention weight vectors of each branch feature map; Represents the branch number index, and ; Indicates the first Spatial attention weight matrix for each branch feature map.

[0124] In the above process, the one-dimensional channel attention weight vector and the two-dimensional spatial attention weight matrix are fused into a three-dimensional joint attention tensor through outer product expansion. This joint attention tensor is then multiplied element-wise with the original branch feature map. The technical effect is to achieve synergistic attention enhancement of the channel and spatial dimensions. This operation not only selects and strengthens key feature channels related to the population based on channel importance, but also performs spatial selective focusing on the feature map based on the importance of spatial location. This allows the enhanced feature map to simultaneously highlight feature dimensions and spatial regions valuable to the task, thereby significantly improving the discriminativeness and specificity of feature representation.

[0125] The specific steps for calculating the importance score of the enhanced feature map for each branch are as follows:

[0126] Calculate the Pearson correlation coefficient between the spatial attention weights and channel attention weights of each branch feature map:

[0127]

[0128] in, Indicates the first Pearson correlation coefficients of branch feature maps; Indicates vectorization operation; Indicates the calculation of covariance; This represents the standard deviation of the channel attention weights for all branches; The standard deviation of the spatial attention mean across all branches; This represents the mean of the spatial attention weight matrix;

[0129] In the above process, by calculating the Pearson correlation coefficient between the channel attention weight vector and the spatial attention weight mean vector, the technical effect is to quantitatively evaluate the degree of synergy and consistency between the channel selection mechanism and the spatial focusing mechanism within each branch. This indicator can reflect whether the feature enhancement of the branch has a clear and consistent goal orientation: if the correlation coefficient is high, it indicates that the features that are activated in the channel are highly correlated with the areas that are focused in the space, which means that the attention mechanism of the branch is synergistically efficient, and the extracted features have strong discriminative power and low information redundancy; otherwise, it indicates that the attention effect is scattered or uncoordinated.

[0130] Vectorization refers to the process of rearranging an input multidimensional tensor into a one-dimensional column vector. The vectorization operation represents the mean of the spatial attention weight matrix. Essentially, it constructs a scalar value where all elements are this scalar value, not a simple average of the entire matrix. Instead, it represents the arithmetic mean of the weights at all spatial locations within the spatial attention weight matrix. It is a single scalar, and the vectorization operation ensures that... and The vectorized forms of these have the same dimension, thus enabling statistical operations such as covariance.

[0131] Calculate the energy focusing index for each branch feature map:

[0132]

[0133] in, Represents the norm; Indicates the first Energy focusing index of branch feature maps; Indicates the height of the enhanced feature map; Indicates the width of the enhanced feature map; Represents the pixel-level index of the enhanced feature map; The vertical index of the pixel in the enhanced feature map; Indicates the first The enhanced feature maps of each branch at each pixel The value at;

[0134] In the above process, the energy focusing index is defined by calculating the ratio of the maximum local energy to the global average energy of the feature map. The technical effect is to quantitatively evaluate the concentration of information and the significance of activation in the feature map of each branch. If the index value is high, it indicates that the feature energy is highly concentrated in a few significant regions, reflecting that the feature response of the branch has a clear focus and strong discriminativeness, which may correspond to the dense core of the population or key individuals. Conversely, it means that the feature activation is relatively uniform and dispersed, which may lack a clear focusing target or contain more flat background responses.

[0135] Calculate the channel activation index for each branch feature map:

[0136]

[0137] in,

[0138]

[0139] in, Indicates the first The entropy of the channel attention weights of each branch feature map; Indicates the total number of channels; Indicates the number of channel indices; Indicates the first The branch feature map has a number of channels. Channel attention weights; Indicates the first Channel activation index of each branch feature map;

[0140] In the above process, by calculating the channel activation index based on information entropy, the technical effect is to quantitatively evaluate the distribution specificity and selection sharpness of the channel attention weights of each branch. The smaller the entropy value of the channel attention weight, the more unbalanced the weight distribution is and the more concentrated it is in a few channels. This means that the channel attention mechanism of this branch has a high degree of selectivity and can clearly identify and strongly activate the key feature channels most relevant to the task. The closer the activation index obtained by subtracting the normalized entropy value from 1 is to 1, the stronger the channel selection ability and the lower the feature redundancy of this branch.

[0141] The importance score of the enhanced feature map for each branch is generated by geometrically averaging the Pearson correlation coefficient, energy focusing index, and channel activation index of each branch feature map.

[0142]

[0143] in, Indicates the first The importance score of each branch enhances the feature map.

[0144] In the above process, the final importance score is synthesized by geometrically averaging the Pearson correlation coefficient, energy focusing index, and channel activation index. The technical effect is to construct a multi-dimensional, balanced, and rigorous comprehensive evaluation system for branch feature quality. The geometric average requires that all three indicators perform well in order to obtain a high score. This forces the importance assessment to simultaneously meet three key conditions: high attentional synergy, strong spatial energy focusing, and good channel selection specificity. This ensures that the selected branch is not only outstanding in a single dimension, but also achieves optimal performance in channel and spatial synergy, energy distribution, and the sharpness of feature selection.

[0145] Indicates the first The importance score of each branch-enhanced feature map directly reflects the overall contribution of that branch to the density estimation task. It is determined by three independent variables: the Pearson correlation coefficient, the energy focusing index, and the channel activation index. A low value in any of these independent variables will significantly lower the overall density estimation score. Therefore, only branches that perform well in all three dimensions can obtain high importance scores;

[0146] In the above embodiments, 20 sets of data on Pearson correlation coefficient and importance score are given to reflect the change of importance score with the change of Pearson correlation coefficient, as shown in Table 1:

[0147] Table 1: Relationship between Pearson correlation coefficient and importance score

[0148]

[0149] As can be seen from Table 1 above, in , In the case of a Pearson correlation coefficient, the higher the importance score, and vice versa.

[0150] The specific steps to obtain the fused feature map that integrates information from each branch are as follows:

[0151] The importance scores of all branches are normalized to obtain the fusion weight coefficient:

[0152]

[0153] in, Indicates the first The fusion weight coefficient of each branch;

[0154] In the above process, the importance scores of each branch are normalized by using the Softmax function. The technical effect is to transform the importance scores into a fusion weight coefficient in the form of a probability distribution. This operation ensures that the sum of all weight coefficients is 1. This not only realizes the quantitative allocation of the contribution of each branch, but also amplifies the advantage of high importance scores through exponential operation, so that the better performing branches get significantly higher weights in the fusion, while suppressing the influence of poorly performing branches.

[0155] The fusion weight coefficients are weighted and summed with the corresponding enhanced feature maps to obtain a fusion feature map that integrates information from each branch:

[0156]

[0157] in, Represents the fused feature map; express The number of channels is uniform for convolution operations.

[0158] In the above process, the formula in This represents a 1×1 convolution operation with a uniform number of channels. Specifically, it means that before weighted summation, the enhanced feature maps of each branch are first processed. Performing a 1×1 convolution aims to adjust or unify the number of channels in each branch feature map to the same target dimension, ensuring they are aligned in the channels so that they can be directly weighted and summed channel by channel. By weighting and summing the normalized fusion weight coefficients with the enhanced feature maps of each branch, the technical effect is to achieve adaptive and focused integration of multi-scale features. It dynamically allocates the contribution of each branch according to its importance, so that the fused feature map can retain key information at different scales while highlighting the most discriminative feature expressions, thus forming a unified feature representation that is complementary and emphasizes key points, providing high-quality input for subsequent density map regression.

[0159] S4: Input the fused feature map into the pre-trained regression network to generate a low-resolution prediction density map. Upsample the low-resolution prediction density map to obtain a high-resolution density map. Input the low-resolution prediction density map into a lightweight network to generate a scaling mask. Multiply the high-resolution density map and the scaling mask element-wise to obtain the prediction density map.

[0160] The specific steps to generate the predicted density map are as follows:

[0161] The fused feature map is input into a pre-trained regression network to generate a low-resolution predicted density map. The pre-training of the regression network involves constructing a training dataset containing a large number of crowd images and their corresponding ground truth density maps, where each ground truth density map is generated by Gaussian kernel convolution using manually labeled head center points. Then, the fused feature map is used as input, and the predicted low-resolution density map is used as the output of the regression network. During training, the mean squared error is used to calculate the difference between the low-resolution predicted density map and the corresponding ground truth low-resolution predicted density map. All parameters of the regression network are iteratively updated using the backpropagation algorithm and gradient descent optimizer until the loss function converges, thus completing the training.

[0162] In the above process, the training dataset is constructed. ,in, Indicates the first Fusion feature map of a group of images. Indicates the corresponding first Zhang's true low-resolution density map This represents the total number of training samples. The fused feature map is input into the regression network, which is usually composed of stacked convolutional layers, batch normalization layers, and activation functions such as ReLU. Finally, a 1×1 convolutional layer is used to compress the number of channels to 1, and the output of the network is the predicted low-resolution density map.

[0163] The mean squared error loss function is used to measure the difference between the predicted density map and the true density map. For a single training sample, the loss is calculated as follows:

[0164]

[0165] in, and These represent the height and width of the density map, respectively. Indicates the first Zhang's fusion feature map is input into the regression network to generate a low-resolution density map of the prediction; This represents the total number of pixels in the density map; Indicates the index of the number of training samples;

[0166] The gradient descent optimizer is used to minimize the overall loss. In each iteration, the gradient of the loss function with respect to the network parameters is calculated:

[0167]

[0168] Indicates the first The gradient of the loss function with respect to the network parameters

[0169] The training dataset is iterated through multiple cycles until the loss function value stabilizes on the validation set and no longer decreases significantly, or until the preset number of training cycles is reached. At this point, the regression network is considered to have been trained and can accurately regress the population density distribution from the fused feature map.

[0170] A high-resolution density map is obtained by bilinear upsampling of the low-resolution predicted density map, specifically as follows:

[0171] The low-resolution predicted density map is upsampled by 32 times using bilinear interpolation to restore its spatial size to be consistent with the original input image, thus obtaining a preliminary high-resolution density map.

[0172] Meanwhile, the low-resolution predicted density map is input into a lightweight convolutional network to generate a scaling mask. Specifically, the same low-resolution predicted density map is input into a lightweight convolutional network, which consists of a single 3×3 convolutional layer and a Sigmoid activation function. The local context information of the density distribution is extracted through the convolutional layer, and a spatially adaptive scaling mask with values ​​between 0 and 1 is generated.

[0173] Finally, the high-resolution density map is multiplied element-wise with the scaling mask to generate the predicted density map.

[0174] In the above process, firstly, a low-resolution predicted density map is generated using a pre-trained regression network to initially capture the global distribution and density intensity of the crowd. Subsequently, bilinear upsampling quickly recovers the spatial size, but may introduce blurring and loss of detail. Therefore, a spatially adaptive scaling mask is learned in parallel from the same low-resolution predicted density map through a lightweight convolutional network. Finally, the upsampling result is multiplied element-wise with the mask, and the upsampling density value is locally calibrated and finely adjusted using the mask. This key operation effectively recovers and enhances the detail representation of crowd edges, sparse regions, etc., and significantly improves the spatial accuracy and visual quality of the final predicted density map.

[0175] The low-resolution predicted density map is upsampled by 32x using bilinear interpolation to restore its spatial size to match that of the original input image, thus obtaining a preliminary high-resolution density map. The purpose of this is that during the forward propagation of the feature extraction network, the image undergoes multiple downsampling operations such as pooling or stride convolution, resulting in the spatial size of the final low-resolution predicted density map being much smaller than that of the original input image. In order to obtain a final density map that corresponds one-to-one with the pixels of the input image and can be used for accurate counting, the spatial size of the low-resolution predicted density map must be restored to its original size. Using 32x bilinear interpolation upsampling is a computationally efficient and smooth method that can quickly expand each "density unit" in the low-resolution map to the corresponding image region while maintaining the overall trend of the density distribution.

[0176] The lightweight convolutional network is designed to achieve spatially adaptive detail calibration with minimal computational overhead. A single 3×3 convolutional layer can effectively capture the dependency between each location in the low-resolution predicted density map and its local neighborhood, i.e., contextual information, and learn the density distribution errors or blurring patterns that may be caused by upsampling. Then, the convolutional output is normalized to between 0 and 1 through the Sigmoid function to form a spatially variable scaling mask. This mask can adaptively enhance or suppress the density values ​​of different regions after upsampling based on local contextual information, thereby restoring and sharpening details, making up for the shortcomings of simple bilinear upsampling. Moreover, the entire module has very few parameters, ensuring the lightweight and efficient generation process.

[0177] S5: Apply nonnegativity constraints to the generated predicted density map and sum the crowd density values ​​of all pixels to obtain the estimated crowd density output in the image to be analyzed.

[0178] To obtain the total number of people in the entire image, the specific steps are as follows:

[0179] Apply a non-negativity constraint to the predicted density map to ensure that the population density value of each pixel is not less than zero:

[0180]

[0181] in, Indicates the non-negative constraint coefficient; Represents pixels The predicted density value at that location; Represents pixels Population density value at the location; Represents the predicted density map; This represents the number of horizontal pixel indices in the predicted density map; The number of vertically flat pixel indices in the predicted density map;

[0182] In the above process, applying a non-negativity constraint to the predicted density map is to ensure the physical rationality and numerical stability of the density estimation. Since the density values ​​predicted by the network may contain small negative values ​​or floating-point errors, while the actual population density cannot be negative, direct summation would lead to counting bias. Introducing a non-negativity constraint coefficient... That is, it is usually a very small positive number, such as 1e-6, and by applying the ReLU function, all predicted values ​​can be clipped to not less than zero. This eliminates non-physical negative value interference and avoids the gradient vanishing problem that may be caused by simply setting it to zero. This ensures that the total number of people obtained by summing all pixel density values ​​is a mathematically rigorous and reliable estimate that is consistent with reality.

[0183] The total number of people in the entire image is obtained by summing the crowd density values ​​of all pixels:

[0184]

[0185] in, Indicates the width of the predicted density map; Indicates the height of the predicted density map; Indicates the total number of people.

[0186] The reason for summing the crowd density values ​​of all pixels in the above process to obtain the total number of people in the entire image is as follows: The core task of crowd density estimation is to transform the discrete individuals in the image into a continuous density distribution representation. The value of each pixel on the predicted density map represents the crowd density of that local area, that is, the number of people per unit area. By summing the values ​​of all pixels in the entire density map, we are essentially integrating over the entire image region. This mathematical operation can accumulate the density values ​​that are continuously distributed in space, thereby accurately restoring the total number of discrete individuals in the image, and finally achieving a seamless conversion from pixel-level density prediction to image-level total number of people.

[0187] Please see Figure 3 The present invention also provides a crowd density estimation system based on feature-aware weighted contrastive learning, the estimation system being used to perform the above-described estimation method, comprising:

[0188] The image acquisition module is used to acquire the original image of the crowd and perform standardization processing. The processed image is input into the input feature extraction network, and the output of the last convolutional layer is extracted as the basic feature map. The basic feature map contains preliminary spatial and channel features related to the crowd.

[0189] The image acquisition module is used to input the image to be analyzed into a pre-trained feature extraction network after standardization. The feature extraction network generates a basic feature map corresponding to the image to be analyzed by analyzing spatial features and channel features.

[0190] The pooling module is used to input the basic feature map into a multi-level parallel dilated convolutional layer. Each parallel branch uses dilated convolutional kernels of different scales to perform convolution operations to generate branch feature maps, and pooling operations are performed on the branch feature maps output by each branch.

[0191] The fusion feature acquisition module is used to process the pooled branch feature maps through an attention mechanism to generate enhanced feature maps, and calculate the corresponding importance score based on the response intensity of each enhanced feature map; normalize the importance scores of all branches to obtain fusion weight coefficients, and then perform a weighted summation of the fusion weight coefficients and the corresponding enhanced feature maps to obtain the fusion feature map corresponding to the image to be analyzed;

[0192] The prediction density acquisition module is used to input the fused feature map into the pre-trained regression network to generate a low-resolution prediction density map, perform upsampling on the low-resolution prediction density map to obtain a high-resolution density map, input the low-resolution prediction density map into a lightweight network to generate a scaling mask, and multiply the high-resolution density map and the scaling mask element-wise to obtain the prediction density map.

[0193] The population statistics module is used to apply non-negativity constraints to the generated predicted density map and sum the population density values ​​of all pixels to obtain the population density estimate output of the image to be analyzed.

[0194] The above formulas are all dimensionless calculations. The formulas are derived from software simulations based on a large amount of collected data to obtain the most recent real-world results. The preset parameters in the formulas are set by those skilled in the art according to the actual situation.

[0195] The above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any other combination thereof. When implemented in software, the above embodiments can be implemented, in whole or in part, as a computer program product. Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution.

[0196] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0197] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

Claims

1. A crowd density estimation method based on feature-aware weighted contrastive learning, characterized by the following steps: include: S1: The image to be analyzed is standardized and then input into a pre-trained feature extraction network. The feature extraction network generates a basic feature map corresponding to the image to be analyzed by analyzing spatial features and channel features. S2: Input the basic feature map into a multi-level parallel dilated convolutional layer. Each parallel branch uses dilated convolutional kernels of different scales to perform convolution operations to generate branch feature maps. Pooling operations are performed on the branch feature maps output by each branch. S3: After processing the pooled branch feature maps through the attention mechanism, enhanced feature maps are generated, and the corresponding importance scores are calculated based on the response intensity of each enhanced feature map; The importance scores of all branches are normalized to obtain the fusion weight coefficients, and the fusion weight coefficients are weighted and summed with the corresponding enhanced feature maps to obtain the fusion feature map corresponding to the image to be analyzed. S4: Input the fused feature map into the pre-trained regression network to generate a low-resolution prediction density map. Upsample the low-resolution prediction density map to obtain a high-resolution density map. Input the low-resolution prediction density map into a lightweight network to generate a scaling mask. Multiply the high-resolution density map and the scaling mask element-wise to obtain the prediction density map. S5: Apply nonnegativity constraints to the generated predicted density map and sum the crowd density values ​​of all pixels to obtain the estimated crowd density output in the image to be analyzed. To obtain the total number of people in the entire image, the specific steps are as follows: Apply a non-negativity constraint to the predicted density map to ensure that the population density value of each pixel is not less than zero: in, Indicates the non-negative constraint coefficient; Represents pixels The predicted density value at the location; Represents pixels Population density value at the location; Represents the predicted density map; This represents the number of horizontal pixel indices in the predicted density map; The number of vertically flat pixel indices in the predicted density map; The total number of people in the entire image is obtained by summing the crowd density values ​​of all pixels: in, Indicates the width of the predicted density map; Indicates the height of the predicted density map; Indicates the total number of people.

2. The crowd density estimation method based on feature-aware weighted contrastive learning according to claim 1, characterized in that, The specific steps to obtain the basic feature map are as follows: The original image of the crowd is obtained, and the pixel values ​​of the original image are decomposed according to the three color channels of red, green and blue. The pixel value of each channel is obtained. The preset target standard average value is subtracted from the pixel value of each channel and then divided by the preset target standard deviation to obtain the standardized image. The standardized image is input into the feature extraction network, which is a pre-trained ResNet-50 network. The feature map output of the last convolutional layer of the ResNet-50 network is extracted as the base feature map.

3. The crowd density estimation method based on feature-aware weighted contrastive learning according to claim 2, characterized in that, The specific steps to obtain the branch feature map of the output of each branch are as follows: The basic feature map is subjected to a hole rate of 2. The convolution kernel operation involves standardizing the features of each channel to zero mean and unit variance through a batch normalization layer, and then scaling the batch normalized pixels using the ReLU activation function to obtain the first branch feature map. The basic feature map is subjected to a hole rate of 4. The convolution kernel operation involves standardizing the features of each channel to zero mean and unit variance through a batch normalization layer. The pixels after batch normalization are then scaled using the ReLU activation function to obtain the second branch feature map. The basic feature map is subjected to a hole rate of 6. The convolution kernel operation involves standardizing the features of each channel to zero mean and unit variance through a batch normalization layer. The batch normalized pixels are then scaled using the ReLU activation function to obtain the third branch feature map.

4. The crowd density estimation method based on feature-aware weighted contrastive learning according to claim 3, characterized in that, The branch feature maps are processed using an attention mechanism, and the specific steps are as follows: Global average pooling and global max pooling are performed on the feature maps output by each branch to obtain the corresponding feature maps. The two feature maps are then sequentially input into a multilayer perceptron with shared weights for processing. The multilayer perceptron consists of two fully connected layers with a non-linear transformation through the ReLU activation function in between. The feature maps processed by the multilayer perceptron are summed element by element, and finally the sum is normalized to between 0 and 1 by the Sigmoid function to generate a channel attention weight vector representing the feature map of each branch. Simultaneously, the pooled feature maps are concatenated along the channel dimension, and the concatenated feature maps are then processed using dilatants with a dilatancy of 1 and a dilatancy of 3, respectively. Convolutional layers are used, and batch normalization is performed on each layer to obtain local detail feature maps and context detail feature maps, respectively. The local detail feature map and the context detail feature map are weighted and fused using a preset learnable scalar parameter of 0.

5. Specifically, the learnable scalar parameter of 0.5 is multiplied by each pixel of the local detail feature map, and then 1 is subtracted from the learnable scalar parameter of 0.5 multiplied by each pixel of the context detail feature map. The two are then added together to obtain the weighted fused feature map. The weighted fused feature map is input into a feature channel that is compressed to 1. Convolutional layers are used, and the results are normalized using the Sigmoid function to obtain the spatial attention weight matrix for each branch feature map.

5. The crowd density estimation method based on feature-aware weighted contrastive learning according to claim 4, characterized in that, The specific steps for generating enhanced feature maps are as follows: Based on the feature map output from each branch, attention enhancement is achieved by expanding the channel attention weight vector and the spatial attention weight matrix through an outer product and then multiplying them element-wise. in, Indicates the first Enhanced feature maps of each branch; Indicates the first Each branch feature map; This indicates the outer product expansion operation; This represents the element-wise multiplication method; Indicates the first Channel attention weight vectors of each branch feature map; Represents the branch number index, and ; Indicates the first Spatial attention weight matrix for each branch feature map.

6. The crowd density estimation method based on feature-aware weighted contrastive learning according to claim 5, characterized in that, The specific steps for calculating the importance score of the enhanced feature map for each branch are as follows: Calculate the Pearson correlation coefficient between the spatial attention weights and channel attention weights of each branch feature map: in, Indicates the first Pearson correlation coefficients of branch feature maps; Indicates vectorization operation; Indicates the calculation of covariance; This represents the standard deviation of the channel attention weights for all branches; The standard deviation of the spatial attention mean across all branches; This represents the mean of the spatial attention weights; Calculate the energy focusing index for each branch feature map: in, Represents the norm; Indicates the first Energy focusing index of branch feature maps; Indicates the height of the enhanced feature map; Indicates the width of the enhanced feature map; Represents the pixel-level index of the enhanced feature map; The vertical index of the pixel in the enhanced feature map; Indicates the first The enhanced feature maps of each branch at each pixel The value at; Calculate the channel activation index for each branch feature map: in, in, Indicates the first The entropy of the channel attention weights of each branch feature map; Indicates the total number of channels; Indicates the number of channel indices; Indicates the first The branch feature map has a channel number of 1 Channel attention weights; Indicates the first Channel activation index of each branch feature map; The importance score of the enhanced feature map for each branch is generated by geometrically averaging the Pearson correlation coefficient, energy focusing index, and channel activation index of each branch feature map. in, Indicates the first The importance score of each branch enhances the feature map.

7. The crowd density estimation method based on feature-aware weighted contrastive learning according to claim 6, characterized in that, The specific steps for generating the fused feature map are as follows: The importance scores of all branches are normalized to obtain the fusion weight coefficient: in, Indicates the first The fusion weight coefficients of each branch; The fusion weight coefficients are weighted and summed with the corresponding enhanced feature maps to obtain the fusion feature map corresponding to the image to be analyzed. in, Represents the fused feature map; express The number of channels is uniform for convolution operations.

8. The crowd density estimation method based on feature-aware weighted contrastive learning according to claim 7, characterized in that, The specific steps to generate the predicted density map are as follows: The fused feature map is input into a pre-trained regression network to generate a low-resolution prediction density map. This low-resolution prediction density map is then bilinearly upsampled to obtain a high-resolution density map. Specifically: The low-resolution predicted density map is upsampled by 32 times using bilinear interpolation to restore its spatial size to be consistent with the original input image, thus obtaining a preliminary high-resolution density map. Meanwhile, the low-resolution predicted density map is input into a lightweight convolutional network to generate a scaling mask. Specifically, the same low-resolution predicted density map is input into a lightweight convolutional network, which consists of a single 3×3 convolutional layer and a Sigmoid activation function. The local context information of the density distribution is extracted through the convolutional layer, and a spatially adaptive scaling mask with values ​​between 0 and 1 is generated. Finally, the high-resolution density map is multiplied element-wise with the scaling mask to generate the predicted density map.

9. A crowd density estimation system based on feature-aware weighted contrastive learning, characterized in that: The estimation system is used to perform the estimation method according to any one of claims 1-8, including: The image acquisition module is used to input the image to be analyzed into a pre-trained feature extraction network after standardization. The feature extraction network generates a basic feature map corresponding to the image to be analyzed by analyzing spatial features and channel features. The pooling module is used to input the basic feature map into a multi-level parallel dilated convolutional layer. Each parallel branch uses dilated convolutional kernels of different scales to perform convolution operations to generate branch feature maps, and pooling operations are performed on the branch feature maps output by each branch. The fusion feature acquisition module is used to process the pooled branch feature maps through an attention mechanism to generate enhanced feature maps, and calculate the corresponding importance score based on the response intensity of each enhanced feature map; normalize the importance scores of all branches to obtain fusion weight coefficients, and then perform a weighted summation of the fusion weight coefficients and the corresponding enhanced feature maps to obtain the fusion feature map corresponding to the image to be analyzed; The prediction density acquisition module is used to input the fused feature map into the pre-trained regression network to generate a low-resolution prediction density map, perform upsampling on the low-resolution prediction density map to obtain a high-resolution density map, input the low-resolution prediction density map into a lightweight network to generate a scaling mask, and multiply the high-resolution density map and the scaling mask element-wise to obtain the prediction density map. The population statistics module is used to apply non-negativity constraints to the generated predicted density map and sum the population density values ​​of all pixels to obtain the population density estimate output of the image to be analyzed.

Images