Crowd counting method based on adaptive global perception and multi-scale feature fusion

Abstract

The present invention belongs to the field of computer vision technology, and discloses a crowd counting method based on adaptive global perception and multi-scale feature fusion. The crowd counting model constructed in the method includes an adaptive global perception module AGP, a multi-scale Mamba aggregation module, and a channel perception Mamba decoder CMD. AGP realizes efficient extraction of global information of the image and improves the reasoning speed; the multi-scale Mamba aggregation module effectively integrates features of different scales, enhances the ability to capture details and global features, and improves the accuracy of crowd counting; CMD strengthens the correlation of information between channels and further improves the counting accuracy of the model; the composite loss function ensures the rapid convergence and high-precision performance of the network in the crowd counting task. While ensuring the model reasoning speed and counting accuracy, the method of the present invention significantly reduces the consumption of computing resources and realizes the lightweight of the model.

Description

Technical Field

[0001] The present invention belongs to the field of computer vision and deep learning technology, and specifically relates to a crowd counting method based on adaptive global perception and multi-scale feature fusion, which is particularly suitable for application scenarios such as security monitoring, urban management, and crowd monitoring in public places, which require estimating the number of people in an image or video frame. Background Art

[0002] Crowd counting, as a technology for counting and estimating the density of people in images or videos, has become an important means to ensure public safety, optimize traffic flow management, and improve urban management efficiency.

[0003] At present, crowd counting methods are mainly based on technological developments in the field of computer vision and are divided into two categories: those based on Convolutional Neural Networks (CNN) and those based on Transformer.

[0004] The CNN-based method extracts local features of the image and generates a density map to achieve crowd counting. It is well suited for dense and multi-scale crowd scenes, but its ability to capture global context information is weak and it is easily affected by perspective distortion and background interference, resulting in large counting errors. The Transformer-based method captures long-range dependencies in the image through the self-attention mechanism, performs well in global feature modeling, and can effectively process crowd images with complex backgrounds and perspective changes. However, the computational complexity of the Transformer's self-attention mechanism is as high as O(N²), which makes it require large computing resources and slow inference speed when processing high-resolution images, making it difficult to meet the real-time requirements of practical applications.

[0005] With the development of state-space models in visual tasks, some researchers have tried to apply the selective scanning mechanism to crowd counting tasks, reducing the computational complexity of the self-attention mechanism to linear complexity, and solving the computational efficiency problem of traditional Transformer in crowd counting. However, there are still some problems that need to be solved in the existing methods:

[0006] First, in the modeling of high-level features, detailed information is lacking, resulting in poor counting results in scenes with sparse crowds or large scale changes; second, the information correlation between feature channels is often ignored in existing models, resulting in insufficient utilization of channel information, affecting counting accuracy; third, existing models consume large amounts of computing resources and have a high number of parameters. Although the computational complexity has been reduced, it is still difficult to achieve lightweight while maintaining counting accuracy, and cannot be effectively deployed in resource-constrained mobile or embedded devices, ultimately limiting the performance and wide application of crowd counting methods in practical applications. Summary of the invention

[0007] The purpose of the present invention is to propose a crowd counting method based on adaptive global perception and multi-scale feature fusion. The method introduces an adaptive global perception module, a multi-scale Mamba aggregation module and a channel-aware Mamba decoder into the constructed crowd counting model, aiming to improve the counting accuracy and reasoning speed in crowded scenes, while helping to reduce computing resource consumption and achieve lightweight models, thus having good versatility and deployment potential.

[0008] In order to achieve the above object, the present invention adopts the following technical scheme:

[0009] The crowd counting method based on adaptive global perception and multi-scale feature fusion includes the following steps:

[0010] Step 1. Preprocess the image data to construct a data set for training the crowd counting model based on adaptive global perception and multi-scale feature fusion in step 2 below;

[0011] Step 2. Build a crowd counting model based on adaptive global perception and multi-scale feature fusion, which includes stem module, adaptive global perception module, multi-scale Mamba aggregation module and channel-aware Mamba decoder;

[0012] The processing of the adaptive global perception module is divided into four stages, namely the first, second, third and fourth stages;

[0013] First, the input image is processed by the stem module to segment the input image into patches. Then, the adaptive global perception module is applied to process the features in the first stage to generate initial features with global long-range dependencies. In the second, third, and fourth stages, the adaptive global perception module is cascaded with the downsampling operation to extract multi-scale features with linear complexity.

[0014] Then, the features of different scales output from the second, third and fourth stages are fed into the multi-scale Mamba aggregation module, which effectively integrates information at different levels and outputs the aggregated features. Finally, the aggregated features are fed into the channel-aware Mamba decoder, which enhances the inter-channel information and generates a crowd density map.

[0015] Step 3. Based on the data set constructed in step 1, the crowd counting model constructed in step 2 is trained to obtain a trained crowd counting model, and the trained crowd counting model is used to estimate the number of people.

[0016] The present invention has the following advantages:

[0017] As described above, the present invention relates to a crowd counting method, in which a crowd counting model based on adaptive global perception and multi-scale feature fusion is constructed, which mainly includes an adaptive global perception module (Adaptive Global Perception Module, AGP), a multi-scale Mamba aggregation module and a channel-aware Mamba decoder. Among them, the efficient extraction of image global information is achieved through AGP, the computational complexity of the self-attention mechanism is reduced to linear, and the reasoning speed is improved; the multi-scale Mamba aggregation module is used to effectively fuse features of different scales, enhance the ability to capture details and global features, and improve the accuracy of crowd counting; in addition, the channel-aware Mamba decoder (Channel-Aware Mamba Decoder, CMD) strengthens the correlation between channel information and further improves the counting accuracy of the model; at the same time, the present invention also designs a composite loss function, and the design of the composite loss function ensures the rapid convergence and high-precision performance of the network in the crowd counting task. The method of the present invention significantly reduces the consumption of computing resources while ensuring the model reasoning speed and counting accuracy, realizes the lightweight of the model, has good versatility and deployment potential on mobile or embedded devices, and is particularly suitable for actual scenarios such as public safety, urban management, and monitoring and management of crowded places. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 This is a network structure diagram of a crowd counting model based on adaptive global perception and multi-scale feature fusion in an embodiment of the present invention;

[0019] Figure 2 A network structure diagram of an adaptive global perception module in an embodiment of the present invention;

[0020] Figure 3 This is a network structure diagram of a multi-scale Mamba fusion module in an embodiment of the present invention;

[0021] Figure 4 4 is a network structure diagram of a channel-aware Mamba decoder in an embodiment of the present invention. DETAILED DESCRIPTION

[0022] The present invention is further described in detail below with reference to the accompanying drawings and specific embodiments:

[0023] This embodiment proposes a crowd counting method based on adaptive global perception and multi-scale feature fusion to solve the problems of high computational complexity, large model parameters, insufficient capture of detail features, and insufficient utilization of information between channels in crowd counting tasks in existing methods. In the method of the present invention, a crowd counting model is proposed. Through modular design, global and detail information in the image is adaptively extracted, solving the problems of high computational complexity, slow reasoning speed, and insufficient counting accuracy in the crowd counting task in the existing technology. Specifically, the crowd counting model reduces the computational complexity of the traditional self-attention mechanism by designing an adaptive global perception module AGP, which is conducive to the realization of efficient global feature extraction; by introducing a multi-scale Mamba aggregation module, features at different levels are effectively integrated to improve the ability to capture detail information; by designing a channel-aware Mamba decoder CMD, the utilization of channel information is enhanced, which is conducive to improving counting accuracy.

[0024] The crowd counting method based on adaptive global perception and multi-scale feature fusion in this embodiment includes the following steps:

[0025] Step 1. Obtain a public dataset and preprocess the images in the dataset to construct a dataset for training the crowd counting model based on adaptive global perception and multi-scale feature fusion in the following step 2.

[0026] The data in this embodiment are all derived from the public data set ShanghaiTech, and the data set includes two parts, Part A and Part B. Part A is an extremely crowded crowd, and Part B is a relatively sparse crowd.

[0027] Each part of the dataset is divided into training data and test data. PartA contains 300 training images and 182 test images, and PartB contains 400 training images and 316 test images.

[0028] First, the data set needs to be preprocessed according to the process of the present invention. The image is resized and split into 256×256 and written into a new folder. The preprocessed image is represented as .

[0029] Step 2. Build a crowd counting model based on adaptive global perception and multi-scale feature fusion.

[0030] like Figure 2 As shown, the crowd counting model constructed in this embodiment includes a stem module, an adaptive global perception module AGP, a multi-scale Mamba aggregation module, and a channel-aware Mamba decoder CMD, etc.

[0031] The processing of the adaptive global perception module is divided into four stages, namely the first, second, third and fourth stages.

[0032] Given an input image ,in and Respectively represent the height and width of the input image, 3 represents the number of channels, then the overall processing flow of the input image in the crowd counting model is as follows:

[0033] First, the input image is processed by the stem module to segment the input image into patches. Then, the adaptive global perception module is applied to process the features in the first stage to generate initial features with global long-range dependencies. .

[0034] Initial Features The feature processing is carried out in the second, third and fourth stages in sequence.

[0035] Among them, the adaptive global perception module AGP is cascaded with the downsampling operation in the second, third and fourth stages respectively to extract multi-scale features with linear complexity.

[0036] The features output in the second, third and fourth stages are , , ,in, , , , Both represent the number of channels.

[0037] Then, the features of different scales output by the second, third and fourth stages are , , It is sent to the multi-scale Mamba aggregation module to effectively integrate information at different levels and output the aggregated features.

[0038] Finally, the features aggregated by the multi-scale Mamba aggregation module are input into the channel-aware Mamba decoder, and the channel-aware Mamba decoder is used to enhance the inter-channel information and generate a crowd density map.

[0039] like Figure 2 As shown, the processing flow of the adaptive global perception module AGP is as follows:

[0040] For the input feature map ,in , and They represent the height, width, and number of channels of the input feature map, respectively, and are first normalized by a layer normalization.

[0041] Features after standardization It is divided into three information flows, which are defined as the first, second and third information flows respectively.

[0042] First information flow First, a linear transformation is performed to convert it into a new feature :

[0043] ;

[0044] .

[0045] in is the weight matrix for linear transformation, Representation layer normalization processing, .

[0046] The linearly transformed feature map It is processed by a 3×3 depthwise separable convolutional layer that performs spatial convolution on each input channel separately.

[0047] The features output by the depthwise separable convolutional layer are passed into the selective scanning 2D module, which scans the input data along multiple directions and selectively processes the information in the feature map to capture spatial dependencies.

[0048] The output features of the selective scanning 2D module are normalized by a layer and then combined with the second information stream The linearly transformed features are multiplied element by element to obtain the multiplied features. The multiplied features After a linear transformation, the third information flow Add element by element to generate the final feature .

[0049] The process is represented as follows:

[0050] ;

[0051] .

[0052] in, represents the depth-wise separable convolution processing, Indicates the selective scan 2D process.

[0053] The AGP module proposed in this paper adopts a selective scanning strategy to reduce the computational complexity of the traditional self-attention mechanism from O(N²) to linear complexity, achieves efficient extraction of global features, and greatly improves the reasoning speed of the model.

[0054] In order to make up for the lack of detailed information in high-level features, the present invention designs a multi-scale Mamba aggregation module, which can fuse feature information of different scales, enhance the network's ability to capture details and global features, and improve the accuracy of crowd counting.

[0055] like Figure 3 As shown in Figure 2, the processing flow of the multi-scale Mamba aggregation module is as follows:

[0056] First, the multi-scale Mamba aggregation module receives the features of different scales output by the second, third and fourth stages. , and , and the features of the output of the third and fourth stages and Upsample separately.

[0057] Then, the processed features of the three different scales are preliminarily fused through element-by-element addition operation to integrate feature information at different levels and obtain the fused features.

[0058] Secondly, the fused features are subjected to three groups of convolution operations with different convolution kernel sizes of 1×1, 3×3, and 5×5 to extract rich local information. After each group of convolution operations, a dilated convolution is introduced to further expand the receptive field and enhance the model's ability to capture spatial information, thereby increasing the diversity of features.

[0059] The 1×1 convolution processing is followed by a dilated convolution processing with a convolution kernel size of 1, the 3×3 convolution processing is followed by a dilated convolution processing with a convolution kernel size of 3, and the 5×5 convolution processing is followed by a dilated convolution processing with a convolution kernel size of 5.

[0060] Finally, the features after element-by-element multiplication of the three dilated convolutions of 1×1, 3×3, and 5×5 are further added element-by-element with the fused features after 1×1 convolution to generate the final fused features. .

[0061] The above process can be expressed as:

[0062] .

[0063] in, represents upsampling, represents the fused features, represents the convolution operation with different convolution kernel sizes, Represents dilated convolution operations with different kernel sizes.

[0064] Through multi-scale feature fusion, the model can combine information from different scales, further enhance the network's perception and processing capabilities of global information, improve the network's accuracy and robustness, while maintaining efficient computing performance.

[0065] like Figure 4 As shown, the processing flow of the channel-aware Mamba decoder CMD is as follows:

[0066] First, the fusion features of the input channel-aware Mamba decoder , input the adaptive global perception module to capture the long-range dependencies in space and obtain the features .

[0067] Then the features The layer normalization is used to improve the balance of feature distribution. The standardized features are further extracted through the depth-separable convolution operation to extract local spatial features. The processing process is expressed as:

[0068] .

[0069] in, Represents fusion features The result processed by the adaptive global perception module.

[0070] A channel-aware mechanism consisting of average pooling and maximum pooling is introduced in the channel-aware Mamba decoder to process the features after the depth-wise separable convolution operation and generate the maximum pooling feature. and average pooling features .

[0071] Then, the maximum pooling feature and average pooling features Perform element-by-element addition and generate channel weights through the Sigmoid function to weight the input features, thereby enhancing important information and suppressing irrelevant information. The process is expressed as:

[0072] .

[0073] in, Represents the Sigmoid activation function, MP represents the maximum pooling operation, and AP represents the average pooling operation.

[0074] Finally, the weighted features are multiplied element by element with the features after the depthwise separable convolution operation, and the multiplied features are added element by element with the output features of the adaptive global perception module to obtain the final output features. .

[0075] .

[0076] Output characteristics After adjusting the number of channels through 1×1 convolution, the crowd density map with a channel number of 1 is output, and the value of each pixel in the crowd density map is added to obtain the final predicted number of people.

[0077] In order to solve the problem that traditional methods fail to capture enough information between channels, the present invention introduces the CMD module, dynamically adjusts the channel weights, strengthens the information correlation between channels, and further improves the counting accuracy and robustness of the model.

[0078] Step 3. Based on the data set constructed in step 1, the crowd counting model constructed in step 2 is trained to obtain a trained crowd counting model, and the trained crowd counting model is used to estimate the number of people.

[0079] During the crowd counting model training process, a composite loss function is constructed.

[0080] The loss function consists of the weighted sum of counting loss, optimal distribution loss ODA and mean square error loss.

[0081] First, the core goal of crowd counting is to make the predicted number of people as close as possible to the actual number of people. Therefore, the present invention uses counting loss to measure the gap by calculating the absolute difference between the predicted number of people and the actual number of people. The calculation formula is as follows:

[0082] ;

[0083] in is the counting loss, For vector norm, and Represent the actual number of people and the predicted number of people respectively.

[0084] In addition, in order to further narrow the distribution gap between the predicted density map and the true density map and enhance the network's ability to fit spatial information, the present invention designs the optimal distribution assignment loss (ODA).

[0085] This loss normalizes the predicted density map and the true density map into probability distributions and calculates the minimum transfer cost between the two to measure their difference in global distribution; the ODA loss is defined as follows:

[0086] ;

[0087] in, represents ODA losses, represents the normalized probability distribution of the true density map, Represents the normalized probability distribution of the predicted density map.

[0088] Represents probability distribution and The optimal transmission cost between is calculated as:

[0089] ;

[0090] in, is the transmission scheme matrix, represents the transmission scheme from point i to point j, Represents the transmission cost from point i to point j, usually expressed as the square of the Euclidean distance, Represents a set of transmission schemes.

[0091] Since ODA loss focuses on the global gap between the two distributions, ODA loss can align the predicted density map and the true density map well when dealing with high-density scenes. However, in sparse areas, due to the small number of data points, ODA loss is difficult to accurately capture the subtle changes in low-density areas, resulting in poor performance of the network in sparse areas.

[0092] In order to solve this problem, the present invention introduces the mean square error, i.e. loss to make up for the defect that ODA loss is insensitive to sparse areas. The loss can more accurately capture the density changes in sparse areas by calculating the square error between the predicted density map and the true density map at each pixel, thereby enhancing the network's prediction ability in sparse scenes.

[0093] Loss is defined as follows:

[0094] ;

[0095] in, represents the mean square error loss, represents the predicted density map, Represents the true density map after adaptive Gaussian kernel smoothing.

[0096] Total loss function, also known as compound loss function The definition is as follows:

[0097] ;

[0098] in, and are loss coefficients, which are used to adjust the weights of ODA loss and mean square error loss respectively.

[0099] By designing a composite loss function including counting loss, ODA and mean square error loss, the convergence speed and accuracy of the network in crowd counting tasks are improved, so that it has higher prediction ability in different density scenarios.

[0100] The model proposed in the present invention constructs an efficient and accurate crowd counting model through global information extraction of the AGP module, feature fusion of the multi-scale Mamba aggregation module, inter-channel information capture of the CMD module and optimization design of the composite loss function, which significantly improves the inference speed and counting accuracy and reduces the demand for computing resources.

[0101] After the model training is completed, the trained crowd counting model is used to estimate the number of people. Specifically, firstly, an image to be estimated is obtained, and then the image is preprocessed according to the preprocessing method of step 1.

[0102] Then, the preprocessed image features are input into the above-mentioned trained crowd counting model, and a crowd density map with a channel number of 1 is output. Each pixel point in the crowd density map is summed to obtain the predicted number of people.

[0103] In addition, in order to verify the performance advantages of the model proposed in the present invention in actual deployment, the present invention also introduces a measurement indicator FPS, which is the number of pictures that the optimized model can process per second.

[0104] The FPS data of the model proposed in the present invention on the NVIDIA GeForce RTX 3060Laptop is compared with the mainstream CSRNet method and MAN method. The processed image resolution is set to 256×256. The specific comparison data is shown in Table 1.

[0105] Table 1 Comparison of FPS index results of different methods

[0106]

[0107] It can be seen from Table 1 that the FPS index of the model proposed in the present invention is significantly better than the CSRNet method and the MAN method involved in the comparison, which proves the effectiveness of the method of the present invention. When the edge device with weaker performance uses the model provided by the present invention in actual deployment, the speed of data processing can be greatly accelerated to cope with sudden crowd gatherings and the like.

[0108] The method of the present invention improves the counting accuracy and reasoning speed in densely populated scenes by designing an adaptive global perception module, an AGP module, a CMD module and a composite loss function in the constructed crowd counting model, while significantly reducing the model parameters and computing resources, ensuring the counting accuracy and reasoning speed in high-density crowd scenes, achieving lightweight models, having good versatility and deployment potential on mobile terminals or embedded devices, and being suitable for application scenarios such as security monitoring, urban management, and crowd flow monitoring in public places that require estimation of the number of people in images or video frames.

[0109] Of course, the above description is only a preferred embodiment of the present invention, and the present invention is not limited to the above embodiments. It should be noted that all equivalent substitutions and obvious deformation forms made by any technician familiar with the field under the guidance of this specification fall within the essential scope of this specification and should be protected by the present invention.

Claims

1. A crowd counting method based on adaptive global perception and multi-scale feature fusion, characterized in that: The steps include: Step 1. Preprocess the image data to construct a data set for training the crowd counting model based on adaptive global perception and multi-scale feature fusion in the following step 2; Step 2. Build a crowd counting model based on adaptive global perception and multi-scale feature fusion, which includes stem module, adaptive global perception module, multi-scale Mamba aggregation module and channel-aware Mamba decoder; The processing of the adaptive global perception module is divided into four stages, namely the first, second, third and fourth stages; First, the input image is processed by the stem module to segment the input image into patches. Then, the adaptive global perception module is applied to process the features in the first stage to generate initial features with global long-range dependencies. In the second, third, and fourth stages, the adaptive global perception module is cascaded with the downsampling operation to extract multi-scale features with linear complexity. Then, the features of different scales output from the second, third and fourth stages are fed into the multi-scale Mamba aggregation module, which effectively integrates the information at different levels and outputs the aggregated features; Finally, the aggregated features are input into the channel-aware Mamba decoder, and the channel-aware Mamba decoder is used to enhance the inter-channel information and generate a crowd density map; Step 3. Based on the data set constructed in step 1, the crowd counting model constructed in step 2 is trained to obtain a trained crowd counting model, and the number of people is estimated using the trained crowd counting model; The processing flow of the adaptive global perception module is as follows: The feature map input to the adaptive global perception module is first normalized by a layer, and the normalized features are divided into three information streams, which are defined as the first, second, and third information streams. The first information stream first undergoes a linear transformation, and the feature map after the linear transformation is processed by a depth-wise separable convolutional layer, which performs spatial convolution on each input channel separately; The features output by the depthwise separable convolutional layer are passed into the selective scanning 2D module, which scans the input data in multiple directions and selectively processes the information in the feature map to capture spatial dependencies. The output features of the selective scanning 2D module are normalized by a layer normalization and then multiplied element-by-element with the features of the second information stream after linear transformation to obtain the multiplied features; The multiplied features are linearly transformed and then added element by element with the third information stream to generate the final features. The processing flow of the multi-scale Mamba aggregation module is as follows: First, the multi-scale Mamba aggregation module receives the features of different scales output by the second, third and fourth stages, and upsamples the features output by the third and fourth stages respectively; Then, the processed features of three different scales are preliminarily fused through element-by-element addition operation to integrate feature information at different levels and obtain fused features; Secondly, the fused features are subjected to three groups of convolution operations with different convolution kernel sizes, namely 1×1, 3×3, and 5×5, to extract rich local information. After each group of convolution operations, a dilated convolution is introduced to further expand the receptive field. The 1×1 convolution is then subjected to a dilated convolution with a convolution kernel size of 1, the 3×3 convolution is then subjected to a dilated convolution with a convolution kernel size of 3, and the 5×5 convolution is then subjected to a dilated convolution with a convolution kernel size of 5. Finally, the features after element-wise multiplication of the dilated convolutions with three different convolution kernel sizes of 1×1, 3×3, and 5×5 are further added element-wise to the fused features after 1×1 convolution to generate the final fused features.

2. The crowd counting method based on adaptive global perception and multi-scale feature fusion according to claim 1 is characterized in that: The processing flow of the channel-aware Mamba decoder is as follows: First, the fused features of the input channel-aware Mamba decoder are input into the adaptive global perception module to capture the long-range dependencies in space, and then standardized through a layer normalization to improve the balance of feature distribution; The standardized features are further extracted from local spatial features through depthwise separable convolution operations; A channel-aware mechanism consisting of average pooling and maximum pooling is introduced in the channel-aware Mamba decoder to process the features after the depthwise separable convolution operation to generate maximum pooling features and average pooling features; Subsequently, the maximum pooling features and the average pooling features are added element by element, and the channel weights are generated by the Sigmoid function to weight the input features, thereby enhancing important information and suppressing irrelevant information; Finally, the weighted features are multiplied element by element with the features after the depthwise separable convolution operation, and the multiplied features are added element by element with the output features of the adaptive global perception module to obtain the final output features. The final output features are subjected to 1×1 convolution to adjust the number of channels, and a crowd density map with a channel number of 1 is output. The values ​​of each pixel in the crowd density map are added to obtain the final predicted number of people.

3. The crowd counting method based on adaptive global perception and multi-scale feature fusion according to claim 1 is characterized in that: In step 1, the process of preprocessing the image data is as follows: The image data are all from the public dataset ShanghaiTech, which consists of two parts: Part A and Part B. Part A is the image data of extremely crowded crowds, while Part B is the image data of sparse crowds. Each part of the dataset is divided into training data and test data; First, the dataset is preprocessed and the images are resized and split into 256×256.

4. The crowd counting method based on adaptive global perception and multi-scale feature fusion according to claim 1 is characterized in that: In the step 3, during the training of the crowd counting model, a composite loss function is constructed; The loss function consists of the weighted sum of the counting loss, the optimal distribution loss ODA and the mean square error loss; The counting loss measures the gap by calculating the absolute difference between the predicted number of people and the actual number of people. The calculation formula is as follows: in is the counting loss, ||·||1 is the L1 norm of the vector, ||z||1 and Represent the real number of people and the predicted number of people respectively; The optimal distribution loss ODA normalizes the predicted density map and the true density map into probability distributions and calculates the minimum transmission cost between the two to measure their difference in global distribution; the ODA loss is defined as follows: in, represents ODA losses, represents the normalized probability distribution of the true density map, Represents the normalized probability distribution of the predicted density map; Represents probability distribution and The optimal transmission cost between is calculated as: Where γ is the transmission scheme matrix, γ ij represents the transmission scheme from point i to point j, C ij represents the transmission cost from point i to point j, usually expressed as the square of the Euclidean distance, and Γ represents the set of transmission schemes; The mean square error loss can more accurately capture the density changes in sparse areas by calculating the square error between the predicted density map and the true density map at each pixel, thereby enhancing the prediction ability of the network in sparse scenes. It is defined as follows: Among them, L2(D,D') represents the mean square error loss, D represents the predicted density map, D ' represents the real density map after adaptive Gaussian kernel smoothing; The total loss function, that is, the composite loss function L d The definition is as follows: Among them, λ1 and λ2 are loss coefficients, which are used to adjust the weights of ODA loss and mean square error loss respectively.

Images