Population density estimation method based on sparse support base estimation

Abstract

The application discloses a crowd density estimation method based on sparse support base estimation, and proposes a target perception attention mechanism enhanced by sparse support base estimation of head features of a monitoring area based on crowd congestion, density, and mutual occlusion; proposes a target scale perception attention mechanism of sparse support base estimation based on the change of head scale caused by the distance of people in a scene, generates a head scale perception rate, controls an adaptive sampling mechanism of convolution, further improves the distinguishability of heads, performs crowd density prediction on the enhanced multi-scale features, obtains a crowd density map, and statistically obtains the number of crowds in the monitoring area; solves the problem that heads are difficult to distinguish due to crowd congestion, different density, and mutual occlusion in a monitoring area where a large number of people gather and appear, and the problem that the scale of heads in an image changes due to the distance of people in a scene, and a crowd density map with good quality can be obtained, and the accuracy of crowd density estimation is improved.

Description

Technical Field

[0001] This invention belongs to the field of image processing technology, specifically relating to a crowd density estimation method based on sparse support basis estimation. Background Technology

[0002] In densely populated and congested environments, stampedes, traffic jams, and crowd conflicts are highly likely to occur, posing a significant threat to people's safety in severe cases. Therefore, designing an accurate and efficient crowd density estimation system is crucial for ensuring people's safety.

[0003] In recent years, crowd density estimation based on deep learning has seen rapid development and widespread application. The main task of crowd density estimation is to estimate the crowd density map in a scene and then calculate the number of people. Currently, deep learning-based crowd density estimation methods can extract more accurate head features, resulting in higher quality and more accurate density maps, thus improving the crowd density estimation performance. However, in real-world scenarios, existing crowd counting methods still face challenges such as difficulty in distinguishing heads due to crowd congestion and occlusion, as well as variations in head scale caused by differences in the distance and density of people in the scene. Summary of the Invention

[0004] The purpose of this invention is to provide a crowd density estimation method based on sparse support basis estimation, which solves the problems of difficulty in distinguishing heads in monitoring areas where a large number of people gather and appear, due to crowding, varying density, and mutual occlusion, as well as the problem of changes in head scale in images due to the distance of people in the scene. It can also obtain a high-quality crowd density map and improve the accuracy of crowd density estimation.

[0005] The technical solution adopted in this invention is a crowd density estimation method based on sparse support basis estimation. Based on crowd crowding, density, and mutual occlusion, a target perception attention mechanism based on sparse support basis estimation is proposed to enhance the head features in the monitoring area. Based on the changes in head scale caused by the different distances of people in the scene, a target scale perception attention mechanism based on sparse support basis estimation is proposed to generate a head scale perception rate and control the adaptive sampling mechanism of convolution to further improve the discriminability of heads. The enhanced multi-scale features are used to predict the crowd density to obtain a crowd density map. The number of people in the monitoring area is obtained by statistically analyzing the crowd density map.

[0006] The invention is further characterized by:

[0007] The specific steps are as follows:

[0008] Step 1: Download the standard crowd density estimation image dataset I and its corresponding ground truth set GT from the official website. Divide the downloaded standard crowd density estimation image dataset I and its corresponding ground truth set GT into training sample set I according to a certain ratio. train Training the ground truth label set GT train and test sample set I test Test truth label set GT test ;

[0009] Step 2: Based on the training sample set I train and truth tag set GT train Create a multi-resolution crowd density map ground truth set DG and a multi-resolution target perception attention map ground truth set AG, based on the test sample set I. test and truth tag set GT test Construct the truth set D of the crowd density map test ;

[0010] Step 3: Construct the network structure for a population density estimation method based on sparse support basis estimation;

[0011] Step 4: Design the loss function for the population density estimation method based on sparse support basis estimation;

[0012] Step 5: Set the training parameters, and set I... train The network structure is constructed by feeding DG and AG into it, and the model is trained according to the training parameters. When the training termination condition is met, the network model M is output. train ;

[0013] Step 6: Test sample set I test and truth tag set GT test For network model M train test;

[0014] Step 7: Input the image of the crowd density to be estimated and its ground truth labels into the tested network model M. train This allows us to obtain an estimated population density.

[0015] The specific process of step 1 is as follows:

[0016] Download the standard crowd density estimation image dataset I = {I1, ..., I...} from the official website. m ,…,I M} and the truth label set GT={G1,…,G m ,…,G M}, I m and G m This represents the m-th image in I and its head position label, 1≤m≤M, where M represents the total number of images in I. m ∈R W×H×CW, H, and C represent the corresponding I. m The width, height, and number of channels of the image are used to divide I into training and testing sample sets in an N:T ratio. and Indicate I train The nth image in the series, Indicate I test The t-th image in the dataset, N+T=M, I train and I test Training ground truth label set GT train and test truth label set GT test They are respectively and GT train The truth label of the nth image in the dataset. GT test The truth label of the t-th image in the dataset, I train and I test The number of people are respectively Indicates GC train The number of people in the nth image. Indicates GC test The number of people in the t-th image.

[0017] Step 2 is as follows:

[0018] Step 2.1: Define the multi-resolution crowd density map truth set variable as DG, initialize it to empty, DG = NULL, and input the training sample set I. train and training ground truth label set GT train To create a ground truth set (DG) for multi-resolution crowd density maps;

[0019] Step 2.2: Define the multi-resolution target perception attention map truth set variable as AG, initialize it to empty, AG = NULL, and use the training sample set I. train Construct multi-resolution target perception attention map ground truth using the multi-resolution crowd density map ground truth set (DG);

[0020] Step 2.3, in test sample set I test and test truth label set GT test Based on this, create a truth set of the population density map D represents test The t-th test image The true value of the population density map.

[0021] Step 2.1 The specific process is as follows:

[0022] Step 2.1.1: Input training sample set I train and training ground truth label set GT train ;

[0023] Step 2.1.2: Define an image count counter variable as n, and initialize n to 1, i.e., n = 1;

[0024] Step 2.1.3, from I train Read the nth image From GT train Read from The corresponding head position tags constitute Head Value Tag Set express The nth z The location label for the individual's head. and These represent the center, width, and height of the human head, respectively. and These represent the center positions of the human head. x and y coordinates, 1 ≤ n z ≤n Z n Z Represents training images The total number of people in the middle;

[0025] Step 2.1.4: Define the counter variable for the number of heads in the training samples as n. z n z Initialize to 1, i.e., n z =1;

[0026] Step 2.1.5 The nth z Creating Gaussian heatmap truth values ​​for individual heads: In Choose any point X = (x, y) from the given information. T Its two-dimensional Gaussian kernel is At the center of its head use Function representation, using G σ (X) Perform convolution to obtain the nth... z Gaussian heatmap of individual heads is truly valuable As shown in formula (1):

[0027]

[0028] Where σ is the scale parameter of the Gaussian kernel;

[0029] Step 2.1.6, Determine n z Is it greater than or equal to n? Z If n z <n Z Then n z =n z +1; return to step 2.1.5, otherwise, process all... Accumulation The corresponding ground truth of the population density map Right now right Sampling by a factor of s yields 2≤s≤S, all constitute The corresponding multi-resolution crowd density map truth set DG n ,Right now

[0030] Step 2.1.7: Determine if n is greater than N. If n ≥ N, the creation of the multi-resolution crowd density map ground truth set is complete, and DG is output; otherwise, DG += DG n If n = n + 1, then return to step 2.1.3.

[0031] Step 2.2 The specific process is as follows:

[0032] Step 2.2.1: Input training sample set I train and the ground truth set DG of multi-resolution crowd density maps;

[0033] Step 2.2.2: Define the image count counter variable as n, and initialize n to 1, i.e., n = 1;

[0034] Step 2.2.3: Define a threshold τ, where τ is 1e -5 Judge the images in sequence The corresponding multi-resolution crowd density map ground truth (DG) n The pixel value X at the corresponding position in the middle i Is it less than τ? If X i ≤τ, X i The value corresponding to the pixel is adjusted to 0 otherwise, to obtain the ground truth AG of the multi-resolution target perception attention map. s n, as shown in formula (2):

[0035]

[0036] Step 2.2.4, AG n Adding AG, i.e., AG+=AG n ;

[0037] Step 2.2.5: Determine if n is greater than N. If n≥N, the creation of the ground truth of the multi-resolution target perception attention map is completed, and AG is output; otherwise, n is incremented by 1, i.e., n=n+1, and return to step 2.2.3.

[0038] Step 3 is as follows:

[0039] The network structure of the crowd density estimation method based on sparse support basis estimation includes an input module, a feature extraction module, an object-aware attention module, a scale-aware attention module, a crowd density estimation module, and an output module.

[0040] The input module takes the training sample set I as input. train Images of crowds 1≤n≤N;

[0041] The feature extraction module consists of four sets of convolutional and pooling layers, connected in the following order: Conv1 (convolutional layer), Maxpooling1 (pooling layer), Conv2 (convolutional layer), Maxpooling2 (pooling layer), Conv3 (convolutional layer), Maxpooling3 (pooling layer), and Conv4 (convolutional layer). The input is fed into the feature extraction module, which extracts the output features of the s-th combined layer as... The feature f at the s-th scale s , 1≤s≤S, W s H s C s f s The width, height, and number of channels, where S represents the total number of scales;

[0042] The s-th object-aware attention module consists of an input layer, an object-aware sparse support basis estimation network, and an output layer. The object-aware sparse support basis estimation network consists of three convolutional layers, one deconvolutional layer, and one max pooling layer connected in series. The specific connection order is as follows: Convolutional layers Pooling layer Deconvolution layer Convolutional layer Convolutional layer Input layer data is Output layer f s 'Estimating the output a of the target-aware sparse support basis network' s It is formed by multiplying the input layer by the dot product and then adding it back to the input layer, i.e. When s = S

[0043] The s-th target scale-aware attention module consists of an input layer, a target scale-aware sparse support basis estimation network, and an output layer. The target scale-aware sparse support basis estimation network consists of three concatenated convolutional layers, specifically in the following order: convolutional layer... Convolutional layer Convolutional layer The input layer data is f s The output layer data is f. s The output of the scale-aware sparse support basis estimation network is d. s f s "is based on d" s For the expansion rate at f s The result of dilated convolution performed on ', i.e. Conv2d(*,*,*,*) is a two-dimensional convolution function, where Indicates the number of input channels. f s "Number of channels, k represents the kernel size, d..." s This represents the expansion rate; when s > 1, f s "Upsampling and f" s-1 The summation constitutes the input features of the (s-1)th target perception module, i.e. Where interpolate(*,*,*,*) represents the upsampling function, W s-1 ×H s-1 Indicates the output size; bilinear indicates the bilinear interpolation method.

[0044] The s-th crowd density estimation module consists of two input layers, a residual connection layer, a convolutional module, and an upsampling layer. The input layer has one input data point f. s The other data is f s The residual connection layer is the sum of the two inputs, and the convolutional module consists of two convolutional layers, namely... and The output of the s-th crowd density estimation module is DM s , When s > 1, the upsampling layer... The output features are upsampled and fed into the (s-1)th population density estimation module;

[0045] The output module is the statistical result of DM1, that is, the number of people C, C = Sum(DM1).

[0046] Step 4 is as follows:

[0047] Total loss L total The loss consists of crowd density map loss (ML) and target perception attention loss (AL), with a total loss L. totalThe calculations are shown in equation (3), ML is calculated as shown in equation (4), and AL is calculated as shown in equation (5):

[0048]

[0049]

[0050]

[0051] in, Let λ1 + λ2 represent the crowd density map loss (ML) and the object perception attention loss (AL) at scale s for the nth training image, respectively, where λ1 + λ2 = 1.0. and represent the predicted crowd density map and the ground truth crowd density map of the nth training image at the s-th scale, respectively. Let N and S represent the target-aware attention density map and the target-aware attention ground truth value predicted by the nth training image at the s-th scale, respectively. N is the size of the training sample set, and S is the total number of scales.

[0052] Step 5 is as follows:

[0053] Step 5.1: Input training image set I train The ground truth set for multi-resolution crowd density estimation (DG) and the ground truth set for multi-resolution target perception attention maps (AG);

[0054] Step 5.2: Set the network model training parameters based on the sparse support basis estimation method for population density estimation, namely, set the learning rate variable as Learning_rate, the maximum number of training iterations variable as Max_iter, the batch size variable as Batch_size, the minimum change value of the target loss function ΔMinLoss, and define the number of training iterations variable as Step, initialize Step to 1;

[0055] Step 5.3: Train the network using the training parameters set in Steps 5.1 and 5.2. The network training ends when the change in the target loss function is less than or equal to ΔMinLoss or Step ≥ Max_iter. Save the network model M. train Otherwise, Step = Step + 1, use the Adam optimizer to reverse-correct the weight coefficients of each network layer in the network model, continue training, and output the network model M when the training termination condition is met. train .

[0056] Step 5.4, if the target loss function L totalIf the network fails to converge or Step < Max_iter, then Step = Step + 1, and the Adam optimizer is used to back-correct the weight coefficients of each network layer in the network model, returning to step 5.3; otherwise, network training ends, and the network model M is saved. train .

[0057] Step 6 is as follows:

[0058] Step 6.1: Input test sample set I test Define the set of predicted density estimation map variables DM test DM test Initialize to empty, i.e., DM test =NULL; Defines the set C of the predicted number of people in the image. test C test Initialize to empty, i.e., C test =NULL;

[0059] Step 6.2: Define the test image counter variable as t, and initialize t to 1, i.e., t = 1;

[0060] Step 6.3, from I test Read the t-th image Will Enter to M train In the middle, output Corresponding density estimation map Will Add to DM test ,Right now And statistical analysis of images Number of people in Will Add to C test ,Right now

[0061] Step 6.4: Determine if t is greater than T. If t ≥ T, the crowd density estimation ends and the crowd density estimation result is output; otherwise, t = t + 1, and return to step 6.3.

[0062] Step 6.5: Evaluate network model M based on the output crowd density estimation results. train .

[0063] The beneficial effects of this invention are:

[0064] This invention presents a crowd density estimation method based on sparse support basis estimation. It introduces a target perception attention module and a target scale perception attention module based on sparse support basis estimation. This allows the method to not only further enhance the features of crowd targets but also adaptively select the receptive field for head targets of different sizes in the scene. This results in high crowd density estimation accuracy even in scenes with background noise interference and varying head target sizes. Attached Figure Description

[0065] Figure 1 This is a flowchart of the population density estimation method based on sparse support basis estimation of the present invention;

[0066] Figure 2 This is a schematic diagram of the multi-resolution target perception attention module structure of the present invention;

[0067] Figure 3 This is a schematic diagram of the multi-resolution scale perception attention module structure of the present invention;

[0068] Figure 4 This is a flowchart of the crowd density map truth generation process for the crowd density estimation method based on sparse support basis estimation of the present invention.

[0069] Figure 5 This is a flowchart of the multi-resolution target perception attention map ground truth generation process of the present invention;

[0070] Figure 6 This is a schematic diagram of the network structure of the population density estimation method based on sparse support basis estimation of the present invention;

[0071] Figure 7 This is a test image of ShanghaiTech_PartA in this embodiment of the invention, as well as a truth label diagram and a truth diagram of the target perception attention map;

[0072] Figure 8 This invention relates to a population density estimation method based on sparse support basis estimation. Figure 7 The test results;

[0073] Figure 9 This is a test image of ShanghaiTech_PartB in this embodiment of the invention, as well as a truth label diagram and a truth diagram of the target perception attention map.

[0074] Figure 10 This invention relates to a population density estimation method based on sparse support basis estimation. Figure 9 The test results. Detailed Implementation

[0075] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments.

[0076] This invention presents a crowd density estimation method based on sparse support basis estimation. The process is as follows: First, to address the difficulty in distinguishing heads due to crowd density variations and occlusion, a target perception attention mechanism based on sparse support basis estimation is proposed to enhance head features in the monitored area. Second, to address the issue of head scale variations caused by people's distance from the scene, a target scale perception attention mechanism based on sparse support basis estimation is proposed to generate a head scale perception rate, thereby controlling the adaptive sampling mechanism of convolution and further improving head discriminability. Next, crowd density prediction is performed on the enhanced multi-scale features to obtain a crowd density map. Finally, the number of people in the monitored area is obtained by statistically analyzing the crowd density map.

[0077] like Figure 1 As shown, please follow these steps:

[0078] Step 1: Download the standard crowd density estimation image dataset I and its corresponding ground truth set GT from the official website. Divide the downloaded standard crowd density estimation image dataset I and its corresponding ground truth set GT into training sample set I according to a certain ratio. train Training the ground truth label set GT train and test sample set I test Test truth label set GT test The specific process is as follows:

[0079] Download the standard crowd density estimation image dataset I = {I1, ..., I...} from the official website. m ,…,I M} and the truth label set GT={G1,…,G m ,…,G M}, I m and G m This represents the m-th image in I and its head position label, 1 ≤ m ≤ M, where M represents the total number of images in I, M = 716. m ∈R W×H×C W, H, and C represent the corresponding I. m The width, height, and number of channels of the image are W=1024, H=768, and C=3. The image I is divided into a training sample set and a test sample set according to the N:T ratio. and Indicate I train The nth image in the series, Indicate I test The t-th image in the dataset, N+T=M, N=400, T=316, I train and I testTraining ground truth label set GT train and test truth label set GT test They are respectively and GT train The truth label of the nth image in the dataset. GT test The truth label of the t-th image in the dataset, I train and I test The number of people are respectively Indicates GC train The number of people in the nth image. Indicates GC test The number of people in the t-th image.

[0080] Step 2: Based on the training sample set I train and truth tag set GT train Create a multi-resolution crowd density map ground truth set DG and a multi-resolution target perception attention map ground truth set AG, based on the test sample set I. test and truth tag set GT test Construct the truth set D of the crowd density map test The specific process is as follows:

[0081] Step 2.1: Define the multi-resolution crowd density map truth set variable as DG, initialize it to empty, DG = NULL, and input the training sample set I. train and training ground truth label set GT train The process of creating a ground truth set (DG) for multi-resolution crowd density maps is as follows:

[0082] Step 2.1.1: Input training sample set I train and training ground truth label set GT train ;

[0083] Step 2.1.2: Define an image count counter variable as n, and initialize n to 1, i.e., n = 1;

[0084] Step 2.1.3, from I train Read the nth image From GT train Read from The corresponding head position tags constitute Head Value Tag Set express The nth z The location label for the individual's head. and These represent the center, width, and height of the human head, respectively. and These represent the center positions of the human head. x and y coordinates, 1 ≤ n z ≤n Z n Z Represents training images The total number of people in the middle;

[0085] Step 2.1.4: Define the counter variable for the number of heads in the training samples as n. z n z Initialize to 1, i.e., n z =1;

[0086] Step 2.1.5 The nth z Creating Gaussian heatmap truth values ​​for individual heads: In Choose any point X = (x, y) from the given information. T Its two-dimensional Gaussian kernel is At the center of its head use Function representation, using G σ (X) Perform convolution to obtain the nth... z Gaussian heatmap of individual heads is truly valuable As shown in formula (1):

[0087]

[0088] Where σ is the scale parameter of the Gaussian kernel;

[0089] Step 2.1.6, Determine n z Is it greater than or equal to n? Z If n z <n Z Then n z =n z +1; return to step 2.1.5, otherwise, process all... Accumulation The corresponding ground truth of the population density map Right now right Sampling by a factor of s yields 2≤s≤S, all constitute The corresponding multi-resolution crowd density map truth set DG n ,Right now

[0090] Step 2.1.7: Determine if n is greater than N. If n ≥ N, the creation of the multi-resolution crowd density map ground truth set is complete, and DG is output; otherwise, DG += DG n If n = n + 1, then return to step 2.1.3.

[0091] Step 2.2: Define the multi-resolution target perception attention map truth set variable as AG, initialize it to empty, AG = NULL, and use the training sample set I. train The process of creating a multi-resolution target perception attention map ground truth set using the multi-resolution crowd density map ground truth set (DG) is as follows:

[0092] Step 2.2.1: Input training sample set I train and the ground truth set DG of multi-resolution crowd density maps;

[0093] Step 2.2.2: Define the image count counter variable as n, and initialize n to 1, i.e., n = 1;

[0094] Step 2.2.3: Define a threshold τ, where τ is 1e -5 Judge the images in sequence The corresponding multi-resolution crowd density map ground truth (DG) n The pixel value X at the corresponding position in the middle i Is it less than τ? If X i ≤τ, X i The value corresponding to the pixel is adjusted to 0 otherwise, to obtain the ground truth of the multi-resolution target perception attention map. As shown in formula (2):

[0095]

[0096] Step 2.2.4, AG n Adding AG, i.e., AG+=AG n ;

[0097] Step 2.2.5: Determine if n is greater than N. If n≥N, the creation of the ground truth of the multi-resolution target perception attention map is completed, and AG is output; otherwise, n is incremented by 1, i.e., n=n+1, and return to step 2.2.3.

[0098] Step 2.3, in test sample set I test and test truth label set GT test Based on this, create a truth set of the population density map D represents test The t-th test image The true value of the population density map is obtained, and its production process is the same as step 2.1.

[0099] Step 3: Construct the network structure for a population density estimation method based on sparse support basis estimation; such as... Figure 6 As shown, the specific process is as follows:

[0100] The network structure of the crowd density estimation method based on sparse support basis estimation includes an input module, a feature extraction module, an object-aware attention module, a scale-aware attention module, a crowd density estimation module, and an output module.

[0101] 1) The input module takes training sample set I as input data. train Images of crowds 1≤n≤N;

[0102] 2) The feature extraction module consists of four sets of convolutional and pooling layers, connected in the following order: Convolutional layer Conv1 & pooling layer Maxpooling1, Convolutional layer Conv2 & pooling layer Maxpooling2, Convolutional layer Conv3 & pooling layer Maxpooling3, Convolutional layer Conv4, ... The input is fed into the feature extraction module, which extracts the output features of the s-th combined layer as... The feature f at the s-th scale s , 1≤s≤S, W s H s C s f s The width, height, and number of channels, where S represents the total number of scales; here, S = 4;

[0103] The specific network layer and convolutional layer parameter settings and output dimension sizes in the feature extraction module are shown in Table 1:

[0104] Table 1

[0105]

[0106]

[0107] 3) The s-th target-aware attention module consists of an input layer, a target-aware sparse support basis estimation network, and an output layer. The target-aware sparse support basis estimation network consists of three convolutional layers, one deconvolutional layer, and one max pooling layer connected in series. The specific connection order is as follows: Convolutional layers Pooling layer Deconvolution layer Convolutional layer Convolutional layer Input layer data is Output layer f s 'Estimating the output a of the target-aware sparse support basis network' s It is formed by multiplying the input layer by the dot product and then adding it back to the input layer, i.e. When s = S

[0108] Table 2 shows the specific network layer and convolutional layer parameter settings and output dimension size of the target-aware sparse support basis estimation network.

[0109] Table 2

[0110]

[0111]

[0112] 4) such as Figure 3 As shown, the s-th target scale-aware attention module consists of an input layer, a target scale-aware sparse support basis estimation network, and an output layer. The target scale-aware sparse support basis estimation network consists of three concatenated convolutional layers, specifically in the following order: convolutional layer... Convolutional layer Convolutional layer The input layer data is f s The output layer data is f. s The output of the scale-aware sparse support basis estimation network is d. s f s "is based on d" s For the expansion rate at f s The result of dilated convolution performed on ', i.e. Conv2d(*,*,*,*) is a two-dimensional convolution function, where Indicates the number of input channels. f s "Number of channels, k represents the kernel size, d..." s This represents the expansion rate; when s > 1, f s "Upsampling and f" s-1 The summation constitutes the input features of the (s-1)th target perception module, i.e. Where interpolate(*,*,*,*) represents the upsampling function, W s-1 ×H s-1 Indicates the output size; bilinear indicates the bilinear interpolation method.

[0113] Table 3 shows the specific network layer, convolutional layer parameter settings, and output dimension size of the target scale-aware sparse support basis estimation network.

[0114] Table 3

[0115]

[0116]

[0117] 5) The s-th crowd density estimation module consists of two input layers, a residual connection layer, a convolutional module, and an upsampling layer. The input layer has one input data stream f. s The other data is f s The residual connection layer is the sum of the two inputs, and the convolutional module consists of two convolutional layers, namely... and The output of the s-th crowd density estimation module is DM s , When s > 1, the upsampling layer... The output features are upsampled and fed into the (s-1)th population density estimation module;

[0118] The specific network layers, convolutional parameter settings, and output dimension size of the crowd density estimation module are shown in Table 4.

[0119] Table 4

[0120]

[0121]

[0122] 6) The output module is the statistical result of DM1, that is, the number of people C, C = Sum(DM1).

[0123] Step 4: Design the loss function for the crowd density estimation method based on sparse support basis estimation; the specific process is as follows:

[0124] Total loss L total The loss consists of crowd density map loss (ML) and target perception attention loss (AL), with a total loss L. total The calculations are shown in equation (3), ML is calculated as shown in equation (4), and AL is calculated as shown in equation (5):

[0125]

[0126]

[0127]

[0128] in, Let λ1 = 0.7 and λ2 = 0.3, representing the crowd density map loss (ML) and object perception attention loss (AL) at scale s for the nth training image, respectively. and represent the predicted crowd density map and the ground truth crowd density map of the nth training image at the s-th scale, respectively. Let N and S represent the target-aware attention density map and the target-aware attention ground truth value predicted by the nth training image at the s-th scale, respectively. N is the size of the training sample set, and S is the total number of scales.

[0129] Step 5: Set the training parameters, and set I... train The network structure is constructed by feeding DG and AG into it, and the model is trained according to the training parameters. When the training termination condition is met, the network model M is output. train The specific process is as follows:

[0130] Step 5.1: Input training image set I train The ground truth set for multi-resolution crowd density estimation (DG) and the ground truth set for multi-resolution target perception attention maps (AG);

[0131] Step 5.2: Set the training parameters for the network model based on the sparse support basis estimation method for population density estimation. Specifically, set the learning rate variable as Learning_rate, the maximum number of training iterations variable as Max_iter, the batch size variable as Batch_size, the minimum change value of the objective loss function ΔMinLoss, and define the number of training iterations variable as Step, initialized to 1. Specific settings are shown in Table 5.

[0132] Table 5

[0133]

[0134] Step 5.3: Train the network using the training parameters set in Steps 5.1 and 5.2. The network training ends when the change in the target loss function is less than or equal to ΔMinLoss or Step ≥ Max_iter. Save the network model M. train Otherwise, Step = Step + 1, use the Adam optimizer to reverse-correct the weight coefficients of each network layer in the network model, continue training, and output the network model M when the training termination condition is met. train .

[0135] Step 5.4, if the target loss function L total If the network fails to converge or Step < Max_iter, then Step = Step + 1, and the Adam optimizer is used to back-correct the weight coefficients of each network layer in the network model, returning to step 5.3; otherwise, network training ends, and the network model M is saved. train .

[0136] Step 6: Test sample set I test and truth tag set GT test For network model M train The test; the specific process is as follows:

[0137] Step 6.1: Input test sample set I test Define the set of predicted density estimation map variables DM test DM testInitialize to empty, i.e., DM test =NULL; Defines the set C of the predicted number of people in the image. test C test Initialize to empty, i.e., C test =NULL;

[0138] Step 6.2: Define the test image counter variable as t, and initialize t to 1, i.e., t = 1;

[0139] Step 6.3, from I test Read the t-th image Will Enter to M train In the middle, output Corresponding density estimation map Will Add to DM test ,Right now And statistical analysis of images Number of people in Will Add to C test ,Right now

[0140] Step 6.4: Determine if t is greater than T. If t ≥ T, the crowd density estimation ends and the crowd density estimation result is output; otherwise, t = t + 1, and return to step 6.3.

[0141] Step 6.5: Evaluate network model M based on the output crowd density estimation results. train .

[0142] The estimation results for all test images were evaluated by calculating their mean absolute error (MAE) and root mean square error (RMSE), as shown in formulas (6) and (7), respectively:

[0143]

[0144]

[0145] in, This represents the predicted number of heads in the t-th test image. This represents the true number of heads in the t-th test image.

[0146] Step 7: Input the image of the crowd density to be estimated and its ground truth labels into the tested network model M. train This allows us to obtain an estimated population density.

[0147] This invention presents a crowd density estimation method based on sparse support basis estimation. It introduces a target-aware attention module and a scale-aware attention module, enabling the method to not only further enhance the features of crowd targets but also adaptively select the receptive field for head targets of different sizes within the scene. This results in high crowd density estimation accuracy even in scenes with background noise interference and varying head target sizes.

[0148] Example

[0149] This invention provides a population density estimation method based on sparse support basis estimation. The method is tested using the standard population dataset, ShanghaiTech dataset. Specific information about the dataset is shown in Table 6.

[0150] Table 6

[0151]

[0152] Dataset description:

[0153] The ShanghaiTech dataset is a large-scale crowd density estimation dataset containing 1198 images. Due to differences in crowd density and image acquisition methods, the dataset is divided into Part A and Part B. Part A images were collected from the internet, exhibiting high crowd density (maximum number of people per image: 3139), and include 300 training images and 182 test images. Part B images were captured from cameras on the streets of Shanghai. Compared to Part A, Part B has lower crowd density, with a maximum number of people per image of 578, and includes 400 training images and 316 test images. The ShanghaiTech dataset comprehensively covers different levels of crowd density and various scenarios, making it one of the most widely used datasets for crowd density estimation tasks.

[0154] Algorithm Evaluation:

[0155] This invention evaluates the crowd density estimation method by using two objective evaluation indicators, Mean Absolute Error (MAE) and Root Mean Square Error (RMSE), as well as subjective prediction results. In contrast, the method that removes the target-aware attention module and the scale-aware attention module is called the Baseline method. The network that introduces the target-aware attention module is called the Target-Aware Attention Crowd Density Estimation Method Based on Sparse Support Basis Estimation (Baseline_RA), and the network that introduces the scale-aware attention module is called the Scale-Aware Attention Crowd Density Estimation Method Based on Sparse Support Basis Estimation (Baseline_SA).

[0156] Evaluation index definition:

[0157] Mean Absolute Error (MAE). The mean absolute error represents the average absolute error between the predicted value and the actual value. The calculation formula is shown in Equation (8).

[0158]

[0159] Root mean square error (RMSE). RMSE represents the square root of the ratio of the square of the deviation between the predicted value and the true value to the total number of images N in the test set. The calculation formula is shown in equation (9).

[0160]

[0161] in, This represents the predicted number of heads in the t-th test image. This represents the true number of heads in the t-th test image.

[0162] The flowchart for creating the ground truth map of multi-resolution crowd density maps is as follows: Figure 4 As shown, the flowchart for creating the ground truth map of the multi-resolution target perception attention map is as follows: Figure 5 As shown.

[0163] Objective evaluation:

[0164] Example 3

[0165] As shown in Table 7, Table 7 presents the comparative experimental results of the present invention with Baseline, Baseline_RA, and Baseline_SA in the ShanghaiTech_PartA dataset in terms of mean absolute error and root mean square error.

[0166] Table 7

[0167]

[0168] As shown in Table 7, on the ShanghaiTech_PartA dataset, the MAE and RMSE of the present invention are 63.7 and 102.9 for the baseline, 55.3 and 90.4 for the baseline_RA, 60.3 and 96.2 for the baseline_SA, and 51.2 and 86.8 for the present invention. Compared with the baseline, the MAE and RMSE of the present invention are reduced by 12.5% ​​and 16.1%, respectively; compared with the baseline_RA, the MAE and RMSE of the present invention are reduced by 4.1% and 3.6%, respectively; and compared with the baseline_SA, the MAE and RMSE of the present invention are reduced by 9.1% and 9.4%, respectively.

[0169] As shown in Table 8, Table 8 presents the comparative experimental results of the present invention with Baseline, Baseline_RA, and Baseline_SA on the ShanghaiTech_PartB dataset in terms of mean absolute error and root mean square error.

[0170] Table 8

[0171]

[0172]

[0173] As shown in Table 8, on the ShanghaiTech_PartB dataset, the MAE and RMSE of the present invention are 11.2 and 15.3 for the Baseline method, 6.1 and 9.3 for the Baseline_RA method, 7.2 and 11.5 for the Baseline_SA method, and 5.3 and 8.9 for the present invention. Compared with the Baseline method, the MAE and RMSE of the present invention are reduced by 5.9% and 6.4%, respectively; compared with the Baseline_RA method, the MAE and RMSE of the present invention are reduced by 0.8% and 0.4%, respectively; and compared with the Baseline_SA method, the MAE and RMSE of the present invention are reduced by 1.9% and 2.6%, respectively.

[0174] As shown in Tables 7 and 8, the target perception attention crowd density estimation method based on sparse support basis estimation and the target scale perception attention crowd density estimation method based on sparse support basis estimation can achieve good estimation of crowd density and achieve relatively accurate counting. However, the crowd density estimation method based on sparse support basis estimation proposed in this invention can further improve the accuracy of crowd density estimation.

[0175] Subjective evaluation:

[0176] Figure 6 The process of using a crowd density estimation network based on sparse support basis estimation to estimate the crowd density of an input crowd image is demonstrated. Figure 7 (a) represents a test image from the ShanghaiTech_PartA dataset. Figure 7 (b) indicates that... Figure 7 The ground truth image of crowd density in image (a), Figure 7 (c) indicates that... Figure 7 (a) is the target-perceived ground truth map of the image. Figure 8 This indicates that the population density estimation method based on sparse support basis estimation proposed in this invention is used to estimate the population density. Figure 7 (a) Prediction results graph; Figure 9(a) represents a test image from the ShanghaiTech_PartB dataset. Figure 9 (b) indicates that... Figure 9 (a) is a true-valued plot of population density. Figure 9 (c) indicates that... Figure 9 (a) is the target-perceived ground truth map of the image. Figure 10 This indicates that the population density estimation method based on sparse support basis estimation proposed in this invention is used to estimate the population density. Figure 9 (a) Prediction results graph; from Figure 9 It can be seen that the population in the ShanghaiTech_PartB dataset is relatively sparse, but the target scale of the heads varies greatly. Figure 7 It can be seen that the population in the ShanghaiTech_PartA dataset is relatively dense, but the scale of the human head targets varies little; from Figure 8 The prediction results and Figure 10 The prediction results show that the population density estimation method based on sparse support basis estimation proposed in this invention predicts the number of people more closely than the true number of people. At the same time, this invention is more accurate in locating the head position, obtains more significant head features, and the population density estimation network results are more accurate.

[0177] Through the above methods, this invention proposes a crowd density estimation method based on sparse support basis estimation, which is used for crowd density estimation in monitored areas where large numbers of people gather and move about. First, to address the difficulty in distinguishing heads due to crowd density variations and occlusion, a target perception attention mechanism based on sparse support basis estimation is proposed to enhance head features in the monitored area. Second, to address the issue of varying head scales due to different distances from people in the scene, a target scale perception attention mechanism based on sparse support basis estimation is proposed to generate a head scale perception rate, thereby controlling the adaptive sampling mechanism of convolution and further improving head distinguishability. Next, crowd density prediction is performed on the enhanced multi-scale features to obtain a crowd density map. Finally, the number of people in the monitored area is statistically analyzed from the crowd density map. Experimental results verify the effectiveness of this invention, demonstrating that it can be applied to intelligent detection systems for crowd density estimation in public places, enabling timely prediction of crowded scenes and achieving the effect of intelligently maintaining public order and protecting human safety.

Claims

1. A population density estimation method based on sparse support basis estimation, characterized in that, The specific steps are as follows: Step 1: Download the standard crowd density estimation image dataset from the official website. and its corresponding truth value label set GT Download the standard crowd density estimation image dataset and its corresponding truth value label set GT Divide into training sample sets according to a certain proportion Training the true label set and test sample set Test truth label set ; Step 2: Based on the training sample set and truth value label set Creating a truth set for multi-resolution crowd density maps and multi-resolution target perception attention map truth set According to the test sample set and truth value label set Creating a truth set for crowd density maps ; Step 3: Construct the network structure for a population density estimation method based on sparse support basis estimation; Step 3 is as follows: The network structure of the crowd density estimation method based on sparse support basis estimation includes an input module, a feature extraction module, a target-aware attention module, a scale-aware attention module, a crowd density estimation module, and an output module. The input module receives training sample sets as input data. Images of crowds ; The feature extraction module consists of four sets of combined convolutional and pooling layers, connected in the following order: convolutional layer... &Pooling Layer Convolutional layer &Pooling Layer Convolutional layer &Pooling Layer Convolutional layer ,Will Input to the feature extraction module to extract the first feature. The output features of the combined layer are used as No. Features at each scale , , , They are respectively Width, height, and number of channels, Indicates the total number of scales; No. The target-aware attention module consists of an input layer, a target-aware sparse support basis estimation network, and an output layer. The target-aware sparse support basis estimation network consists of three convolutional layers, one deconvolutional layer, and one max-pooling layer connected in series. The specific connection order is as follows: convolutional layers... Pooling layer deconvolution layer Convolutional layer Convolutional layer The input layer data is Output layer The output of the network is estimated from the target-aware sparse support basis. It is formed by multiplying the input layer by the dot product and then adding it back to the input layer, i.e. ,when hour, ; No. The target scale-aware attention module consists of an input layer, a target scale-aware sparse support basis estimation network, and an output layer. The target scale-aware sparse support basis estimation network is composed of three concatenated convolutional layers, specifically in the following order: convolutional layer... Convolutional layer Convolutional layer The input layer data is The output layer data is The output of the scale-aware sparse support basis estimation network is , Therefore For the expansion rate at The result of dilated convolution performed on top, i.e. , It is a two-dimensional convolution function, where Indicates the number of input channels. express Number of channels Indicates the size of the convolution kernel. Indicates the expansion rate, when At that time, After upsampling and Adding together constitutes the first The input features of each target perception module, i.e. ,in, Represents the upsampling function. Indicates the output size. This indicates the bilinear interpolation method; No. The crowd density estimation module consists of two input layers, a residual connection layer, a convolutional module, and an upsampling layer. One input layer contains data... The other data is The residual connection layer is the sum of the two inputs, and the convolutional module consists of two convolutional layers, namely... and , No. The output of the individual crowd density estimation module is , ,when At that time, the upsampling layer The output features are upsampled and fed into the first... Individual population density estimation module; The output module is... The statistical results, i.e., the number of people. , ; Step 4: Design the loss function for the population density estimation method based on sparse support basis estimation; Step 5: Set the training parameters. , and The network structure is fed in, and the model is trained according to the training parameters. When the training termination condition is met, the network model is output. ; Step 6: Test the sample set and truth value label set For network models test; Step 7: Input the image of the crowd density to be estimated and its ground truth labels into the tested network model. This allows us to obtain an estimated population density.

2. The population density estimation method based on sparse support basis estimation according to claim 1, characterized in that, The specific process of step 1 is as follows: Download the standard crowd density estimation image dataset from the official website. and truth value label set , and express The first in Zhang image and its head location label, , express The total number of images in the middle, , They represent the corresponding The width, height, and number of channels of the image, according to The proportions are divided into training sample set and test sample set respectively. and , express The first in Zhang Image express The first in Zhang Image , and Training ground truth label set and test truth value label set They are respectively and , express The first in The truth labels of the images. express The first in The truth labels of the images. and The number of people are respectively , express The Middle The number of people in each image. express The Middle The number of people in each image.

3. The population density estimation method based on sparse support basis estimation according to claim 1, characterized in that, Step 2 is as follows: Step 2.1: Define the truth set variable for the multi-resolution crowd density map as follows. Initialized to empty. Input training sample set and training truth label set To create a truth set for multi-resolution crowd density maps ; Step 2.2: Define the truth set variable of the multi-resolution target perception attention map as follows. Initialized to empty. Through the training sample set and multi-resolution crowd density map truth set Construct ground truth maps for multi-resolution target perception attention maps; Step 2.3: In the test sample set and test truth value label set Based on this, create a truth set of the population density map , express The Middle Zhang test image The true value of the population density map.

4. The population density estimation method based on sparse support basis estimation according to claim 3, characterized in that, Step 2.1 The specific process is as follows: Step 2.1.1: Input the training sample set and training truth label set ; Step 2.1.2: Define the image count counter variable as follows. , Initialize to 1, that is ; Step 2.1.3, from Read the first One image ,from Read from The corresponding head position tags constitute Head Value Tag Set , , express The Middle The location tag for the individual's profile picture. , , and These represent the center, width, and height of the human head, respectively. , and These represent the center positions of the human head. of coordinates and coordinate, , Represents training images The total number of people in the middle; Step 2.1.4: Define the variable as a counter for the number of heads in the training samples. , Initialize to 1, that is ; Step 2.1.5 The Middle Creating Gaussian heatmap truth values ​​for individual heads: In Choose any point Its two-dimensional Gaussian kernel is In the center of its head ,use Function representation, using right Perform convolution to obtain the first... Gaussian heatmap of individual heads is truly valuable As shown in formula (1): (1) in, is the scaling parameter of the Gaussian kernel; Step 2.1.6, Judgment Is it greater than or equal to? ,if ,but Return to step 2.1.5; otherwise, process all... Accumulation The corresponding ground truth of the population density map ,Right now ,right conduct downsampling by times , ,all constitute Corresponding multi-resolution crowd density map ground value set ,Right now ; Step 2.1.7, Judgment Is it greater than ,if The production of the ground truth set for the multi-resolution crowd density map is complete, and the output is ready. ;otherwise, ,but Return to step 2.1.

3.

5. The population density estimation method based on sparse support basis estimation according to claim 3, characterized in that, Step 2.2 The specific process is as follows: Step 2.2.1: Input the training sample set and multi-resolution crowd density map truth set ; Step 2.2.2: Define the image count counter variable as follows. , Initialize to 1, that is ; Step 2.2.3: Define a threshold. , for Judge the images in sequence The corresponding multi-resolution crowd density map ground truth Pixel value at the corresponding position Is it less than ,if ,Will The value corresponding to the pixel is adjusted to 0 otherwise, to obtain the ground truth of the multi-resolution target perception attention map. As shown in formula (2): （2） Step 2.2.4, join in ,Right now ; Step 2.2.5, Judgment n Is it greater than N ,if The production of the ground truth map for multi-resolution target perception attention map is complete, and the output is complete. ; otherwise, Accumulate by 1, that is Return to step 2.2.

3.

6. The population density estimation method based on sparse support basis estimation according to claim 1, characterized in that, Step 4 is as follows: Total loss Loss from crowd density map and target perception attention loss Composition, total loss The calculation is shown in equation (3). The calculation is shown in equation (4). The calculation is shown in equation (5): （3） （4） （5） in, , They represent the first The training images in the first Population density map loss at scale Loss of attention in target perception , , , Representing the first The training images in the first Population density map predicted at different scales, and ground truth of the population density map. , They represent the first The training images in the first Scale-based prediction of target perception attention density map and ground truth of target perception attention. It refers to the size of the training sample set. It is the total number of scales.

7. The population density estimation method based on sparse support basis estimation according to claim 1, characterized in that, Step 5 is as follows: Step 5.1: Input the training image set True value set for multi-resolution crowd density estimation and multi-resolution target perception attention map truth set ; Step 5.2: Set the training parameters for the network model based on the sparse support basis estimation method for population density estimation, i.e., set the learning rate variable as... The maximum number of training iterations is the variable. The size of each batch of data varies. Minimum change of the objective loss function Define the training iteration number variable as , Initialize to 1; Step 5.3: Train the network according to the training parameters set in Steps 5.1 and 5.

2. When the change in the target loss function is less than or equal to... or At this point, network training is complete, and the network model is saved. ;otherwise ,use The optimizer back-corrects the weight coefficients of each network layer in the network model, continues training, and outputs the network model when the training termination condition is met. ; Step 5.4, if the target loss function Unconverged or ,but ,use The optimizer reverse-engineers the weight coefficients of each network layer in the network model and returns to step 5.3; otherwise, network training ends and the network model is saved. .

8. The population density estimation method based on sparse support basis estimation according to claim 1, characterized in that, Step 6 is as follows: Step 6.1: Input the test sample set Define the set of variables for the predicted density estimation map. , Initialize to empty, that is Define the set of predicted population counts in the image. , Initialize to empty, that is ; Step 6.2: Define the test image counter variable as follows. , Initialize to 1, that is ; Step 6.3, from Read the first Image ,Will Enter to In the middle, output Corresponding density estimation map ,Will Add to ,Right now and statistical images Number of people in ,Will Add to ,Right now ; Step 6.4, Judgment Is it greater than ,if If the crowd density estimation is complete, the crowd density estimation result is output; otherwise... Return to step 6.3; Step 6.5: Evaluate the network model based on the output crowd density estimation results. .

Images