Method for counting crowds in complex scene based on dynamic calibration of frequency-space characteristics

Abstract

The invention provides a crowd counting method in a complex scene based on frequency-space feature dynamic calibration, which adopts frequency-space feature dynamic calibration and a Transform-driven point query mechanism, and comprises the following steps of: firstly, adopting a data enhancement method of random overturning, scaling and cutting on a training set image to obtain a training set image; uniformly zooming to a fixed size, and inputting into a VGG-16 backbone network for multi-scale feature extraction; thirdly, fusing frequency space features output by different layers through a WTConvFeatureFusion module, and constructing a feature calibration mechanism in combination with dynamic position coding; then, inputting the calibrated feature map into a Transform encoder-decoder architecture, embedding learnable position anchor points on encoding features by using a point query generation function, and iteratively optimizing spatial distribution through a decoder; and finally, outputting a normalized coordinate point set by a prediction module, calculating classification-regression combination loss by adopting a Hungary matching algorithm, reversely optimizing model parameters, and realizing end-to-end population quantity statistics and position positioning. According to the invention, crowd counting can be accurately and effectively carried out in a complex scene.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular to a crowd counting method in complex scenes based on dynamic calibration of frequency-space features. Background Art

[0002] In recent years, dense crowd counting has shown important application value in public security monitoring, smart city management and other fields. Its core challenge lies in multi-scale adaptive modeling and occlusion robustness processing in complex scenarios. The current mainstream methods can be divided into two categories: detection-based local regression and density map-based global estimation. However, both face the following inherent bottlenecks: (1) Spatial-frequency domain feature mismatch: Traditional convolutional networks have the problem of high-frequency overfitting and low-frequency underutilization in frequency domain response, resulting in sensitivity to frequency domain interference such as sudden illumination changes and motion blur; (2) Insufficient dynamic scale perception: Existing methods process multi-scale crowds through fixed pyramids or attention mechanisms, which makes it difficult to adaptively calibrate the frequency-space feature weights from sparse to ultra-dense distribution; (3) Position regression bias accumulation: Heat map-based methods have quadratic quantization errors when decoding coordinates, while direct coordinate regression networks are susceptible to local noise interference due to the lack of frequency domain priors. It is worth noting that, unlike conventional target detection tasks, the uniqueness of dense crowd counting lies in: (1) cross-frequency domain feature coupling: effective representation requires capturing both high-frequency details (such as individual edges) and low-frequency structures (such as the overall distribution of the crowd). In traditional methods, the two are often processed by serial stacking, resulting in frequency domain conflicts;

[0003] (2) Dynamic density distribution: The same scene may have both sparse areas (dominated by low frequencies) and crowded areas (dominated by high frequencies), and the existing single path feature extraction is difficult to take both into account.

[0004] Benefiting from the progress in joint modeling of frequency domain analysis and Transformer, recent research has begun to explore solutions for joint frequency-space representation. Specifically, the WTConv module decomposes spatial features into multi-band components through wavelet transform, achieving a global receptive field with logarithmic parameter growth, significantly improving the ability to capture low-frequency shape features. Meanwhile, the Transformer's attention mechanism dynamically calibrates spatial-frequency domain weights through learnable point queries. The collaboration of the two provides new approaches for counting in complex scenes. However, achieving end-to-end dynamic calibration of frequency domain decomposition and spatial queries remains a technical challenge that urgently needs to be overcome. Summary of the Invention

[0005] The present invention proposes a crowd counting method in complex scenes based on dynamic calibration of frequency-space characteristics, which can accurately and effectively count people in complex scenes.

[0006] The present invention adopts the following technical solutions.

[0007] A crowd counting method for complex scenarios based on dynamic calibration of spatial-frequency features is proposed. The method adopts dynamic calibration of spatial-frequency features and a Transformer-driven point query mechanism. First, the training set images are subjected to data augmentation methods such as random flipping, scaling, and cropping, and uniformly scaled to a fixed size before being input into a VGG-16 backbone network for multi-scale feature extraction. Then, the spatial-frequency features output by different layers are fused through the WTConvFeatureFusion module, and a feature calibration mechanism is constructed in combination with dynamic position encoding. The calibrated feature map is then input into the Transformer encoder-decoder architecture, and a point query generation function is used to embed learnable position anchors into the encoded features. The spatial distribution is then iteratively optimized through the decoder. Finally, the prediction module outputs a normalized coordinate point set, and the Hungarian matching algorithm is used to calculate the classification-regression combined loss. The model parameters are then optimized inversely to achieve end-to-end crowd counting and location positioning.

[0008] The method comprises the following steps:

[0009] Step S1: Perform data augmentation operations such as random flipping and scaling on the input image, uniformly crop it to a fixed size, and then input it into the VGG-16 backbone network to extract multi-level convolutional features; fuse feature maps of different scales through the WTConv module to construct a feature volume with joint spatial-frequency domain representation;

[0010] Step S2: The feature map with position encoding after feature fusion is input into the Transformer encoder for global attention calculation; the spatial query point is initialized on the encoded feature through the learnable point query generation function, and the query point is embedded into the feature map with the help of position encoding;

[0011] Step S3: query points and features Figure 1 The data is then fed into the Transformer decoder for decoding, and the features output by the decoder are subjected to point classification and coordinate regression, and the normalized crowd position coordinates are output;

[0012] Step S4: Use the Hungarian algorithm to match the predicted points with the real annotations, calculate the classification loss and regression loss simultaneously, jointly optimize the classification loss and regression loss, update the network parameters through back propagation, and continuously save the optimal model according to the verification accuracy; obtain the crowd counting result corresponding to the image.

[0013] Step S1 specifically includes the following steps:

[0014] Step S11: Select a training dataset, randomly scale and flip the dataset data, and crop it into a fixed-size input image I input ∈R 3×H×W, where H represents the height of the cropped image and W represents the height of the cropped image; the image is sent to the VGG16 backbone network for feature extraction. The feature extraction network uses a series of convolution kernels of size 3×3 to obtain a larger receptive field. The receptive field is calculated as follows: F(i) = (F(i+1)-1)×Stride+Ksize

[0015] Where F(i) is the receptive field of the i-th layer, Stride is the step size of the i-th layer, and Ksize is the convolution kernel size. The three feature maps obtained after different extraction stages are obtained. and They are then fed into a feature fusion module constructed by wavelet convolution;

[0016] Step S12: Input three feature maps of different sizes and Since the number of channels of the three feature maps is different, in order to smoothly carry out the subsequent feature fusion operation, a 1×1 convolution kernel is used for dimensionality reduction processing to compress the number of channels.

[0017]

[0018] in, is the input feature map, is the convolution kernel parameter of 1×1, b(C out ) corresponds to C out The offset value of the output channel; and All compressed into and Align the number of channels of the three feature maps;

[0019] Wavelet convolution is used to construct a feature fusion module. The feature map is enhanced after three operations: wavelet decomposition WT, frequency domain convolution, and inverse wavelet reconstruction IWT. The specific formula is as follows:

[0020] Y = IWT(Conv(W, WT(X)))

[0021] Where X is the input tensor, W is the weight tensor of the depth convolution kernel, whose size is k×k, and the number of input channels is four times that of X. This operation separates the convolution and allows the small kernel to operate in a larger area of ​​the input, that is, increases its receptive field relative to the input;

[0022] First, the low-frequency and high-frequency content of the input are filtered and downscaled using WT; then, a small kernel depth convolution is performed on different frequency maps, and finally the output is constructed using IWT;

[0023]

[0024] in is the input of this layer, Represents the three high-frequency graphs described at level i; in order to combine the outputs of different frequencies, WT and its inverse transform, i.e., IWT(X+Y)=IWT(X)+IWT(Y), are used to perform

[0025] Add the convolution results at different levels, where Z (i) is the aggregate output starting from level i;

[0026] and After the dimensionality reduction operation, it is sent to wavelet convolution for enhancement, converted to the same size using upsampling operation, and then fused by addition:

[0027]

[0028] Where k is the size of the wavelet convolution kernel, UpSample(·) is the bilinear interpolation upsampling;

[0029]

[0030]

[0031] in, It is the feature map after preliminary feature fusion;

[0032]

[0033] F fuse It is the final feature map of the feature fusion module formed by wavelet convolution;

[0034] Step S13: Then position encode the obtained feature map and send it to the position encoder to generate a position vector code for each pixel in the image. The dimension of the vector is determined by the parameter num_pos_feats. First, calculate the cumulative sum of the non-masked area to get y embed and x embed , respectively, represent the position information in the y direction and x direction:

[0035] y embed =cumsum(~mask,dim=1)

[0036] x embed =cumsum(~mask,dim=w)

[0037] Among them, ~mask means to negate the mask, cumsum() means the cumulative sum along the specified dimension; then y embed and x embedNormalize it to be within the specified range; then use sine and cosine functions to generate position encoding:

[0038]

[0039] where dim t Finally, the generated pos x and pos y They are concatenated together to form a positional encoding vector and arranged into the same shape as the input feature map.

[0040] In step S11, the public crowd counting datasets ShanghaiTech, UCF-QNRF, and JHU_CROWD++ are used as training datasets.

[0041] Step S2 specifically includes the following steps:

[0042] Step S21: Construct a Transformer encoder to globally model the input feature map and position encoding; in order to balance computational efficiency and receptive field, first split the feature map into several non-overlapping windows, and then inject spatial information into the windowed features.

[0043]

[0044] Q=K=X win +P win

[0045] where h w and w w The height and width of the window are calculated, and the dependencies between features are calculated within the window to capture local context information. The attention output is added to the original features and then normalized to constrain the features:

[0046] X out =LayerNorm(X win +Attention(Q, K, V)

[0047] The ReLU activation function introduces nonlinear changes to enhance the expressiveness of features:

[0048] FFN(X)=W2(σ(W1X+b1))+b2

[0049] Where σ is the ReLU activation function, and finally the processed window features are reorganized into a complete feature map;

[0050] Step S22: Generate point queries for the feature map. First, generate evenly distributed point query coordinates on the input image according to the specified stride. Then, assign spatial position information to each point query and obtain the features of the corresponding position:

[0051] Where s is the specified stride, which controls the density of point queries and ultimately generates a set of point query coordinates.

[0052] Step S3 specifically includes the following steps:

[0053] Step S31: Construct a Transformer decoder. The decoder first divides the feature map into local windows to reduce the amount of computation. Then, it locates the target position in the feature map through point query. It uses the self-attention mechanism combined with position encoding to generate dynamic query features to achieve the effect of local interaction:

[0054]

[0055] Where Q = K = tgt + query pos , and finally align the encoder with the query features through a cross-attention mechanism:

[0056]

[0057] Where K = memory + pos, and the point query feature sequence is obtained;

[0058] Step S32: For the point query feature output by the Transformer decoder, the prediction head converts the feature dimension to num_classes+1 through a linear layer and maps out the category probability distribution:

[0059] P class =Softmax(W c ·h s +b c )

[0060] Where W c is the weight of category prediction, b c is the deviation of category prediction, h s The point query feature output by the decoder; the prediction head transforms h s Mapped to a 2D offset, the output is constrained by Sigmoid, and the final coordinates are generated by combining the initial point query coordinates.

[0061] Step S4 specifically includes the following steps:

[0062] Step S41: Match the predicted points with the real points, apply the Hungarian algorithm to the cost matrix of each sample, and find the matching pair with the minimum cost. The cost matrix consists of classification cost and coordinate cost:

[0063] cost_class i，j = -log P(pred_class i =tgt_label j )

[0064] cost_point i，j =||pred_point_abs i -tgt_points j ||2

[0065] C=α·cost_class+β·cost_point

[0066] Where α and β are weight hyperparameters, and P() is the probability of Softmax output;

[0067] Step S42: Calculate the overall model loss function Loss, the calculation formula is as follows:

[0068] Loss=Loss class +Loss point

[0069] The overall optimization goal consists of two parts, Loss class is the classification loss of the entire model, Loss point

[0070] It is the positioning coordinate loss of the entire model. The specific calculation formula is as follows:

[0071]

[0072] where y i is the true label, p i is the probability that the i-th point predicted by the model belongs to the target category, N represents the number of matching point queries, x i is the normalized real coordinate, is the normalized coordinate predicted by the model;

[0073] Iterative training is performed according to the specified parameters. The overall loss function is calculated according to the above formula and the gradient is continuously updated for iteration. The model is verified at the predetermined number of iterations according to the verification interval flag, and the best model is saved. When the number of iterations reaches the maximum, the training process ends and the best model weight is returned.

[0074] Step S43: summing the obtained prediction points to obtain the predicted number of people in the input image, and outputting the predicted number of people and the coordinates of each prediction point.

[0075] In step S43 , the output predicted number of people and the coordinates of each predicted point are displayed in a visual form on the original input image.

[0076] In step S4, the classification loss calculated simultaneously is cross entropy, and the regression loss is smooth L1.

[0077] When performing high-precision crowd counting on an original image, the method only requires point annotation of the original image to perform the counting, without the need for complex annotation of a density heat map.

[0078] Compared with the prior art, the present invention has the following beneficial effects:

[0079] 1. The crowd counting method constructed in this paper, based on dynamic calibration of frequency-space features, can effectively address multi-scale distribution and occlusion issues in complex scenarios, significantly improving counting accuracy in both dense and sparse areas. By integrating the spatial-frequency domain joint representation through the WTConv module and combining it with the global attention mechanism of the Transformer, robust modeling of crowd distribution is achieved.

[0080] 2. In view of the uniqueness of crowd counting, the present invention introduces WTConv to separate high- and low-frequency features through wavelet decomposition, and realizes the global receptive field with logarithmic parameter growth, thereby enhancing the collaborative perception of the overall distribution of low-frequency crowds and the edges of high-frequency individuals.

[0081] 3. The learnable point query of the present invention adaptively allocates frequency domain weights according to regional density (sparse / dense), solving the feature mismatch problem of traditional methods in cross-density scenarios.

[0082] 4. The present invention fully utilizes the complementary advantages of frequency domain prior and spatial dynamics: the frequency component (the overall shape of the crowd) is enhanced by WTConv to improve the robustness to illumination changes and motion blur; the high-frequency details (individual boundaries) are fine-tuned through point queries of the Transformer decoder to reduce positioning deviation. BRIEF DESCRIPTION OF THE DRAWINGS

[0083] The present invention is further described in detail below with reference to the accompanying drawings and specific embodiments:

[0084] Attachment Figure 1 It is a schematic diagram of the principle of the present invention. DETAILED DESCRIPTION

[0085] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0086] It should be noted that the following detailed descriptions are illustrative and intended to provide further explanation of the present application. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which the present application belongs.

[0087] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present application. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprise" and / or "include" are used in this specification, they indicate the presence of features, steps, operations, devices, components and / or combinations thereof.

[0088] like Figure 1 As shown in the figure, a crowd counting method in complex scenarios based on dynamic calibration of frequency-space features is proposed. The method adopts dynamic calibration of frequency-space features and a Transformer-driven point query mechanism. First, the training set images are subjected to data augmentation methods such as random flipping, scaling, and cropping, and uniformly scaled to a fixed size before being input into the VGG-16 backbone network for multi-scale feature extraction. Then, the frequency-space features output by different layers are fused through the WTConvFeatureFusion module, and a feature calibration mechanism is constructed in combination with dynamic position encoding. The calibrated feature map is then input into the Transformer encoder-decoder architecture, and a point query generation function is used to embed learnable position anchors in the encoded features. The spatial distribution is then iteratively optimized through the decoder. Finally, the prediction module outputs a normalized coordinate point set, and the Hungarian matching algorithm is used to calculate the classification-regression combination loss. The model parameters are then inversely optimized to achieve end-to-end crowd counting and location positioning.

[0089] The method comprises the following steps:

[0090] Step S1: Perform data augmentation operations such as random flipping and scaling on the input image, uniformly crop it to a fixed size, and then input it into the VGG-16 backbone network to extract multi-level convolutional features; fuse feature maps of different scales through the WTConv module to construct a feature volume with joint spatial-frequency domain representation;

[0091] Step S2: The feature map with position encoding after feature fusion is input into the Transformer encoder for global attention calculation; the spatial query point is initialized on the encoded feature through the learnable point query generation function, and the query point is embedded into the feature map with the help of position encoding;

[0092] Step S3: query points and features Figure 1The data is then fed into the Transformer decoder for decoding, and the features output by the decoder are subjected to point classification and coordinate regression, and the normalized crowd position coordinates are output;

[0093] Step S4: Use the Hungarian algorithm to match the predicted points with the real annotations, calculate the classification loss and regression loss simultaneously, jointly optimize the classification loss and regression loss, update the network parameters through back propagation, and continuously save the optimal model according to the verification accuracy; obtain the crowd counting result corresponding to the image.

[0094] Step S1 specifically includes the following steps:

[0095] Step S11: Select a training dataset, randomly scale and flip the dataset data, and crop it into a fixed-size input image I input ∈R 3×H×W , where H represents the height of the cropped image and W represents the height of the cropped image; the image is sent to the VGG16 backbone network for feature extraction. The feature extraction network uses a series of convolution kernels of size 3×3 to obtain a larger receptive field. The receptive field is calculated as follows: F(i) = (F(i+1)-1)×Stride+Ksize

[0096] Where F(i) is the receptive field of the i-th layer, Stride is the step size of the i-th layer, and Ksize is the convolution kernel size. The three feature maps obtained after different extraction stages are obtained. and They are then fed into a feature fusion module constructed by wavelet convolution;

[0097] Step S12: Input three feature maps of different sizes and Since the number of channels of the three feature maps is different, in order to smoothly carry out the subsequent feature fusion operation, a 1×1 convolution kernel is used for dimensionality reduction processing to compress the number of channels.

[0098]

[0099] in, is the input feature map, is the convolution kernel parameter of 1×1, b(C out ) corresponds to C out The offset value of the output channel; and All compressed into and Align the number of channels of the three feature maps;

[0100] Wavelet convolution is used to construct a feature fusion module. The feature map is enhanced after three operations: wavelet decomposition WT, frequency domain convolution, and inverse wavelet reconstruction IWT. The specific formula is as follows:

[0101] Y = IWT(Conv(W, WT(X)))

[0102] Where X is the input tensor, W is the weight tensor of the depth convolution kernel, whose size is k×k, and the number of input channels is four times that of X. This operation separates the convolution and allows the small kernel to operate in a larger area of ​​the input, that is, increases its receptive field relative to the input;

[0103] First, the low-frequency and high-frequency content of the input are filtered and downscaled using WT; then, a small kernel depth convolution is performed on different frequency maps, and finally the output is constructed using IWT;

[0104]

[0105] in is the input of this layer, Represents the three high-frequency graphs described at level i; in order to combine the outputs of different frequencies, WT and its inverse transform, i.e., IWT(X+Y)=IWT(X)+IWT(Y), are used to perform

[0106] Add the convolution results at different levels, where Z (i) is the aggregate output starting from level i;

[0107] and After the dimensionality reduction operation, it is sent to wavelet convolution for enhancement, converted to the same size using upsampling operation, and then fused by addition:

[0108]

[0109] Where k is the size of the wavelet convolution kernel, UpSample(·) is the bilinear interpolation upsampling;

[0110]

[0111] in, It is the feature map after preliminary feature fusion;

[0112]

[0113] F fuse It is the final feature map of the feature fusion module formed by wavelet convolution;

[0114] Step S13: Then position encode the obtained feature map and send it to the position encoder to generate a position vector code for each pixel in the image. The dimension of the vector is determined by the parameter num_pos_feats. First, calculate the cumulative sum of the non-masked area to get y embed and x embed , respectively, represent the position information in the y direction and x direction:

[0115] y embed =cumsum(~mask, dim=1)

[0116] x embed =cumsum(~mask,dim=w)

[0117] Among them, ~mask means to negate the mask, cumsum() means the cumulative sum along the specified dimension; then y embed and x embed Normalize it to be within the specified range; then use sine and cosine functions to generate position encoding:

[0118]

[0119] where dim t Finally, the generated pos x and pos y They are concatenated together to form a positional encoding vector and arranged into the same shape as the input feature map.

[0120] In step S11, the public crowd counting datasets ShanghaiTech, UCF-QNRF, and JHU_CROWD++ are used as training datasets.

[0121] Step S2 specifically includes the following steps:

[0122] Step S21: Construct a Transformer encoder to globally model the input feature map and position encoding; in order to balance computational efficiency and receptive field, first split the feature map into several non-overlapping windows, and then inject spatial information into the windowed features.

[0123]

[0124] Q=K=X win +P win

[0125] where h w and w wThe height and width of the window are calculated, and the dependencies between features are calculated within the window to capture local context information. The attention output is added to the original features and then normalized to constrain the features:

[0126] X out =LayerNorm(X win +Attention(Q, K, V)

[0127] The ReLU activation function introduces nonlinear changes to enhance the expressiveness of features:

[0128] FFN(X)=W2(σ(W1X+b1))+b2

[0129] Where σ is the ReLU activation function, and finally the processed window features are reorganized into a complete feature map;

[0130] Step S22: Generate point queries for the feature map. First, generate evenly distributed point query coordinates on the input image according to the specified stride. Then, assign spatial position information to each point query and obtain the features of the corresponding position:

[0131] Where s is the specified stride, which controls the density of point queries and ultimately generates a set of point query coordinates.

[0132] Step S3 specifically includes the following steps:

[0133] Step S31: Construct a Transformer decoder. The decoder first divides the feature map into local windows to reduce the amount of computation. Then, it locates the target position in the feature map through point query. It uses the self-attention mechanism combined with position encoding to generate dynamic query features to achieve the effect of local interaction:

[0134]

[0135] Where Q = K = tgt + query pos , and finally align the encoder with the query features through a cross-attention mechanism:

[0136]

[0137] Where K = memory + pos, and the point query feature sequence is obtained;

[0138] Step S32: For the point query feature output by the Transformer decoder, the prediction head converts the feature dimension to num_classes+1 through a linear layer and maps out the category probability distribution:

[0139] P class =Softmax(W c ·h s +b c )

[0140] Where W c is the weight of category prediction, b c is the deviation of category prediction, h s The point query feature output by the decoder; the prediction head transforms h s Mapped to a 2D offset, the output is constrained by Sigmoid, and the final coordinates are generated by combining the initial point query coordinates.

[0141] Step S4 specifically includes the following steps:

[0142] Step S41: Match the predicted points with the real points, apply the Hungarian algorithm to the cost matrix of each sample, and find the matching pair with the minimum cost. The cost matrix consists of classification cost and coordinate cost:

[0143] cost_class i，j = -log P(pred_class i =tgt_label j )

[0144] cost_point i，j =||pred_point_abs i -tgt_points j ||2

[0145] C=α·cost_class+β·cost_point

[0146] Where α and β are weight hyperparameters, and P() is the probability of Softmax output;

[0147] Step S42: Calculate the overall model loss function Loss, the calculation formula is as follows:

[0148] Loss=Loss class +Loss point

[0149] The overall optimization goal consists of two parts, Loss class is the classification loss of the entire model, Loss point It is the positioning coordinate loss of the entire model. The specific calculation formula is as follows:

[0150]

[0151] where yi is the true label, p i is the probability that the i-th point predicted by the model belongs to the target category, N represents the number of matching point queries, x i is the normalized real coordinate, is the normalized coordinate predicted by the model;

[0152] Iterative training is performed according to the specified parameters. The overall loss function is calculated according to the above formula and the gradient is continuously updated for iteration. The model is verified at the predetermined number of iterations according to the verification interval flag, and the best model is saved. When the number of iterations reaches the maximum, the training process ends and the best model weight is returned.

[0153] Step S43: summing the obtained prediction points to obtain the predicted number of people in the input image, and outputting the predicted number of people and the coordinates of each prediction point.

[0154] In step S43 , the output predicted number of people and the coordinates of each predicted point are displayed in a visual form on the original input image.

[0155] In step S4, the classification loss calculated simultaneously is cross entropy, and the regression loss is smooth L1.

[0156] When performing high-precision crowd counting on an original image, the method only requires point annotation of the original image to perform the counting, without the need for complex annotation of a density heat map.

[0157] The method described in this example leverages the multi-level wavelet decomposition characteristics of the WTConv module to achieve high-precision crowd counting with only point annotations of the original image, eliminating the need for complex annotations such as density heatmaps. Recognizing the unique nature of crowd counting tasks, this method leverages the frequency domain decomposition's ability to dynamically calibrate multi-scale features: high-frequency components accurately capture individual edge details, while low-frequency components effectively model group distribution patterns. The synergistic effect of these two components provides a key spatial-frequency domain joint representation for counting tasks in dense scenes. Existing methods typically focus only on single-scale features or employ fixed receptive fields. This method, through a learnable point query mechanism, adaptively integrates feature information from different frequency bands, significantly improving the model's ability to discriminate between sparse and dense areas. Addressing the structural characteristics of crowd distribution in complex scenes, this method innovatively combines local detail perception with global distribution modeling: WTConv's low-frequency pathway enhances shape perception, while the Transformer decoder's high-frequency pathway fine-tunes localization accuracy. The complementary advantages of these two components effectively reduce counting bias across density scenarios. The crowd counting method constructed in this paper based on dynamic calibration of frequency-space features can simultaneously achieve high-precision individual positioning and robust group statistics, and achieves excellent accuracy on benchmark datasets such as ShanghaiTech and UCF-QNRF.

[0158] The above description is only a preferred embodiment of the present invention. All equivalent changes and modifications made according to the scope of the patent application of the present invention should fall within the scope of the present invention.

Claims

1. A crowd counting method in complex scenarios based on dynamic calibration of frequency-space features, characterized by: The proposed method uses dynamic calibration of spatial-frequency features and a Transformer-driven point query mechanism. First, the training set images are subjected to data augmentation using random flipping, scaling, and cropping. The images are then uniformly scaled to a fixed size and fed into a VGG-16 backbone network for multi-scale feature extraction. Next, the spatial-frequency features output by different layers are fused using the WTConvFeatureFusion module, and a feature calibration mechanism is constructed in conjunction with dynamic position encoding. Then, the calibrated feature map is input into the Transformer encoder-decoder architecture, and the point query generation function is used to embed learnable location anchors on the encoded features. The spatial distribution is iteratively optimized through the decoder. Finally, the prediction module outputs the normalized coordinate point set, and the Hungarian matching algorithm is used to calculate the classification-regression combination loss. The model parameters are then optimized in reverse order to achieve end-to-end crowd counting and location positioning.

2. The method for counting people in complex scenes based on dynamic calibration of frequency-space characteristics according to claim 1, characterized in that: The method comprises the following steps: Step S1: Perform data augmentation operations such as random flipping and scaling on the input image, crop it to a fixed size, and then input it into the VGG-16 backbone network to extract multi-level convolutional features; The WTConv module is used to fuse feature maps of different scales to construct a feature volume with joint spatial-frequency domain representation. Step S2: The feature map with position encoding after feature fusion is input into the Transformer encoder for global attention calculation; the spatial query point is initialized on the encoded feature through the learnable point query generation function, and the query point is embedded into the feature map with the help of position encoding; Step S3: The query point and feature map are sent to the Transformer decoder for decoding. The features output by the decoder are subjected to point classification and coordinate regression, and the normalized crowd position coordinates are output. Step S4: Use the Hungarian algorithm to match the predicted points with the real annotations, calculate the classification loss and regression loss simultaneously, jointly optimize the classification loss and regression loss, update the network parameters through back propagation, and continuously save the optimal model according to the verification accuracy; obtain the crowd counting result corresponding to the image.

3. The method for counting people in complex scenes based on dynamic calibration of frequency-space characteristics according to claim 2, characterized in that: Step S1 specifically includes the following steps: Step S11: Select a training dataset, randomly scale and flip the dataset data, and crop it into a fixed-size input image I input ∈R 3×H×W , where H represents the height of the cropped image and W represents the height of the cropped image; the image is sent to the VGG16 backbone network for feature extraction. The feature extraction network uses a series of convolution kernels of size 3×3 to obtain a larger receptive field. The calculation formula of the receptive field is as follows: F(i)=(F(i+1)-1)×Stride+Ksize Where F(i) is the receptive field of the i-th layer, Stride is the step size of the i-th layer, and Ksize is the convolution kernel size. The three feature maps obtained after different extraction stages are obtained. and They are then fed into a feature fusion module constructed by wavelet convolution; Step S12: Input three feature maps of different sizes and Use 1×1 convolution kernel to reduce dimension and compress the number of channels. in, is the input feature map, is the convolution kernel parameter of 1×1, b(C out ) corresponds to C out The offset value of the output channel; and All compressed into and Align the number of channels of the three feature maps; Wavelet convolution is used to construct a feature fusion module. The feature map is enhanced after three operations: wavelet decomposition WT, frequency domain convolution, and inverse wavelet reconstruction IWT. The specific formula is as follows: Y = IWT(Conv(W, WT(X))) Where X is the input tensor, W is the weight tensor of the depth convolution kernel, whose size is k×k, and the number of input channels is four times that of X. This operation separates the convolution and allows the small kernel to operate in a larger area of ​​the input, that is, increases its receptive field relative to the input; First, the low-frequency and high-frequency content of the input are filtered and downscaled using WT; then, a small kernel depth convolution is performed on different frequency maps, and finally the output is constructed using IWT; in is the input of this layer, Represents the three high-frequency graphs described at level i; in order to combine the outputs of different frequencies, WT and its inverse transform, i.e., IWT(X+Y)=IWT(X)+IWT(Y), are used to perform Add the convolution results at different levels, where Z (i) is the aggregate output starting from level i; and After the dimensionality reduction operation, it is sent to wavelet convolution for enhancement, converted to the same size using upsampling operation, and then fused by addition: Where k is the size of the wavelet convolution kernel, UpSample(·) is the bilinear interpolation upsampling; in, It is the feature map after preliminary feature fusion; F fuse It is the final feature map of the feature fusion module formed by wavelet convolution; Step S13: Then position encode the obtained feature map and send it to the position encoder to generate a position vector code for each pixel in the image. The dimension of the vector is determined by the parameter num_pos_feats. First, calculate the cumulative sum of the non-masked area to get y embed and x embed , respectively, represent the position information in the y direction and x direction: the embed [cumsum(~mask,dim=1) x embed =cumsum(~mask,dim=w) Among them, ~mask means to negate the mask, cumsum() means the cumulative sum along the specified dimension; then y embed and x embed Normalize it to be within the specified range; then use sine and cosine functions to generate position encoding: where dim t Finally, the generated pos x and pos y They are concatenated together to form a positional encoding vector and arranged into the same shape as the input feature map.

4. The method for counting people in complex scenes based on dynamic calibration of frequency-space characteristics according to claim 3, characterized in that: In step S11, the public crowd counting datasets ShanghaiTech, UCF-QNRF, and JHU_CROWD++ are used as training datasets.

5. The method for counting people in complex scenes based on dynamic calibration of frequency-space characteristics according to claim 2, characterized in that: Step S2 specifically includes the following steps: Step S21: Construct a Transformer encoder to globally model the input feature map and position encoding; in order to balance computational efficiency and receptive field, first split the feature map into several non-overlapping windows, and then inject spatial information into the windowed features. Q=K=X win +Pw win where h w and w w The height and width of the window are calculated, and the dependencies between features are calculated within the window to capture local context information. The attention output is added to the original features and then normalized to constrain the features: X out =LayerNorm(X win +Attention(Q,K,V)) The ReLU activation function introduces nonlinear changes to enhance the expressiveness of features: FFN(X)=W2(σ(W1X+b1))+b2 Where σ is the ReLU activation function, and finally the processed window features are reorganized into a complete feature map; Step S22: Generate point queries for the feature map. First, generate evenly distributed point query coordinates on the input image according to the specified stride. Then, assign spatial position information to each point query and obtain the features of the corresponding position: Where s is the specified stride, which controls the density of point queries and ultimately generates a set of point query coordinates.

6. The method for counting people in complex scenes based on dynamic calibration of frequency-space characteristics according to claim 2, characterized in that: Step S3 specifically includes the following steps: Step S31: Construct a Transformer decoder. The decoder first divides the feature map into local windows to reduce the amount of computation. Then, it locates the target position in the feature map through point query. It uses the self-attention mechanism combined with position encoding to generate dynamic query features to achieve the effect of local interaction: Where Q = K = tgt + query pos , and finally align the encoder with the query features through a cross-attention mechanism: Where K = memory + pos, and the point query feature sequence is obtained; Step S32: For the point query feature output by the Transformer decoder, the prediction head converts the feature dimension to num_classes+1 through a linear layer and maps out the category probability distribution: P class =Softmax(W c ·h s +b c ) Where W c is the weight of category prediction, b c is the deviation of category prediction, h s The point query feature output by the decoder; the prediction head transforms h s Mapped to a 2D offset, the output is constrained by Sigmoid, and the final coordinates are generated by combining the initial point query coordinates.

7. The method for counting people in complex scenes based on dynamic calibration of frequency-space characteristics according to claim 2, characterized in that: Step S4 specifically includes the following steps: Step S41: Match the predicted points with the real points, apply the Hungarian algorithm to the cost matrix of each sample, and find the matching pair with the minimum cost. The cost matrix consists of classification cost and coordinate cost: cost_class i，j =-log P(pred_class i =tgt_label j ) cost_point i，j =||pred_point_abs i -tgt_points j ||2 C=α·cost_class+β·cost_point Where α and β are weight hyperparameters, and P() is the probability of Softmax output; Step S42: Calculate the overall model loss function Loss, the calculation formula is as follows: Loss=Loss class +Loss point The overall optimization goal consists of two parts, Loss class is the classification loss of the entire model, Loss point It is the positioning coordinate loss of the entire model. The specific calculation formula is as follows: where y i is the true label, p i is the probability that the i-th point predicted by the model belongs to the target category, N represents the number of matching point queries, x i is the normalized real coordinate, is the normalized coordinate predicted by the model; Iterative training is performed according to the specified parameters. The overall loss function is calculated according to the above formula and the gradient is continuously updated for iteration. The model is verified at the predetermined number of iterations according to the verification interval flag, and the best model is saved. When the number of iterations reaches the maximum, the training process ends and the best model weight is returned. Step S43: summing the obtained prediction points to obtain the predicted number of people in the input image, and outputting the predicted number of people and the coordinates of each prediction point.

8. The method for counting people in complex scenes based on dynamic calibration of frequency-space characteristics according to claim 7, characterized in that: In step S43 , the output predicted number of people and the coordinates of each predicted point are displayed in a visual form on the original input image.

9. The method for counting people in complex scenes based on dynamic calibration of frequency-space characteristics according to claim 7, characterized in that: In step S4, the classification loss calculated simultaneously is cross entropy, and the regression loss is smooth L1.

10. The method for counting people in complex scenes based on dynamic calibration of frequency-space characteristics according to claim 1, characterized in that: When performing high-precision crowd counting on an original image, the method only requires point annotation of the original image to perform the counting, without the need for complex annotation of a density heat map.

Images