A video crowd counting method based on frame difference priori and dynamic-static decoupling network

Abstract

This invention discloses a video crowd counting method based on a frame difference prior and a dynamic-static decoupling network. The method first performs random cropping and flipping enhancement on consecutive frames, and constructs a dual-branch network containing spatiotemporal and spatial branches. Frame difference priors are calculated based on consecutive frames and auxiliary frames, and dynamic and static guiding information is extracted through convolution. The spatiotemporal branch is used to model cross-frame dynamic features, while the spatial branch is used to extract single-frame static semantics. The two branches fuse the frame difference prior at multiple scales for feature guidance, and combine a spatiotemporal aggregation module and a gating refinement module to strengthen dynamic information modeling and static structure representation, respectively. Finally, a density map is generated through the fusion network and a regression head to achieve video crowd counting; after training, the optimal model is saved for inference. This invention effectively improves counting accuracy and model robustness through frame difference priors and dynamic-static decoupling modeling.

Description

Technical Field

[0001] This invention relates to the field of crowd counting technology, and in particular to a video crowd counting method based on a frame difference prior dynamic-static decoupling network. Background Technology

[0002] With the rapid development of intelligent surveillance systems and the increasing frequency of large-scale public events, video crowd counting has become a key technology in areas such as public safety, traffic management, and crowd control. The goal of video crowd counting is to accurately estimate the number of people in each video frame. Current mainstream methods address this problem by estimating a crowd density map and summing the density maps.

[0003] Existing research on video people counting can be broadly categorized into two types: one based on single-frame images, utilizing only spatial information and lacking temporal modeling capabilities; the other based on video sequences, modeling using multi-frame information, but typically focusing more on salient motion regions and lacking sufficient representation capabilities for static or weakly moving regions, easily leading to undercounting. Furthermore, existing methods underutilize inter-frame difference information, resulting in interference between dynamic and static information, thus affecting counting accuracy. Therefore, how to effectively utilize inter-frame difference information to achieve decoupled modeling of dynamic and static features and improve overall counting performance has become an urgent problem to be solved. Summary of the Invention

[0004] Purpose of the invention: The purpose of this invention is to provide a video crowd counting method based on frame difference prior and dynamic-static decoupling network, which utilizes the full interaction between the spatiotemporal domain and the spatial domain to accurately estimate the number of people contained in a video frame.

[0005] Technical solution: A video crowd counting method based on frame difference prior and dynamic-static decoupling network, comprising the following steps:

[0006] S1, all frames of the video are divided into training set, validation set and test set according to time sequence and a certain ratio to ensure that the video frames in each set maintain temporal continuity; during the training phase, T consecutive frames are used as a sliding window as the input sequence of the network, and the frame after the T consecutive frames is selected as an auxiliary frame to generate frame difference prior information; uniform random cropping and horizontal flipping operations are applied to the T+1 consecutive input frames in the training set to achieve data augmentation, thereby obtaining an expanded training sample set; when the sliding window moves to the end of the video sequence and causes the auxiliary frame to be missing, it is filled by moving forward one frame to ensure the integrity of the input sequence.

[0007] S2, a frame difference prior network is constructed. First, the difference information between adjacent frames is calculated based on the T consecutively input images and their auxiliary frames to obtain frame difference features. The frame difference features are mapped through three convolutional layers and a sigmoid activation function to generate a dynamic prior map reflecting the motion change region in the video sequence. The dynamic prior map is further thresholded to generate a static weight map representing the distribution of static regions. The dynamic prior map and the static weight map are then used for decoupling and guiding modeling of the dynamic and static features in subsequent networks.

[0008] S3. A dynamic-static decoupled dual-branch decoding network is constructed. Features extracted by the feature encoder are input into parallel spatiotemporal and spatial branches for decoding. The spatiotemporal branch uses a spatiotemporal aggregation module and an aggregation attention module to model cross-frame information of continuous frame features to extract dynamic spatiotemporal features. The spatial branch uses an overlapable embedding module and a spatial channel-gated linear unit to spatially refine the features to enhance static structural information. During decoding, the dynamic prior map is incremented by one and then multiplied element-wise with the spatiotemporal branch features. Simultaneously, the static weight map is inverted, incremented by two, and then multiplied element-wise with the spatial branch features. This process is repeated at different stages to achieve decoupled modeling and enhancement of dynamic and static features. Finally, the output features of the two branches are fused to generate a crowd density map for the current frame.

[0009] S4. After multiple rounds of training on a dynamic-static decoupled network that simultaneously includes frame difference priors for encoding and decoding, the network is tested on a validation set. The network weights corresponding to the best results on the validation set are saved to obtain the optimal network model. Each T frames in the test set are used as samples and input into the optimal network model to estimate the number of people contained in each T frames.

[0010] Furthermore, in step S3, the feature encoder uses the ShiftViT-T model, which removes the global average pooling layer and the fully connected layer, to extract the spatial features of the input frame. The encoder used in this invention can also be replaced by a feature encoder such as ResNet.

[0011] Furthermore, the implementation process of the frame difference prior is as follows: For a continuously input T-frame RGB video sequence I = I1, I2, ..., I... T Add an auxiliary frame I at the end of the sequence. T+1 This constitutes the extended frame sequence I1, I2,..., I... T I T+1 Then, the difference information between two adjacent frames is calculated to obtain the frame difference feature:

[0012] D i =|I i+1 -I i |, i = 1, 2, ..., T

[0013] Among them, D i This represents the frame difference map corresponding to the i-th frame, where |·| represents the pixel-wise absolute value operation, thus obtaining the frame difference set D = D1, D2, ..., D... T The frame difference features are input into a multi-layer convolutional network for feature mapping. Each convolutional layer uses a 3×3 kernel, and the feature map is progressively downsampled during convolution to reduce its spatial resolution to 1 / 8 of the original input image, adapting to the feature scale in the subsequent decoding stage. Then, a dynamic prior map is generated using the Sigmoid activation function.

[0014] P d =σ(Conv(D))

[0015] Among them, P d The dynamic prior map, with values ​​ranging from 0 to 1, is used to characterize motion-changing regions in the video sequence. Further, the dynamic prior map is thresholded to generate a static mask map.

[0016]

[0017] Where τ is a preset threshold, M s This represents a static mask image, used to characterize static or weakly moving regions in a video. Through the above process, a dynamic prior image and a static mask image can be obtained to guide subsequent dynamic-static decoupling modeling.

[0018] Furthermore, in step S3, the spatiotemporal aggregation module is implemented as follows: the spatiotemporal aggregation module takes the features output by the feature encoder as input, performs cross-frame information interaction and fusion on the features of consecutive frames to extract dynamic spatiotemporal features; the spatiotemporal aggregation module consists of two multi-column spatiotemporally depth-separable convolutional modules and an aggregation attention module, and performs feature transfer through residual connections, and its output can be expressed as:

[0019]

[0020] in, This represents two multi-column spatiotemporally separable convolution operations. This indicates a convergent attention operation. and These represent the input and output features of the spatiotemporal aggregation module, respectively. The multi-column spatiotemporal depth separable convolution module adopts a multi-branch structure, performing spatiotemporal convolution operations on the input features under different receptive fields to capture multi-scale dynamic information; the aggregation attention module is used to perform weighted fusion of the output features of each branch, thereby highlighting key dynamic regions and suppressing redundant information.

[0021] During the decoding process, multiple spatiotemporal aggregation modules are cascaded sequentially to form a spatiotemporal branch. A pixel rearrangement upsampling operation is introduced between the second and third modules to increase the feature resolution from 1 / 16 to 1 / 8, in order to match the dynamic prior map P. d The resolution remains consistent; after the upsampling operation, the features are modulated element-wise with the dynamic prior map, specifically as follows:

[0022]

[0023] Where ⊙ represents element-wise multiplication, P d Represents a dynamic prior graph, 1+P d Used to enhance dynamic region response while preserving original feature information. This represents the upsampled output of the second module in the spatiotemporal aggregation branch. This represents the input to the third module of the spatiotemporal aggregation branch. Furthermore, after the output of the last spatiotemporal aggregation module, the features are again modulated element-wise with the dynamic prior map to further enhance the representation ability of the moving region.

[0024] Further, in step S3, the implementation process of the gated spatial channel refinement module is as follows: The gated spatial channel refinement module takes the features output by the feature encoder as input, and refines the spatial structure and enhances the channels of the features to improve the representation ability of static or weakly moving regions; the gated spatial channel refinement module is composed of an overlapping embedding module and a spatial channel gated linear unit module, and performs feature transfer through residual connection, and its output can be expressed as:

[0025]

[0026] Where ε(·) denotes an overlappable embedding operation. This indicates a spatial channel-gated linear unit operation. These represent the input and output characteristics of the gated space channel module, respectively.

[0027] The overlapping embedding module performs local modeling of input features through overlapping convolution operations to preserve fine-grained spatial structure information; the spatial channel-gated linear unit introduces gating mechanisms in both spatial and channel dimensions to adaptively enhance features in important regions while suppressing redundant information.

[0028] During the decoding process, multiple gated spatial channel refinement modules are cascaded sequentially to form spatial branches, and a pixel rearrangement upsampling operation is introduced between the second and third modules to increase the feature resolution from 1 / 16 to 1 / 8, in order to match the static mask image M. sThe resolution remains consistent; after the upsampling operation, the features are modulated element-wise with the static mask image, specifically as follows:

[0029]

[0030] in, This represents the upsampled output of the second module in the gated space channel branch. This represents the input of the third module in the gated space channel branch, 2-M. s This is used to enhance the feature response of static and weakly moving regions while maintaining obvious moving regions. Furthermore, after the output of the last gated spatial channel refinement module, the features are again modulated element-wise with the static mask image to further enhance the expressive power of static regions and reduce dynamic interference.

[0031] Furthermore, in step S3, the specific structure of the fusion network is as follows: the fusion network is used to process the feature F output by the spatiotemporal branch. t Feature F of the refined branch output of the gated space channel s The data is then fused and mapped to generate the final population density map.

[0032] Specifically, the output features of the two branches are first added element-wise to obtain the fused feature F. fuse :

[0033] F fuse =F t +F s

[0034] Subsequently, the fused features are input into the density regression module, which consists of a single 1×1 convolutional layer used to perform channel compression and linear mapping on the fused features, outputting the crowd density map P of the current frame.

[0035] P = Conv 1×1 (F fuse )

[0036] Since feature fusion and channel compression have been gradually completed through multi-layer feature extraction and upsampling operations in the aforementioned encoding and decoding process, the fused features have strong semantic expressive power. Therefore, the fusion network does not need to adopt a complex structure and can achieve effective feature integration simply by adding elements one by one and 1×1 convolution.

[0037] Compared with the prior art, the significant advantages of this invention are as follows:

[0038] 1. This invention proposes a dynamic-static decoupling network based on frame difference, which fully explores the complementary information between dynamic and static regions and effectively achieves the fusion of the two features. It exhibits better performance, stronger generalization ability and higher robustness in video crowd counting tasks. Unlike traditional models that only rely on spatiotemporal interaction, this invention starts from the perspective of dynamic-static separation and constructs a unified framework that can simultaneously model spatiotemporal and spatial relationships, thereby significantly improving the accuracy of the model.

[0039] 2. This invention proposes a frame difference prior method, which explicitly extracts motion change information in the video by modeling the inter-frame differences of consecutive video frames and generating dynamic prior maps and static mask maps, thereby achieving effective differentiation between dynamic and static regions. Unlike existing methods that implicitly model temporal information, this invention guides the feature learning process by introducing frame difference priors, enabling the model to more accurately focus on motion regions and strengthen its representation ability for weakly moving and static crowds, thereby effectively reducing undercounting and miscounting in complex scenes and further improving the accuracy and stability of video crowd counting.

[0040] 3. This invention designs a spatiotemporal aggregation module, which performs multi-scale modeling of continuous frame features through multi-column spatiotemporally depth-separable convolution, and combines an aggregation attention mechanism to adaptively weight and fuse features from different branches, thereby effectively capturing complex dynamic change information in video sequences. Compared with traditional single-scale or simple temporal modeling methods, this invention can fully model cross-frame dynamic relationships under different receptive fields, and enhance feature representation ability and training stability through residual connections, thereby further improving the model's perception ability of dynamic crowd regions and overall counting accuracy.

[0041] 4. This invention proposes a gated spatial channel refinement module, which performs local spatial modeling of features through overlapping embedding and combines spatial channel gated linear units to adaptively modulate features in both spatial and channel dimensions, thereby enhancing key structural information and suppressing redundant information. Compared with traditional spatial modeling methods that rely solely on convolution or simple attention mechanisms, this invention can more effectively characterize the fine-grained structural features of static and weakly moving regions, and further improves the representation ability of low-motion crowd regions under the guidance of static masks, thereby making up for the problem of insufficient characterization of static information in temporal modeling and improving overall counting performance. Attached Figure Description

[0042] Figure 1 This is the overall flowchart of the present invention;

[0043] Figure 2 This is a diagram of the dynamic-static decoupling network framework based on frame difference prior of the present invention;

[0044] Figure 3This is a framework diagram of the spatiotemporal aggregation module of the present invention;

[0045] Figure 4 This is a framework diagram of the gated space channel refinement module of the present invention;

[0046] Figure 5 This is a comparison chart of the prediction effects of the present invention; Detailed Implementation

[0047] The present invention will now be described in further detail with reference to the accompanying drawings and specific embodiments.

[0048] like Figure 1 The diagram shown is the overall flowchart of a video people counting method based on a frame difference prior and a dynamic-static decoupling network provided by the present invention. Figure 2 The following is a diagram of the dynamic-static decoupling network framework based on frame difference prior. The implementation steps of this invention are as follows:

[0049] Step 1: Data preprocessing;

[0050] Specifically, the steps include the following:

[0051] The last 10% of the video frames in the training set were selected as the validation set. Before feature extraction, random cropping of the same region was performed on every consecutive T-frame video sequence, and the cropped region was uniformly horizontally flipped with a probability of 0.5 to achieve data augmentation.

[0052] Step two: Construct a motion-static decoupling network based on frame difference prior. This includes the following steps:

[0053] Step 21 involves performing frame difference processing on T consecutive video frames and the next auxiliary frame, followed by convolution and Sigmoid activation function processing on the resulting T frame difference images. Specifically, this includes the following steps:

[0054] For consecutive RGB video frames I∈R T×3×H×W Where 3, H, and W represent the number of channels, height, and width of the video frame, respectively. For example... Figure 2 As shown, for a continuous RGB video frame sequence I1, I2,..., I... T First, add an auxiliary frame I at the end of the sequence. T+1 This forms the extended frame sequence I1, I2,...,I T I T+1 Then, the frame difference map between two adjacent frames is calculated:

[0055] D i =|I i+1 -I i |, i = 1, 2, ..., T, (1)

[0056] Where Di Let |·| represent the frame difference map corresponding to the i-th frame, and |·| represent the pixel-wise absolute value operation. Finally, we can obtain T frame difference maps D = D1, D2, ..., D... T The frame difference map is input into three convolutional layers for feature mapping. The kernel size of each convolutional layer is set to 3×3. During convolution, the feature map is progressively downsampled, reducing its spatial resolution layer by layer until it is reduced to 1 / 8 of the original input size, thus adapting to the scale of the prediction density map output in the subsequent decoding stage. The three-layer convolutional mapping process can be represented as follows:

[0057]

[0058] Here, F3 is the prior feature map downsampled to 1 / 8 scale of the input image. Further, a dynamic prior map P is generated using the Sigmoid activation function. d =σ(F3). Where Conv(·) represents the convolution operation, σ(·) represents the Sigmoid activation function, and P... d This represents the generated dynamic prior map. The dynamic prior map has values ​​ranging from 0 to 1 and is used to characterize regions of motion change in the video sequence.

[0059] Furthermore, a static mask image is generated by thresholding the dynamic prior image:

[0060]

[0061] Where τ is a preset threshold, M s This represents a static mask image, used to characterize the distribution of static regions in a video sequence.

[0062] Step 22 involves inputting the continuously input T-frame RGB video sequence into a feature encoder for feature extraction, and then performing downsampling and upsampling operations on the feature map to obtain a feature scale that matches the subsequent decoding stage. Specifically, this includes the following steps:

[0063] For a continuously input video frame sequence I∈R T×3×H×W The input sequence is then subjected to hierarchical feature extraction via a feature encoding network. This network employs a visual Transformer-based structure to encode the input sequence in stages, gradually reducing the spatial resolution of the feature maps through downsampling during the encoding process. Ultimately, a high-level semantic feature representation with a scale of 1 / 32 of the input image is obtained. Where C represents the number of feature channels. To adapt to the feature fusion requirements of the subsequent decoding stage, the encoded features are upsampled, restoring the feature map from a 1 / 32 scale to a 1 / 16 scale. The upsampling process is implemented using a pixel shuffling method, which can be represented as follows:

[0064] Fup =PS(F enc (4)

[0065] Where PS(·) represents the pixel shuffling operation. C represents the feature map after upsampling, and C′ represents the number of feature channels after channel rearrangement.

[0066] Step 23 involves modeling the spatiotemporal aggregation branch of dynamic information in the video sequence, performing temporal modeling and feature enhancement on the features obtained in Step 22. Specifically, this includes the following steps:

[0067] The upsampled feature F up The input is processed by the spatiotemporal aggregation branch, which consists of four cascaded spatiotemporal aggregation modules. Information is transferred between these modules using residual connections to achieve stable learning and effective fusion of deep features. Between the second and third spatiotemporal aggregation modules, a pixel rearrangement upsampling operation is introduced to increase the spatial resolution of the feature map from 1 / 16 to 1 / 8, while simultaneously reducing the channel dimension, to match the dynamic prior map P generated in step 21. d Keep the resolution consistent

[0068] After the pixel rearrangement operation, the upsampled features are modulated element-wise with the dynamic prior map, specifically as follows:

[0069]

[0070] Where ⊙ represents element-wise multiplication, P d Represents a dynamic prior graph, 1+P d Used to enhance the dynamic region response while preserving the original features. This represents the upsampled output of the second module in the spatiotemporal aggregation branch. This represents the input to the third module of the spatiotemporal aggregation branch. Furthermore, after the output of the fourth spatiotemporal aggregation module, the features are again modulated element-wise with the dynamic prior map to further enhance the network's response to moving regions.

[0071] like Figure 3 As shown, the spatiotemporal aggregation module includes two multi-column spatiotemporally depth-separable convolutional modules and one aggregation attention module. The multi-column spatiotemporally depth-separable convolutional modules consist of three parallel branches, each employing depth-separable convolutions with different dilation rates to extract temporal features from different receptive fields. The aggregation attention module includes parallel global attention branches and local attention branches, used to model global contextual information and local fine-grained features respectively, and to fuse and enhance these features. Specifically, as follows:

[0072]

[0073] Where S(·) represents two multi-column spatiotemporally separable convolution operations, and A(·) represents the aggregation attention operation. These represent the input and output characteristics of the spatiotemporal aggregation module, respectively.

[0074] Step 24 involves constructing a gated spatial channel refinement branch for modeling static spatial information, and refining the spatial structure and enhancing the features obtained in Step 22. This specifically includes the following steps:

[0075] The feature F obtained in step 22 up The input is processed by the gated space channel refinement branch, which consists of four gated space channel refinement modules connected in series. The modules are connected by residual connection to transmit information.

[0076] Between the second and third gated spatial channel refinement modules, a pixel rearrangement upsampling operation is introduced to increase the spatial resolution of the feature map from 1 / 16 to 1 / 8, so as to match the static mask map M generated in step 21. s The resolution remains consistent.

[0077] After the pixel rearrangement operation, the upsampled features are modulated element-wise with the static mask image, specifically as follows:

[0078]

[0079] Here, ⊙ represents element-wise multiplication. This represents the upsampled output of the second module in the gated space channel branch. This represents the input of the third module in the gated space channel branch, 2-M. s This is used to enhance the feature responses of weakly moving and stationary regions while maintaining obvious moving regions, thereby highlighting spatial structure information. Furthermore, after the output of the fourth gated spatial channel refinement module, the features are again modulated element-wise with the static mask image to further enhance the expressive power of static regions.

[0080] like Figure 4 As shown, the gated spatial channel refinement module includes an overlapping embedding module and a spatial channel gated linear unit. The overlapping embedding module sets the stride of the convolutional layer to be smaller than the kernel size, causing adjacent receptive regions to overlap, thereby enhancing the continuity and spatial correlation between features. The spatial channel gated linear unit includes parallel depthwise separable convolutions and pointwise convolutions, used to jointly modulate the spatial and channel dimensions of the input features, thereby extracting and enhancing the static spatial structure features of the image frame. The output of the gated spatial channel refinement module can be expressed as:

[0081]

[0082] Where ε(·) denotes an overlappable embedding operation. This represents a space channel gated linear unit. These represent the input and output features of the gated spatial channel module, respectively. By refining the branches of the gated spatial channel, the spatial representation capability of weak motion regions and static regions can be effectively enhanced under the guidance of static priors, thereby compensating for the inadequacy of modeling low-motion populations by relying solely on temporal information.

[0083] Step 24 involves fusing the output features of the spatiotemporal aggregation branch and the gated spatial channel refinement branch to generate the final crowd density map. This includes the following steps:

[0084] The spatiotemporal branch output feature F obtained in step 23 is... t The spatial branch output feature F obtained in step 24 s By performing element-wise addition and fusion, the fusion feature F is obtained. fuse Subsequently, the fused features are input into the density regression head, and the features are channel compressed and mapped through a 1×1 convolutional layer to output a crowd density map of T consecutive frames.

[0085] Since the feature channel dimension has been gradually reduced and feature integration has been completed through multiple upsampling operations in the aforementioned feature encoding and decoding process, only a simple 1×1 convolution is needed in the density regression stage to achieve effective feature mapping, thereby reducing computational complexity while ensuring prediction accuracy.

[0086] Step 3: Train a motion-static decoupling network based on frame difference;

[0087] Specifically, the steps include the following:

[0088] For the ShiftViT-T model, weights pre-trained on ImageNet-1K were used for initialization. The optimizer for the frame difference-based dynamic-static decoupling network was Adam, with an initial learning rate of 1e-5, a weight decay of 1e-4, a maximum number of iterations of 300, and the model weights that performed best on the validation set were saved.

[0089] Step four: Test the static / dynamic decoupling network based on frame difference. This includes the following steps:

[0090] The video frames in the test set are divided into a sliding sequence of T consecutive frames as input, arranged chronologically. The frame following the T consecutive frames is selected as an auxiliary frame, and these are sequentially input into the optimal dynamic-static decoupling network model based on frame difference, yielding a density map Pre of T consecutive frames output by the density regression head. Subsequently, Pre is summed frame by frame to calculate the model's estimation of the number of people in the test video frames.

[0091] Depend on Figure 5 As can be seen, the static-dynamic decoupling network based on frame difference proposed in this invention can effectively characterize the spatial distribution information of crowd regions, and the generated predicted density map has high consistency with the real density map in terms of target distribution and density change trends. Even in scenarios with complex backgrounds, occlusion, and scale variations, this invention can still effectively suppress background noise interference and improve the accuracy and stability of video crowd counting results.

[0092] The above embodiments are merely illustrative of the technical concept of the present invention and should not be construed as limiting the scope of protection of the present invention. Any modifications made to the technical solutions based on the technical concept proposed in this invention shall fall within the scope of protection of this invention.

Claims

1. A video crowd counting method based on a frame difference prior dynamic-static decoupling network, characterized in that, Includes the following steps: S1: Divide the video frames into training set, validation set and test set in chronological order to maintain the temporal continuity of the video frames in each set; use T consecutive frames as the input sequence and the frame immediately following the T frames as the auxiliary frame to form a T+1 frame image sequence; when the sliding window moves to the end of the video sequence, use the last T consecutive frames as the input and select the frame preceding it as the auxiliary frame. S2: Based on the T+1 frame image sequence, the motion change information between adjacent frames in the first T frames is calculated using the logarithmic frame difference method; the motion change information is subjected to dynamic-static prior separation processing to obtain a soft dynamic prior map and a static enhancement weight map; wherein, the soft dynamic prior map is used to characterize the dynamic regions in the image, and the static enhancement weight map is used to assign enhanced feature response weights to weak motion regions and static regions. S3: Construct a dynamic-static decoupled feature encoding and decoding network; input the continuous T-frame images into the feature encoding network for multi-stage feature extraction to obtain encoded features; construct parallel spatiotemporal aggregation branches and gated spatial refinement branches to decode the encoded features; during the decoding process, after enhancing the soft dynamic prior image, it is fused with the features of the spatiotemporal aggregation branch through element-wise multiplication to enhance the feature response of dynamic regions; fuse the static mask image with the features of the gated spatial refinement branch through element-wise multiplication, and perform the fusion on the decoded features at multiple scales to achieve separation modeling of dynamic and static information and feature enhancement; finally, fuse the output features of the two branches and generate the crowd density map of the current frame through a density regression decoder; S4: During the testing phase, the continuous frame sequence is input into the trained network model to obtain the crowd density map corresponding to each frame in the video, and the sum of all pixel values ​​in the density map is used to obtain the estimated number of people in the corresponding frame.

2. The video crowd counting method based on frame difference prior and dynamic-static decoupling network according to claim 1, characterized in that, In step S2, the specific method of the dynamic-static prior separation process includes: Each logarithmic frame difference feature is input into a multi-layer convolutional network with shared weights for feature extraction. The number of channels is first increased in dimensionality and then reduced to a single channel. Then, it is mapped to the (0, 1) interval through the Sigmoid activation function to obtain a soft dynamic prior map that reflects the motion changes between frames. The soft dynamic prior map is thresholded to obtain a binarized map, and a static enhancement weight map is generated based on the binarized map to enhance weak motion regions and static regions.

3. The video crowd counting method based on frame difference prior and dynamic-static decoupling network according to claim 1, characterized in that, In step S3, the spatiotemporal aggregation branch includes: The decoding structure consists of multiple cascaded spatiotemporal aggregation modules with identical structures. Upsampling operations are set between adjacent modules to gradually restore feature resolution and achieve multi-scale spatiotemporal feature fusion. Each spatiotemporal aggregation module is formed by stacking multiple consecutive multi-branch spatiotemporal convolutional modules and then using an aggregation attention module to achieve local and global feature aggregation. The multi-branch spatiotemporal convolutional module is used to compress the input features using 1×1×1 convolution, and then extracts temporal and spatial features with different receptive fields through multiple parallel depthwise separable 3D convolutions with different dilation rates. The multi-branch features are then concatenated and restored to the original number of channels using 1×1×1 convolution to obtain a multi-scale spatiotemporal representation output. The multi-branch spatiotemporal convolutional module is applied multiple times to obtain multi-scale receptive fields and deeper aggregated spatiotemporal representations. The aggregated attention module consists of a sliding window attention branch and a spatial dimensionality reduction attention branch running in parallel. The sliding window attention branch uses a sliding window centered on the current pixel position to perform attention calculations on features within the neighborhood to extract local fine-grained texture information and short-term dynamic features. The spatial dimensionality reduction attention branch first performs spatial downsampling on the input features, and then performs attention calculations based on the dimensionality-reduced features to obtain large-scale scene context information and cross-regional dependencies. Subsequently, the attention features output by the two branches are fused to obtain spatiotemporal enhanced features that combine local detail information and global context information.

4. The video crowd counting method based on frame difference prior and dynamic-static decoupling network according to claim 1, characterized in that, In step S3, the gated spatial refinement branch is composed of several cascaded gated spatial refinement modules; multiple upsampling layers are embedded between each gated spatial refinement module to gradually restore the image resolution and realize single-frame spatial refinement feature extraction at multiple scales. The gated spatial refinement module includes an overlapping embedding module and a convolutional gated linear unit module. The overlapping embedding module sets the stride of the convolutional layer to be smaller than the kernel size, so that adjacent receptive regions overlap, thus preserving more edge texture information and local spatial continuity during feature embedding. Compared with ordinary embedding structures that use non-overlapping partitioning, the overlapping embedding module can reduce spatial information loss and enhance the correlation modeling ability between adjacent regions, thereby improving the ability to represent static structural details in complex scenes. The convolutional gated linear unit module first expands the input features through a projection layer, doubling the number of output channels. Then, the expanded features are divided into two branches along the channel dimension. The first branch, after feature reshaping, is input into parallel depthwise separable convolutions and pointwise convolutions to simultaneously extract local spatial information and cross-channel feature information. The outputs of the two convolutions are then summed element-wise, flattened, and passed through an activation function to generate gate weights. The second branch, as a linear mapping branch, retains the original context information. The gate weights are then multiplied element-wise with the second branch to achieve adaptive enhancement of key static regions and suppression of irrelevant background information. Finally, the fused features are channel-mapped through a projection layer to obtain enhanced static spatial structure features.

Images