Single-domain generalization crowd counting method based on mask-guided block-level memory alignment
By adopting a single-domain generalized crowd counting method based on mask-guided block-level memory alignment, the problems of weak generalization ability and slow inference speed of crowd counting in cross-domain scenarios are solved, and stable cross-domain counting and real-time counting capabilities are achieved, which are applicable to scenarios such as public safety monitoring and urban management.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- GUANGDONG OCEAN UNIVERSITY
- Filing Date
- 2026-05-19
- Publication Date
- 2026-07-14
AI Technical Summary
Existing crowd counting methods have weak generalization ability in cross-domain scenarios, slow inference speed, and are easily affected by annotation noise, making it difficult to meet the requirements of real-time counting.
A single-domain generalized crowd counting method based on mask-guided block-level memory alignment is adopted. Through the hierarchical linkage of feature extraction module, global density memory module, domain-specific filtering unit, pure density memory module and density classification module, the model is trained using an enhanced single-domain crowd image dataset to extract stable domain-invariant features, and the training model is optimized through multi-loss joint optimization.
It significantly improves the model's cross-domain generalization ability in unknown scenarios, reduces inference time, lowers the dependence on labeled data, is suitable for a variety of complex cross-domain scenarios, and meets the requirements for real-time counting.
Smart Images
Figure CN122391997A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the technical field of image detection, and in particular to a single-domain generalized crowd counting method based on mask-guided block-level memory alignment. Background Technology
[0002] Crowd counting technology aims to automatically estimate the number of people in images or videos using computer vision methods, and has broad application prospects in scenarios such as public safety monitoring, large-scale event management, and traffic flow scheduling.
[0003] Current crowd counting methods are mostly based on density regression, which maps the input image to a density map and obtains the crowd count through integration. Although they perform well in single scenarios, due to inter-domain differences such as variations in shooting equipment, viewpoint, lighting, and environmental background, the models often suffer from severe performance degradation and poor generalization ability in unfamiliar target scenes. Existing research also employs domain adaptation techniques, which fine-tune the model by introducing target domain data or annotations to mitigate the impact of inter-domain differences. However, these methods are highly dependent on target domain data and annotations, limiting their versatility and timeliness in practical deployment scenarios where target domain data and annotation costs are high or data privacy is restricted. To address this issue, single-domain generalization has become a new research direction. However, existing single-domain generalization methods for population counting still suffer from the following shortcomings: On the one hand, existing methods lack effective extraction and constraint mechanisms for domain-invariant features, making it impossible to stably learn universal density features unaffected by domain shifts, resulting in weak cross-domain generalization ability. On the other hand, some methods introduce complex multi-branch structures or additional auxiliary tasks to improve generalization performance, leading to slow model inference speeds and difficulty in meeting the application requirements of real-time counting. At the same time, existing methods are susceptible to annotation noise interference during training, and the model has poor robustness to noise annotations, further affecting counting accuracy and generalization stability. Summary of the Invention
[0004] To address the issues of weak cross-domain generalization ability, slow inference speed, and susceptibility to annotation noise in existing crowd counting technologies, this invention proposes a single-domain generalized crowd counting method based on mask-guided block-level memory alignment. This method can effectively improve the cross-domain generalization ability and inference speed of crowd counting, while reducing annotation noise interference.
[0005] To achieve the above-mentioned technical effects, the technical solution of the present invention is as follows: A single-domain generalized crowd counting method based on mask-guided block-level memory alignment includes the following steps: S1. Obtain the original single-domain crowd image, and preprocess the original single-domain crowd image to obtain an enhanced single-domain crowd image dataset; S2. Construct a single-domain generalized population counting model based on mask-guided block-level memory alignment. The single-domain generalized population counting model includes a feature extraction module, a global density memory module, a domain-specific filtering unit, a pure density memory module, and a density classification module connected in sequence. S3. Using the enhanced single-domain crowd image dataset, perform multi-loss joint optimization training on the single-domain generalized crowd counting model to obtain a trained single-domain generalized crowd counting model; S4. Input the single-domain crowd image of the scene to be detected into the trained single-domain generalized crowd counting model, and the trained single-domain generalized crowd counting model outputs the corrected crowd density map after filtering the background area. S5. Perform pixel-level integration on the modified crowd density map to obtain the final crowd counting result.
[0006] Preferably, the preprocessing of the original single-domain crowd image includes: S11. Perform a photometric transformation on the original single-domain crowd image, including color perturbation, Gaussian blurring, and image sharpening, to obtain a single-domain crowd photometric enhanced image. S12. Perform random cropping and random horizontal flipping operations on the original single-domain crowd image and the single-domain crowd photometric enhancement image to obtain the original preprocessed image and the single-domain crowd enhancement preprocessed image. S13. The original preprocessed image of the single-domain population and the enhanced preprocessed image of the single-domain population are combined to form the enhanced single-domain population image data.
[0007] Preferably, the feature extraction module is an encoder-decoder backbone network. The encoder-decoder backbone network receives the enhanced single-domain crowd image data, extracts the multi-scale features of the enhanced single-domain crowd image data, and fuses the multi-scale features into pixel-level features.
[0008] Preferably, the global density memory module includes a global density memory unit and a global density memory library. The global density memory module is used to process the input pixel-level features, including: S201. Input the pixel-level features into the global density memory unit and aggregate the pixel-level features into block-level features; S202. Using the global density memory, retrieve several density prototypes and calculate the cosine similarity between the block-level feature and each density prototype; S203. Based on the true labels of the block-level features, divide the block-level features into a foreground block set and a background block set, and match the corresponding density prototypes for foreground blocks of different density levels based on the cosine similarity. S204. Using a preset global density memory loss, constrain the block-level features of the foreground block set to align with the density prototypes of the corresponding density levels, constrain the block-level features of the background block set to move away from all density prototypes, and finally output pixel-level optimized features carrying global density priors.
[0009] Preferably, the calculation of the cosine similarity between the block-level features and each density prototype is... as follows:
[0010] in, Represents block-level features. Represents the j-th density prototype. Representing block-level features L2 norm, Represents the j-th density prototype The L2 norm.
[0011] Preferably, the domain-specific filtering unit is used to process the input pixel-level optimized features carrying global density priors, including: S211. Input the pixel-level optimized features carrying global density priors into the domain-specific filtering unit, and calculate the pixel-level feature difference mask and the block-level feature difference mask respectively. S212. Upsample the block-level feature difference mask to the same size as the pixel-level feature difference mask and then fuse it with the pixel-level feature difference mask to obtain a reliability mask; S213. Apply the reliability mask to the pixel-level optimized features using the Hadamard product to separate the domain-invariant features and the domain-specific features.
[0012] Preferably, the pure density memory module includes a pure density projection unit, a pure density memory unit, and a pure density memory bank. The pure density memory module is used to process the input domain-invariant features, including: S221. Input the domain-invariant features into the pure density projection unit, and project the domain-invariant features onto the latent subspace through the pure density projection unit to obtain the projected features; S222. Input the projection features into the pure density memory unit, aggregate the projection features using the pure density memory unit, and output aggregated block-level features; S223. Call the preset pure density prototype in the pure density memory, use the preset pure density memory loss to constrain the aggregated block-level features of the foreground block set to align with the pure density prototype, constrain the aggregated block-level features of the background block set to move away from the pure density prototype, and finally output the refined domain invariant features.
[0013] Preferably, the density classification module includes an auxiliary density-aware classification head and a density regression head. The density classification module is used to process the input refined domain-invariant features, including: S231. Input the refined domain-invariant features into the auxiliary density-aware classification head, use the auxiliary density-aware classification head to predict the population counting interval of the refined domain-invariant features, and use cross-entropy loss for supervision and constraint, and output semantically complete optimized domain-invariant features. S232. Input the optimized domain invariant features into the density regression head, output a preliminary density map, multiply the preliminary density map pixel by pixel with the binarized spatial mask, filter the background area, and output a corrected crowd density map.
[0014] Preferably, the steps for obtaining the spatial mask are as follows: S2321. Input the pixel-level features output by the feature extraction module into a preset block-level classification branch, and output a block-level classification map from the block-level classification branch; S2322. Binarize the block-level classification image to obtain the spatial mask of the filtered background region.
[0015] Preferably, the total loss function for multi-loss joint optimization training of the single-domain generalized crowd counting model is calculated as follows:
[0016] in, Represents the total loss function. This represents the density regression loss of the original single-domain crowd image. This represents the density regression loss for enhancing single-domain crowd image data. This represents the global density memory loss of the original single-domain crowd image. This represents the global density memory loss for enhancing single-domain crowd image data. This represents the pure density memory loss of the original single-domain crowd image. This represents the pure density memory loss used to enhance single-domain crowd image data. The weight coefficients represent the density-aware classification loss. This represents the auxiliary density-aware classification loss of the original single-domain crowd image. This represents the auxiliary density-aware classification loss for enhancing single-domain crowd image data. The weighting coefficients represent the block-level classification loss. This represents the block-level classification loss of the original single-domain crowd image. This represents the block-level classification loss for enhancing single-domain crowd image data. The weight coefficients representing the similarity alignment loss are... This represents the similarity alignment loss of the original single-domain crowd image. This represents the similarity alignment loss for enhancing single-domain crowd image data. The weighting coefficients represent the mask prediction loss. This represents the mask prediction loss of the original single-domain crowd image. This represents the mask prediction loss for enhancing single-domain crowd image data.
[0017] Compared with the prior art, the beneficial effects of the technical solution of the present invention are: This invention proposes a single-domain generalized crowd counting method based on mask-guided block-level memory alignment. First, it trains a single-domain generalized crowd counting model based on mask-guided block-level memory alignment using an enhanced single-domain crowd image dataset. The training data does not require target domain data or manual annotation. Through the hierarchical linkage of the feature extraction module, global density memory module, domain-specific filtering unit, pure density memory module, and density classification module within the single-domain generalized crowd counting model, and relying on the mask-guided block-level memory alignment mechanism, it effectively mines and constrains stable domain-invariant features, significantly improving the model's cross-domain generalization ability in unknown scenarios. Second, the global density memory module and pure density memory module are activated only during training to learn density prototypes and guide feature alignment; no memory query is required during inference, avoiding additional computational overhead. Furthermore, the enhanced single-domain crowd image dataset is used as the single-source domain data for training, eliminating the need for target domain data or multi-source domain data. This avoids the data acquisition and annotation costs of domain adaptation methods and circumvents distribution conflict problems in multi-source domain generalization, thus broadening its applicability. Attached Figure Description
[0018] Figure 1 This is a flowchart illustrating a single-domain generalized crowd counting method based on mask-guided block-level memory alignment proposed in an embodiment of the present invention. Figure 2 This is a flowchart illustrating the preprocessing of the original single-domain crowd image proposed in this embodiment of the invention. Figure 3 This diagram illustrates the structure of the single-domain generalized population counting model proposed in this embodiment of the invention. Figure 4 This is a flowchart illustrating the process of processing input pixel-level features as proposed in this embodiment of the invention. Figure 5 A visualization of the reliability mask proposed in the embodiments of the present invention; Figure 6 This is a flowchart illustrating the process of processing the input pixel-level optimized features carrying global density priors, as proposed in this embodiment of the invention. Figure 7 This is a flowchart illustrating the process of processing the domain-invariant features of the input as proposed in this embodiment of the invention. Figure 8 This is a flowchart illustrating the process of processing the refined domain-invariant features of the input as proposed in this embodiment of the invention. Detailed Implementation
[0019] The accompanying drawings are for illustrative purposes only and should not be construed as limiting the scope of this patent. It is understandable to those skilled in the art that some well-known details may be omitted from the accompanying drawings; The technical solution of the present invention will be further described below with reference to the accompanying drawings and embodiments.
[0020] Example 1 like Figure 1 As shown, this embodiment proposes a single-domain generalized crowd counting method based on mask-guided block-level memory alignment, including the following steps: S1. Obtain the original single-domain crowd image, and preprocess the original single-domain crowd image to obtain an enhanced single-domain crowd image dataset; See Figure 2 The preprocessing of the original single-domain crowd image includes: S11. Perform a photometric transformation on the original single-domain crowd image, including color perturbation, Gaussian blurring, and image sharpening, to obtain a single-domain crowd photometric enhanced image. S12. Perform random cropping and random horizontal flipping operations on the original single-domain crowd image and the single-domain crowd photometric enhancement image to obtain the original preprocessed image and the single-domain crowd enhancement preprocessed image. S13. The original preprocessed image of the single-domain population and the enhanced preprocessed image of the single-domain population are combined to form the enhanced single-domain population image data.
[0021] S2. Construct a single-domain generalized population counting model based on mask-guided block-level memory alignment. The single-domain generalized population counting model includes a feature extraction module, a global density memory module, a domain-specific filtering unit, a pure density memory module, and a density classification module connected in sequence. S3. Using the enhanced single-domain crowd image dataset, perform multi-loss joint optimization training on the single-domain generalized crowd counting model to obtain a trained single-domain generalized crowd counting model; S4. Input the single-domain crowd image of the scene to be detected into the trained single-domain generalized crowd counting model, and the trained single-domain generalized crowd counting model outputs the corrected crowd density map after filtering the background area. S5. Perform pixel-level integration on the modified crowd density map to obtain the final crowd counting result.
[0022] In this embodiment, the training of a single-domain generalized crowd counting model based on mask-guided block-level memory alignment is first completed using an enhanced single-domain crowd image dataset. The training data does not require target domain data or manual annotation. Through the hierarchical linkage of the feature extraction module, global density memory module, domain-specific filtering unit, pure density memory module, and density classification module within the single-domain generalized crowd counting model, and relying on the mask-guided block-level memory alignment mechanism, stable domain-invariant features are effectively mined and constrained, significantly improving the model's cross-domain generalization ability in unknown scenarios. Secondly, the global density memory module and pure density memory module are activated only during training to learn density prototypes and guide feature alignment; no memory query is required during inference, avoiding additional computational overhead. Furthermore, the enhanced single-domain crowd image dataset is used as a single-source domain data for training, eliminating the need for target domain data or multi-source domain data. This avoids the data acquisition and annotation costs of domain adaptation methods and circumvents distribution conflict issues in multi-source domain generalization, resulting in a wider range of applications.
[0023] Example 2 This embodiment further illustrates the single-domain generalized crowd counting model based on mask-guided block-level memory alignment proposed in the above embodiments. See also Figure 3 The single-domain generalized crowd counting model comprises a feature extraction module, a global density memory module (GDM), a domain-specific filtering unit (DSFU), a pure density memory module (PDM), and a density classification module connected in sequence. The domain-specific filtering unit utilizes the block-level density prior provided by the GDM to guide the generation of a reliability mask that integrates pixel-level and block-level features. Both memory modules (GDM+PDM) employ a block-level feature aggregation strategy instead of directly storing pixel-level features susceptible to noise contamination. The memory modules are activated only during training and do not require memory lookup during inference, thus achieving a simultaneous improvement in generalization performance and inference efficiency.
[0024] The feature extraction module is an encoder-decoder backbone network. The encoder-decoder backbone network receives the enhanced single-domain crowd image data, extracts the multi-scale features of the enhanced single-domain crowd image data, and fuses the multi-scale features into pixel-level features.
[0025] The global density memory module includes a global density memory unit (GDMU) and a global density memory bank (GDMB). The global density memory module is used to process the input pixel-level features. (See [link to relevant documentation]). Figure 4 ,include: S201. Input the pixel-level features into the global density memory unit and aggregate the pixel-level features into block-level features; S202. Using the global density memory, retrieve K=24 learnable density prototypes, each corresponding to a specific density level, and calculate the cosine similarity between the block-level feature and each density prototype; the calculation of the cosine similarity between the block-level feature and each density prototype... as follows:
[0026] in, Represents block-level features. Represents the j-th density prototype. Representing block-level features L2 norm, Represents the j-th density prototype The L2 norm; S203. Based on the true labels of the block-level features, divide the block-level features into foreground block sets. and background block set , This represents the i-th block-level feature in the foreground block set. This represents the number of block-level features in the foreground block set. This represents the i-th block-level feature in the background block set. This represents the number of block-level features in the background block set; and based on the cosine similarity, it matches corresponding density prototypes for foreground blocks of different density levels; that is, a density prototype with high similarity corresponds to a high matching density level; in the foreground block set... A density level is assigned based on its density value. This establishes a one-to-one correspondence between density levels and density prototypes in the memory bank.
[0027] S204. Using a preset global density memory loss, constrain the block-level features of the foreground block set to align with the density prototypes of the corresponding density levels, constrain the block-level features of the background block set to move away from all density prototypes, and finally output pixel-level optimized features carrying global density priors.
[0028] In S204, the block-level features of the foreground block set are aligned with the density prototypes of the corresponding density levels, while the block-level features of the background block set are far removed from all density prototypes, thereby providing a stable global density prior as follows:
[0029] in, Indicates global density memory loss. This represents the first loss of the background block set. Indicates the first loss of the foreground block set; See Figure 5The domain-specific filtering unit aims to generate a reliability mask that separates domain-invariant and domain-specific components from density features. The generation of the reliability mask is inspired by the instance normalization contrast approach, which compares the original input... and its enhanced version The instance normalization (IN) feature is used to filter out domain-related content, resulting in a binarized mask. (in It is a threshold used to determine whether the error value reflects inconsistencies in content information caused by domain offset.
[0030] To improve robustness, the domain-specific filtering unit incorporates a global density prior provided by the global density memory module to guide mask generation. The domain-specific filtering unit processes the input pixel-level optimized features carrying the global density prior; see [link to relevant documentation]. Figure 6 ,include: S211. Input the pixel-level optimized features carrying the global density prior into the domain-specific filtering unit to calculate the pixel-level feature difference mask. Block-level feature difference mask ; S212. Upsample the block-level feature difference mask to the same size as the pixel-level feature difference mask, and then fuse it with the pixel-level feature difference mask to obtain a reliability mask. Reliability mask The calculation expression is as follows:
[0031] S213. Apply the reliability mask to the pixel-level optimized features using the Hadamard product to separate the domain-invariant features and domain-specific features. The Hadamard product represents element-wise multiplication and is used for element-wise weighting of the mask and feature map; the domain-invariant features are... Corresponding regions, and using their complementary masks (1- Capture domain-specific features; to make the mask generation process learnable, this invention designs a reliability mask. A supervised mask prediction head that generates prediction masks during training using binary cross-entropy (BCE) loss. :
[0032] This allows the network to predict the reliability mask without auxiliary input during inference.
[0033] The pure density memory module refines the domain-invariant features generated by the domain-specific filtering unit by aligning them with the pure density memory. The pure density memory module includes a pure density projection unit (PDPU), a pure density memory unit (PDMU), and a pure density memory (PDMB). The pure density memory uses the same convolutional structure as the global density memory. The pure density memory unit stores K (K=24) pure density prototypes, with a structure consistent with the global density memory unit. During training, the pure density memory module processes the input domain-invariant features, see [link to relevant documentation]. Figure 7 ,include: S221. Input the domain-invariant features into the pure density projection unit, and project the domain-invariant features onto the latent subspace through the pure density projection unit to obtain the projected features; the purpose is to reduce sparsity and enhance feature representation capability. S222. Input the projection features into the pure density memory unit, aggregate the projection features using the pure density memory unit, and output aggregated block-level features; S223. Call the preset pure density prototype in the pure density memory, use the preset pure density memory loss to constrain the aggregated block-level features of the foreground block set to align with the pure density prototype, constrain the aggregated block-level features of the background block set to move away from the pure density prototype, and finally output the refined domain invariant features.
[0034] The training objective of the pure density memory module follows the same formula as that of the global density memory module:
[0035] in, This represents pure density memory loss. This represents the second loss of the background block set. Indicates the second loss of the foreground block set; The density classification module includes an auxiliary density-sensing classification head and a density regression head. To ensure that the domain-invariant features of the projection retain their original density semantics, this invention employs an auxiliary density-aware classification head. This head predicts a coarse population count interval rather than regressing the precise density. This invention uses cross-entropy (CE) loss as the classification method for the count intervals in the supervised labeled data.
[0036] in, This represents the auxiliary density-aware classification loss, also known as the cross-entropy (CE) loss. It is the true probability distribution of the counting interval. It predicts the probability distribution.
[0037] The density classification module is used to process the input refined domain-invariant features, see [link to relevant documentation]. Figure 8 ,include: S231. Input the refined domain-invariant features into the auxiliary density-aware classification head, use the auxiliary density-aware classification head to predict the population counting interval of the refined domain-invariant features, and use cross-entropy loss for supervision and constraint, and output semantically complete optimized domain-invariant features. S232. Input the optimized domain invariant features into the density regression head, output a preliminary density map, multiply the preliminary density map pixel by pixel with the binarized spatial mask, filter the background area, and output a corrected crowd density map.
[0038] The steps for obtaining the space mask are as follows: S2321. Input the pixel-level features output by the feature extraction module into a preset block-level classification branch, and output a block-level classification map from the block-level classification branch; S2322. Binarize the block-level classification image to obtain the spatial mask of the filtered background region.
[0039] Although the global density memory module and the pure density memory module provide stable density priors through similarity modeling based on density prototypes, the original features and their enhanced versions may still produce different similarity distributions in the memory bank when faced with the same potential density pattern. To address the inconsistency in the similarity distributions of the original and enhanced features in the memory bank, this invention proposes Similarity Alignment Loss (SAL), which calculates the similarity distributions of the original features and their enhanced versions in the memory banks of the global density memory module and the pure density memory module (SAL). , For the similarity distribution of GDM, , The Euclidean distance between the similarity distributions of PDMs is used to maintain the consistency of the distributions through loss constraints, thus stabilizing prototype learning.
[0040]
[0041] In this embodiment, a single-domain generalized crowd counting method based on mask-guided block-level memory alignment is provided. This method uses single-source domain data as the training basis, extracts multi-scale image features through an encoder-decoder backbone network and fuses them into pixel-level features. It provides stable structural cues by using two memory-based density prior modules: a global density memory module and a pure density memory module. A reliability mask is generated by a domain-specific filtering unit to separate domain-invariant features from domain-specific features. Similarity alignment loss (SAL) and density-aware constraint (DAC) are used to ensure the stability of prototype learning and semantic integrity. Finally, the model is jointly optimized through multiple losses. During inference, there is no need to access the memory bank. The total number of people is obtained by directly outputting the density map from the optimized model and integrating it.
[0042] This approach, through mask-guided memory interaction, progressively aligns and purifies features, effectively addressing the core issues of existing single-domain generalized crowd counting methods, such as weak cross-domain generalization ability, slow inference speed, and susceptibility to annotation noise. It significantly improves the model's robustness across different shooting devices, perspectives, and environments. In various complex cross-domain scenarios, its counting accuracy and inference efficiency demonstrate significant advantages, achieving a simultaneous leap in performance and speed compared to traditional single-domain generalization methods. Furthermore, this solution does not rely on target domain data or multi-source domain data, has minimal additional parameters and computational overhead, and combines high efficiency with practicality. It can be deployed in public safety monitoring equipment, urban management terminals, and ecological monitoring systems to meet cross-scenario crowd counting needs.
[0043] The single-domain generalized crowd counting method based on mask-guided block-level memory alignment proposed in this embodiment has the following core effects and advantages: 1. Enhanced Cross-Domain Generalization Ability: By employing a memory-mask-memory architecture, leveraging block-level density priors (with strong noise resistance) provided by the global density memory module and the pure density memory module, and combining this with the domain feature separation mechanism of the domain-specific filtering unit, stable domain-invariant features are effectively mined, resisting the dual impact of annotation noise and domain offset. In multiple cross-domain benchmark tests, the counting accuracy of this method outperforms existing domain generalization methods.
[0044] 2. Inference Speed Optimization: The global density memory module and the pure density memory module are activated only during training to learn density prototypes and guide feature alignment. No memory lookup is required during inference, avoiding additional computational overhead. Compared to methods like DCCUS and MPCount, which require memory lookups, this method offers faster inference speeds and is more suitable for real-time deployment scenarios.
[0045] 3. Reduced data dependency: It only relies on single source domain data for training, without the need for target domain data or multi-source domain data, avoiding the data collection and labeling costs of domain adaptation methods, while avoiding the distribution conflict problem in multi-source domain generalization, and has a wider range of applications.
[0046] 4. Lightweight and highly practical model: Using VGG16-BN as the backbone network, the additional modules (global density memory module, domain-specific filtering unit, pure density memory module, etc.) have fewer parameters, and the overall computational overhead is controllable. It is suitable for deployment in resource-constrained scenarios such as monitoring equipment and embedded terminals, and can meet the practical application needs of public safety, urban management, ecological monitoring, etc.
[0047] Example 3 To achieve generalized crowd counting in cross-domain scenarios, this embodiment describes the steps of jointly optimizing and training the single-domain generalized crowd counting model using the enhanced single-domain crowd image dataset with multiple losses. The specific steps are as follows: Step 1: Input the enhanced single-domain crowd image dataset into the encoder-decoder backbone network (VGG16-BN) to extract multi-scale features and fuse them into pixel-level features.
[0048] Step 2: Input pixel-level features into the global density memory module. First, aggregate the pixel-level features into block-level features through grouped downsampling convolution of the global density memory unit. Then, calculate the cosine similarity between the block-level features and the density prototypes in the global density memory. Divide the blocks into foreground and background sets according to the true labels. Constrain the foreground blocks to align with the corresponding density prototypes and the background blocks to move away from all prototypes through global density memory loss, providing global density prior regularization for pixel-level features.
[0049] Step 3: Domain-Specific Feature Filtering. The pixel-level features of the enhanced single-domain crowd image data, after global density prior regularization, are input into the domain-specific filtering unit. Pixel-level and block-level difference masks are calculated separately, and the resulting masks are fused to obtain the reliability mask. Then, the mask prediction head is learned and generated by supervising the binary cross-entropy loss. and through Hadama accumulation It operates on pixel-level features to separate domain-invariant features from domain-specific features.
[0050] Step 4: Input the domain-invariant features output by the domain-specific filtering unit into the pure density memory module, project them to the latent subspace through the pure density projection unit, then use the pure density memory unit to aggregate the projected features into block-level features, then align the block-level features with the pure density prototypes in the pure density memory library, constrain the feature alignment through pure density memory loss, and further obtain refined domain-invariant features.
[0051] Step 5: Semantic Integrity Constraints. An auxiliary density-aware classification head is used to predict the counting intervals of the domain-invariant features processed by the pure density memory module. Cross-entropy loss is employed for supervision to ensure the semantic integrity of the feature density.
[0052] Step Six: Similarity Distribution Alignment. Calculate the similarity distribution of the original features and the enhanced features on the global density memory and the pure density memory. Constrain the two distributions to be consistent using SAL loss to stabilize prototype learning.
[0053] Step 7: Input the pixel-level features output by the feature extraction module into the PC branch, which predicts a block-level classification map (PCM). Made from real PCM Supervision is provided through binary cross-entropy (BCE) loss:
[0054] The predicted PCM is binarized to obtain a spatial mask. It is related to the preliminary density map output by the density branch. Perform pixel-by-pixel multiplication to filter out the background area.
[0055]
[0056] Step 8: Use the density map after filtering the background area. Compared to real images The pixel-level mean square error (MSE) is used to supervise the density regression head.
[0057] Step Nine: Joint Optimization with Multiple Losses. A joint optimization approach using multiple losses is employed, including GDM loss, PDM loss, block classification (PC) loss, similarity distribution alignment (SAL) loss, DSFU mask prediction loss, and density regression loss. The entire framework is trained end-to-end to ensure the model can simultaneously learn robust domain-invariant features, stable density prototype representations, and accurate density regression capabilities. The final total loss function for the single-domain generalized crowd counting model after joint optimization with multiple losses is calculated as follows:
[0058] in, Represents the total loss function. This represents the density regression loss of the original single-domain crowd image. This represents the density regression loss for enhancing single-domain crowd image data. This represents the global density memory loss of the original single-domain crowd image. This represents the global density memory loss for enhancing single-domain crowd image data. This represents the pure density memory loss of the original single-domain crowd image. This represents the pure density memory loss used to enhance single-domain crowd image data. The weight coefficients represent the density-aware classification loss. This represents the auxiliary density-aware classification loss of the original single-domain crowd image. This represents the auxiliary density-aware classification loss for enhancing single-domain crowd image data. The weighting coefficients represent the block-level classification loss. This represents the block-level classification loss of the original single-domain crowd image. This represents the block-level classification loss for enhancing single-domain crowd image data. The weight coefficients representing the similarity alignment loss are... This represents the similarity alignment loss of the original single-domain crowd image. This represents the similarity alignment loss for enhancing single-domain crowd image data. The weighting coefficients represent the mask prediction loss. This represents the mask prediction loss of the original single-domain crowd image. This represents the mask prediction loss for enhancing single-domain crowd image data.
[0059] Example 4 This embodiment describes a training method for a single-domain generalized crowd counting model. The single-domain generalized crowd counting model structure includes a feature extractor, a density regression head, an auxiliary density-aware classification head, a global density memory (GDM) module, a domain-specific filtering unit (DSFU), a pure density memory (PDM) module, and a density classification module. The domain-specific filtering unit incorporates a global density prior provided by the global density memory module to guide mask generation. The model structure of this invention also designs a mask prediction head, which generates a predicted mask during training using binary cross-entropy (BCE) loss. The feature extractor employs a VGG16-BN encoder-decoder architecture to extract multi-layer features from the input image and ultimately generate density and classification features. The density regression head consists of a 1×1 convolutional layer, which outputs a density map; the classification head consists of two convolutional layers and a dropout layer (dropout rate of 0.3), which predicts whether a 16×16 region contains a target object, and the output is processed by a sigmoid activation function.
[0060] The mask prediction head (err_head) contains a 1×1 convolutional layer and a sigmoid activation function to generate a binary mask to distinguish between domain-invariant features and domain-specific features.
[0061] The memory module comprises two memories: the Global Density Memory (GDMB) and the Pure Density Memory (PDMB), both with dimensions of 24×256, used to store feature prototypes at different density levels. The model achieves density level classification (24 levels) by calculating the cosine similarity between block-level features and density prototypes in the memory, and employs contrastive learning loss to optimize feature representation.
[0062] In addition, to enhance the diversity of input data, photometric transformations are applied to the input image, including color jittering, Gaussian blur, and sharpening; at the same time, image patches of 320×320 size are randomly cropped from the input image, and random horizontal flipping (probability 0.5) is applied to the original image and the enhanced image.
[0063] During training, the method employs the AdamW optimizer with a learning rate set to 1×10⁻⁶. - ³, the weight decay coefficient is 1×10 -4 It also incorporates the OneCycleLR learning rate scheduling strategy, where the maximum learning rate is set to 1×10. - ³, The total number of training rounds is set to 300, with 15 steps per round. The threshold for filtering density features using the instance normalization mask is set to 0.5. The batch size is set to 16.
[0064] The weights for each loss function are set as follows: The coefficient of the classification loss... Set to 10; coefficient of DSFU mask prediction loss Set to 10; coefficient of contrastive learning loss for the memory bank. Set to 1; coefficient of consistency loss Set to 10; coefficient of density interval classification loss Set to 10. All density regression losses use mean squared error (MSE) loss, and classification losses use binary cross-entropy loss.
[0065] The training experiments in this embodiment were conducted on a single NVIDIA RTX 4090 GPU with 24 GB of video memory. The PyTorch framework was used for implementation.
[0066] With the above configuration, this embodiment can effectively improve the model's crowd counting performance in single-domain generalization scenarios while ensuring training stability.
[0067] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the present invention, and are not intended to limit the implementation of the present invention. Those skilled in the art can make other variations or modifications based on the above description. It is neither necessary nor possible to exhaustively describe all embodiments here. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the claims of the present invention.
Claims
1. A single-domain generalized crowd counting method based on mask-guided block-level memory alignment, characterized in that, Includes the following steps: S1. Obtain the original single-domain crowd image, and preprocess the original single-domain crowd image to obtain an enhanced single-domain crowd image dataset; S2. Construct a single-domain generalized population counting model based on mask-guided block-level memory alignment. The single-domain generalized population counting model includes a feature extraction module, a global density memory module, a domain-specific filtering unit, a pure density memory module, and a density classification module connected in sequence. S3. Using the enhanced single-domain crowd image dataset, perform multi-loss joint optimization training on the single-domain generalized crowd counting model to obtain a trained single-domain generalized crowd counting model; S4. Input the single-domain crowd image of the scene to be detected into the trained single-domain generalized crowd counting model, and the trained single-domain generalized crowd counting model outputs the corrected crowd density map after filtering the background area. S5. Perform pixel-level integration on the modified crowd density map to obtain the final crowd counting result.
2. The single-domain generalized crowd counting method based on mask-guided block-level memory alignment according to claim 1, characterized in that, The preprocessing of the original single-domain crowd image includes: S11. Perform a photometric transformation on the original single-domain crowd image, including color perturbation, Gaussian blurring, and image sharpening, to obtain a single-domain crowd photometric enhanced image. S12. Perform random cropping and random horizontal flipping operations on the original single-domain crowd image and the single-domain crowd photometric enhancement image to obtain the original preprocessed image and the single-domain crowd enhancement preprocessed image. S13. The original preprocessed image of the single-domain population and the enhanced preprocessed image of the single-domain population are combined to form the enhanced single-domain population image data.
3. The single-domain generalized crowd counting method based on mask-guided block-level memory alignment according to claim 1, characterized in that, The feature extraction module is an encoder-decoder backbone network. The encoder-decoder backbone network receives the enhanced single-domain crowd image data, extracts the multi-scale features of the enhanced single-domain crowd image data, and fuses the multi-scale features into pixel-level features.
4. The single-domain generalized crowd counting method based on mask-guided block-level memory alignment according to claim 1, characterized in that, The global density memory module includes a global density memory unit and a global density memory library. The global density memory module is used to process the input pixel-level features, including: S201. Input the pixel-level features into the global density memory unit and aggregate the pixel-level features into block-level features; S202. Using the global density memory, retrieve several density prototypes and calculate the cosine similarity between the block-level feature and each density prototype; S203. Based on the true labels of the block-level features, divide the block-level features into a foreground block set and a background block set, and match the corresponding density prototypes for foreground blocks of different density levels based on the cosine similarity. S204. Using a preset global density memory loss, constrain the block-level features of the foreground block set to align with the density prototypes of the corresponding density levels, constrain the block-level features of the background block set to move away from all density prototypes, and finally output pixel-level optimized features carrying global density priors.
5. The single-domain generalized crowd counting method based on mask-guided block-level memory alignment according to claim 4, characterized in that, The calculation of the cosine similarity between the block-level features and each density prototype is described. as follows: in, Represents block-level features. Represents the j-th density prototype. Representing block-level features L2 norm, Represents the j-th density prototype The L2 norm.
6. The single-domain generalized crowd counting method based on mask-guided block-level memory alignment according to claim 4, characterized in that, The domain-specific filtering unit is used to process the input pixel-level optimized features carrying global density priors, including: S211. Input the pixel-level optimized features carrying global density priors into the domain-specific filtering unit, and calculate the pixel-level feature difference mask and the block-level feature difference mask respectively. S212. Upsample the block-level feature difference mask to the same size as the pixel-level feature difference mask and then fuse it with the pixel-level feature difference mask to obtain a reliability mask; S213. Apply the reliability mask to the pixel-level optimized features using the Hadamard product to separate the domain-invariant features and the domain-specific features.
7. The single-domain generalized crowd counting method based on mask-guided block-level memory alignment according to claim 6, characterized in that, The pure density memory module includes a pure density projection unit, a pure density memory unit, and a pure density memory bank. The pure density memory module is used to process the input domain-invariant features, including: S221. Input the domain-invariant features into the pure density projection unit, and project the domain-invariant features onto the latent subspace through the pure density projection unit to obtain the projected features; S222. Input the projection features into the pure density memory unit, aggregate the projection features using the pure density memory unit, and output aggregated block-level features; S223. Call the preset pure density prototype in the pure density memory, use the preset pure density memory loss to constrain the aggregated block-level features of the foreground block set to align with the pure density prototype, constrain the aggregated block-level features of the background block set to move away from the pure density prototype, and finally output the refined domain invariant features.
8. The single-domain generalized crowd counting method based on mask-guided block-level memory alignment according to claim 7, characterized in that, The density classification module includes an auxiliary density-aware classification head and a density regression head. The density classification module is used to process the input refined domain-invariant features, including: S231. Input the refined domain-invariant features into the auxiliary density-aware classification head, use the auxiliary density-aware classification head to predict the population counting interval of the refined domain-invariant features, and use cross-entropy loss for supervision and constraint, and output semantically complete optimized domain-invariant features. S232. Input the optimized domain invariant features into the density regression head, output a preliminary density map, multiply the preliminary density map pixel by pixel with the binarized spatial mask, filter the background area, and output a corrected crowd density map.
9. The single-domain generalized crowd counting method based on mask-guided block-level memory alignment according to claim 8, characterized in that, The steps for obtaining the space mask are as follows: S2321. Input the pixel-level features output by the feature extraction module into a preset block-level classification branch, and output a block-level classification map from the block-level classification branch; S2322. Binarize the block-level classification image to obtain the spatial mask of the filtered background region.
10. The single-domain generalized crowd counting method based on mask-guided block-level memory alignment according to any one of claims 1-9, characterized in that, The total loss function for multi-loss joint optimization training of the single-domain generalized crowd counting model is calculated as follows: in, Represents the total loss function. This represents the density regression loss of the original single-domain crowd image. This represents the density regression loss for enhancing single-domain crowd image data. This represents the global density memory loss of the original single-domain crowd image. This represents the global density memory loss for enhancing single-domain crowd image data. This represents the pure density memory loss of the original single-domain crowd image. This represents the pure density memory loss used to enhance single-domain crowd image data. The weight coefficients represent the density-aware classification loss. This represents the auxiliary density-aware classification loss of the original single-domain crowd image. This represents the auxiliary density-aware classification loss for enhancing single-domain crowd image data. The weighting coefficients represent the block-level classification loss. This represents the block-level classification loss of the original single-domain crowd image. This represents the block-level classification loss for enhancing single-domain crowd image data. The weight coefficients representing the similarity alignment loss are... This represents the similarity alignment loss of the original single-domain crowd image. This represents the similarity alignment loss for enhancing single-domain crowd image data. The weighting coefficients represent the mask prediction loss. This represents the mask prediction loss of the original single-domain crowd image. This represents the mask prediction loss for enhancing single-domain crowd image data.