Density level segmentation-based multivariate paradigm decoupling crowd counting method and system

Abstract

The invention discloses a multivariate paradigm decoupling crowd counting method and system based on density level segmentation, and belongs to the technical field of crowd counting. The method comprises the steps that a target image is acquired, the target image is input into a segmentation mask generator generated through training of a Gaussian distribution mask generator, a density level mask is obtained, and the segmentation mask generator comprises an image encoder and a density level segmentation module; inputting the target image into a multivariate paradigm counting module to obtain a frame position information graph, a point position information graph and a density position information graph; inputting the density level mask, the frame position information graph, the point position information graph and the density position information graph into a result fusion module to obtain a composite result graph; and performing summation based on the composite result image to obtain the total number of crowds in the target image. The method improves the accuracy of crowd counting.

Description

Technical Field

[0001] This application belongs to the field of crowd counting technology, and in particular relates to a multi-paradigm decoupled crowd counting method and system based on density level segmentation. Background Technology

[0002] Crowd counting, a core research topic in computer vision, has increasing application value in scenarios such as public safety monitoring, large-scale event management, and emergency response. With the acceleration of urbanization and the normalization of crowd gatherings in public places, the use of intelligent technology to accurately estimate and analyze crowd density has become a key means to prevent congestion and stampedes, optimize resource allocation, and improve urban management efficiency.

[0003] Existing research often employs direct regression methods to determine the number of people. This method establishes a direct mapping between global features of an image and the total number of people. However, it has significant limitations in practical applications due to its inability to provide spatial distribution information and its susceptibility to interference from complex backgrounds. To improve counting accuracy and obtain richer scene information, three mainstream technical paradigms have emerged: bounding box detection, point localization, and density map generation. With the increasing complexity of application scenarios, these single-paradigm solutions still suffer from the following shortcomings in practical deployment: First, real-world surveillance images often exhibit significant viewpoint distortion effects, resulting in multiple areas with vastly different crowd densities within the same frame. A single counting paradigm cannot effectively address this uneven density simultaneously. Second, there is a trade-off between the ability of different counting paradigms to handle different density scenes and the richness of location information they can provide. Third, while existing density-aware methods have begun to attempt to enable models to perceive the crowding level of regions through attention mechanisms or density mask generation, these methods typically only adapt different model parameters or branches for the same counting paradigm, without fundamentally addressing the inherent limitations of a single paradigm.

[0004] Existing crowd counting methods mostly employ a single counting paradigm (such as density map regression, point localization, or bounding box detection), making it difficult to simultaneously adapt to variations in density regions within an image. These methods suffer from low counting accuracy, insufficient adaptability, and high computational cost. Summary of the Invention

[0005] This application aims to address at least one of the technical problems existing in the prior art. To this end, this application proposes a multi-paradigm decoupled crowd counting method and system based on density level segmentation, which improves the accuracy of crowd counting.

[0006] Firstly, this application provides a multi-paradigm decoupled population counting method based on density level segmentation, the method comprising: The target image is acquired and input into a segmentation mask generator trained by a Gaussian distribution mask generator to obtain a density level mask. The segmentation mask generator includes an image encoder and a density level segmentation module. The target image is input into the multivariate instance counting module to obtain the bounding box location information map, the point location information map, and the density location information map; The density level mask, bounding box location information map, point location information map, and density location information map are input into the result fusion module to obtain the composite result map; The total number of people in the target image is obtained by summing the composite result images.

[0007] According to one embodiment of this application, the step of inputting the target image into a segmentation mask generator trained by a Gaussian distribution mask generator to obtain a density level mask includes: The target image is input into an image encoder for image encoding and position encoding to obtain encoded features. The image encoder includes a block embedding Transformer encoding module and a position encoding module. The encoded features are input into the density level segmentation module to perform multi-stage feature enhancement, resulting in a density level mask.

[0008] According to one embodiment of this application, the step of inputting the target image into an image encoder for image encoding and position encoding to obtain encoded features includes: The target image is input into the block embedding Transformer encoding module, and the image is divided into multiple image blocks through block embedding. Flatten multiple image patches to obtain multiple image patch vectors; Multiple image patch vectors are positionally encoded to obtain an image patch sequence; The image patch sequence is input into multiple Transformer layers, and global and local features of the image patch sequence are extracted through MHSA and feedforward neural networks to obtain encoded features.

[0009] According to one embodiment of this application, the step of inputting the encoded features into the density level segmentation module to perform multi-stage feature enhancement on the encoded features to obtain a density level mask includes: The encoded features are input into the transposed convolution module for upsampling to restore the spatial size of the encoded features and obtain the preliminary upsampled features; The initial upsampled features are input into the activation layer for further extraction and refinement of local contextual features to obtain local enhanced features; The local enhancement features are input into the multi-head self-attention module to model the global dependency relationship of the two-dimensional spatial dimension of the local enhancement features, capture contextual information, and obtain global perception features; The globally perceived features are input into the two-dimensional convolutional feature mapping module to perform channel dimension mapping and density level prediction, thereby obtaining the density level mask.

[0010] According to one embodiment of this application, the step of inputting the target image into the multivariate instance counting module to obtain a bounding box location information map, a point location information map, and a density location information map includes: The target image is input into the bounding box detection module, and features are extracted by the first feature extractor to obtain multiple feature maps of different scales. The multiple feature maps of different scales are input into the top-down feature modulation module respectively, and the multiple feature maps of different scales are integrated through the scale feedback mechanism to obtain the bounding box position information map. The target image is input into the point localization module, and the point localization feature map is obtained by the second feature extractor. The point localization feature map is upsampled and then input into the convolution module for depth convolution to obtain the point position information map. The target image is input into the density map generation module, where features are extracted by the third feature extractor and upsampling and focus localization are performed by the focus density map regression head to obtain a density location information map. The multivariate example counting module includes a bounding box detection module, a point localization module, and a density map generation module. The density map generation module includes a third feature extractor and a focal density map regression head.

[0011] According to one embodiment of this application, the step of inputting the target image into the density map generation module, performing feature extraction through a third feature extractor, and upsampling and focus localization through a focus density map regression head to obtain a density location information map includes: The target image is input into the density map generation module, and features are extracted by the third feature extractor to obtain the first multi-scale feature map, the second multi-scale feature map, the third multi-scale feature map and the fourth multi-scale feature map; The first multi-scale feature map, the second multi-scale feature map, the third multi-scale feature map, and the fourth multi-scale feature map are input into the transition layer for feature fusion to obtain a multi-scale fused feature map. The multi-scale fused feature map is input into the convolutional layer and the normalization layer for depth convolution and normalization to obtain the first convolutional feature map; The first convolutional feature map is input into the activation layer for activation, resulting in the second convolutional feature map. The second convolutional feature map is input into a two-dimensional transposed convolutional layer for transposition, and candidate local maxima are extracted to obtain a density location information map.

[0012] According to one embodiment of this application, the step of inputting the density level mask, the bounding box location information map, the point location information map, and the density location information map into the result fusion module to obtain a composite result map includes: The density level mask is represented as a box binary mask, a dot binary mask, and a density binary mask according to the size of the density region; Element-wise multiplication of the binary mask and the box position information map yields the filter box position information map; Element-wise multiplication of the point binary mask and the point location information map yields the filtered point location information map; Element-wise multiplication of the density binary mask and the density location information map yields the filtered density location information map. The filter box location information map, filter point location information map, and filter density location information map are summed pixel by pixel to obtain the composite result map.

[0013] Secondly, this application provides a multi-paradigm decoupled population counting system based on density level segmentation, the system comprising: The acquisition module is used to acquire the target image and input the target image into a segmentation mask generator trained by a Gaussian distribution mask generator to obtain a density level mask. The segmentation mask generator includes an image encoder and a density level segmentation module. The processing module is used to input the target image into the multivariate instance counting module to obtain the bounding box location information map, the point location information map, and the density location information map; The fusion module is used to input the density level mask, the bounding box location information map, the point location information map, and the density location information map into the result fusion module to obtain the composite result map; The counting module is used to sum the results based on the composite image to obtain the total number of people in the target image.

[0014] Thirdly, this application provides an electronic device including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the multi-paradigm decoupled crowd counting method based on density level segmentation as described in the first aspect above.

[0015] Fourthly, this application provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the multi-paradigm decoupled population counting method based on density level segmentation as described in the first aspect above.

[0016] Fifthly, this application provides a chip including a processor and a communication interface, the communication interface being coupled to the processor, the processor being used to run programs or instructions to implement the multi-paradigm decoupled crowd counting method based on density level segmentation as described in the first aspect.

[0017] In a sixth aspect, this application provides a computer program product, including a computer program that, when executed by a processor, implements the multi-paradigm decoupled crowd counting method based on density level segmentation as described in the first aspect above.

[0018] Additional aspects and advantages of this application will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of this application.

[0019] The present invention provides a multi-paradigm decoupled population counting method based on density level segmentation, which has the following advantages over the prior art: (1) This invention achieves pixel-level density perception of the target image by introducing a segmentation mask generator, and automatically divides it into three semantic regions: low density, medium density, and high density. Together, these constitute a structured analysis of the spatial distribution of crowds in the image, providing more accurate spatial guidance for subsequent counting paradigm matching. By introducing a density-level segmentation mechanism, the image is divided into low-density, medium-density, and high-density regions, and the most suitable counting paradigm is matched for each region. This effectively reduces the limitations of a single method in complex scenes and significantly improves the overall counting accuracy and robustness.

[0020] (2) This invention introduces a non-overlapping focal density map structure in the density map generation module, directly representing the individual position through local maxima, reducing the overlap and ambiguity problem of traditional Gaussian density maps in dense areas, and improving the counting accuracy and interpretability in high-density scenes. By fusing the bounding box detection and point localization results, it can achieve counting in high-density areas while preserving the individual bounding box or center point position in low-density areas, taking into account both the requirements of counting accuracy and rich positional information, and improving counting accuracy and robustness.

[0021] (3) This invention standardizes the prediction format by transforming the location information map, improves the adaptability of density regions by using segmentation masks, performs regional weighted fusion of prediction results by density level masks, and generates counting results by result aggregation and post-processing. This simplifies the post-processing process, improves real-time performance and deployment efficiency, and reduces cross-density region interference by weighted fusion of output results from different counting paradigms at the region level, thereby improving overall counting consistency and spatial rationality. Attached Figure Description

[0022] The above and / or additional aspects and advantages of this application will become apparent and readily understood from the description of the embodiments taken in conjunction with the following drawings, in which: Figure 1 This is one of the flowcharts of the multi-paradigm decoupled population counting method based on density level segmentation provided in the embodiments of this application; Figure 2 This is the second flowchart of the multi-paradigm decoupled population counting method based on density level segmentation provided in the embodiments of this application; Figure 3 This is a schematic diagram of the density level segmentation module provided in an embodiment of this application; Figure 4 This is a model structure diagram of the box detection module provided in the embodiments of this application; Figure 5 This is a model structure diagram of the point positioning module provided in the embodiments of this application; Figure 6 This is a model structure diagram of the density map generation module provided in the embodiments of this application; Figure 7 This is a schematic diagram of the structure of the multi-paradigm decoupled crowd counting system based on density level segmentation provided in the embodiments of this application; Figure 8 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application. Detailed Implementation

[0023] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0024] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or more. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0025] The following description, in conjunction with the accompanying drawings, details the multi-paradigm decoupled crowd counting method based on density level segmentation, the multi-paradigm decoupled crowd counting system based on density level segmentation, the electronic device, and the readable storage medium provided in this application, through specific embodiments and application scenarios.

[0026] Among them, the multi-paradigm decoupled crowd counting method based on density level segmentation can be applied to the terminal, specifically executed by the hardware or software in the terminal.

[0027] The terminal includes, but is not limited to, portable communication devices such as mobile phones or tablets with touch-sensitive surfaces (e.g., touchscreen displays and / or touchpads). It should also be understood that, in some embodiments, the terminal may not be a portable communication device, but rather a desktop computer with touch-sensitive surfaces (e.g., touchscreen displays and / or touchpads).

[0028] The following embodiments describe a terminal including a display and a touch-sensitive surface. However, it should be understood that the terminal may include one or more other physical user interface devices such as a physical keyboard, mouse, and joystick.

[0029] The density-level segmentation-based multi-paradigm decoupled crowd counting method provided in this application can be executed by an electronic device or a functional module or entity within an electronic device that can implement the density-level segmentation-based multi-paradigm decoupled crowd counting method. The electronic devices mentioned in this application include, but are not limited to, mobile phones, tablets, computers, cameras, and wearable devices. The following description uses an electronic device as the execution subject to illustrate the density-level segmentation-based multi-paradigm decoupled crowd counting method provided in this application.

[0030] Boundary detection methods originate from general object detection techniques, counting objects by identifying and locating the bounding boxes of each head or human body in an image. This method provides instance-level precise location and size information, offering data support for downstream tasks such as individual behavior analysis. Point localization methods achieve counting and localization by regressing the precise coordinates of the center point of each head. Compared to bounding box detection, its annotation cost is lower, exhibiting good balance in medium-density scenes. Density map generation methods are currently the mainstream technique for handling high-density and severely occluded scenes. It smooths discrete head point annotations using a Gaussian kernel function to generate a continuous density distribution map, and estimates the total number of people by integrating this map.

[0031] Figure 1 This is one of the flowcharts illustrating the multivariate paradigm decoupled crowd counting method based on density level segmentation provided in this application embodiment, such as... Figure 1As shown, the multi-paradigm decoupled population counting method based on density level segmentation includes steps 110, 120, 130 and 140.

[0032] Step 110: Obtain the target image and input it into the segmentation mask generator trained by the Gaussian distribution mask generator to obtain the density level mask. The segmentation mask generator includes an image encoder and a density level segmentation module. It is easy to understand that the Gaussian distribution mask generator is used to generate the learning and training supervision signals required when training the segmentation mask generator, that is, the ground truth value of the segmentation mask generated by the real human head coordinates.

[0033] Specifically, the Gaussian distribution mask generator first calculates the crowding score for each pixel in the image, which is used to train the density levels. For the input image... ,in and The width and height of the image are given by [insert dimensions here], and each pixel in the image is represented by [insert dimensions here]. ,in The image contains The individual's head, its position coordinates are represented as Each pixel is calculated using a Gaussian function. Go to each head position The distance is considered, and the crowding score of the pixel is obtained by combining the effects of all distances. Pixel To the head position The Euclidean distance is defined as The calculation formula is as follows: in, Let it represent the Euclidean distance between the i-th pixel and the j-th head position. Let x be the x-coordinate of the i-th pixel. Let x be the x-coordinate of the j-th person's head position. Let y be the y-coordinate of the i-th pixel. Let be the y-coordinate of the j-th person's head position.

[0034] The Gaussian function is used to measure the crowding level of each head on the pixel. The calculation formula is shown below: Where σ is the standard deviation of the Gaussian function, used to control the range of influence of the degree of crowding. Let n be the crowding score, and n be the total number of heads in the image.

[0035] The crowding scores are obtained by sequentially calculating the crowding scores of m pixels in the image. The heatmap of the overall density level is denoted as T. To further assign appropriate counting paradigms to the heatmap to obtain masking results, in the low-crowding-score intervals... and high range The mask normalization curves for three counting paradigms are set, and the calculation formulas are shown below: in, This represents three counting paradigms: bounding box detection, point localization, and density map generation. To show the rate at which the mask confidence decreases as the congestion level changes. The mask normalization curves for the three counting paradigms are shown. The lower interval of the crowding score. The high interval of the crowding score.

[0036] Through two thresholds and The settings ensure a smooth transition between different counting paradigms. For pixels Crowding score It can be based on the low interval of each counting paradigm and high range Determine the corresponding counting paradigm, and then use the appropriate mask normalization curve to obtain the true mask value of that pixel. .

[0037] The truth value of the mask This further participates in the training of the segmentation mask generator. Specifically, the mean squared error (MSE) is used as the loss function, and the formula for calculating the loss function is shown below: in, The number of training images, For the first The ground truth value of the segmentation mask for the image. The segmentation mask predicted by the model. This is the loss function for the segmentation mask generator.

[0038] The high-quality segmentation mask generated by the segmentation mask generator can provide accurate region division basis for subsequent multivariate paradigm counting models.

[0039] In some embodiments, inputting the target image into a segmentation mask generator trained by a Gaussian distribution mask generator to obtain a density level mask includes: The target image is input into an image encoder for image encoding and position encoding to obtain encoded features. The image encoder includes a block embedding Transformer encoding module and a position encoding module. The encoded features are input into the density level segmentation module to perform multi-stage feature enhancement, resulting in a density level mask.

[0040] Figure 2 This is the second flowchart illustrating the multivariate paradigm decoupled population counting method based on density level segmentation provided in this application embodiment, as follows: Figure 2 As shown, the target image to be counted is obtained. The target image is an RGB image. The target image is input into the segmentation mask generator. The image encoder ViT-H in the segmentation mask generator uses the Transformer architecture to process the image. It captures the long-range dependencies between image regions through a multi-head self-attention mechanism, which makes it more suitable for scenes with large density changes. The image encoder includes a block embedding Transformer encoding module and a position encoding module.

[0041] In some embodiments, the step of inputting the target image into an image encoder for image encoding and position encoding to obtain encoded features includes: The target image is input into the block embedding Transformer encoding module, and the image is divided into multiple image blocks through block embedding. Flatten multiple image patches to obtain multiple image patch vectors; Multiple image patch vectors are positionally encoded to obtain an image patch sequence; The image patch sequence is input into multiple Transformer layers, and global and local features of the image patch sequence are extracted through MHSA and feedforward neural networks to obtain encoded features.

[0042] It is easy to understand that block embedding is responsible for embedding the target image Divided into indivual An image patch, where H, W, and C represent the height, width, and number of channels, respectively. Each image patch is flattened to... It is mapped to a feature space of dimension D through a linear transformation, and then positional encoding is added. Supplementing spatial information, the final sequence is generated. As input to the Transformer, the calculation formula is as follows: in, Both indicate that they will The linear transformation weight matrix mapped to the feature space of dimension D. The flattened image patch vector, Indicates position code, For image blocks After linear transformation, it is mapped to Vectors in a dimensional feature space To be Mapping to Dimensions Linear transformation weight matrix of space, for The corresponding bias term, This represents the first element in the input sequence of the Transformer. Each element.

[0043] Block embedding Transformer encoding consists of L stacked Transformer layers, each containing an MHSA and a feedforward neural network. For the Lth... Layer input The formula for calculating MHSA is as follows: in, , For querying the matrix, For the first The output of the layer, To query the weight matrix, The key matrix, The key weight matrix is... For value matrices, For value weight matrix, For each dimension of attention head, Key matrix transpose, For the output of the multi-head self-attention mechanism, This is the num_heads-th attention head.

[0044] Furthermore, residual connections and normalization layers are used to alleviate problems such as vanishing or exploding gradients. The calculation formula is shown below: in, The features are those obtained after residual connections and layer normalization. This is a layer normalization operation.

[0045] The output calculation formula for the block embedding Transformer coding module is shown below: in, and These are the weight matrices for the two fully connected network layers. and For the corresponding bias term, The output of the Transformer layer. This is the output of the feedforward neural network. This is the GELU activation function.

[0046] Repeat L times to obtain Since the Transformer itself lacks spatial location awareness, ViT (VisionTransformer) employs positional encoding to preserve the spatial structure of the image. In the multi-paradigm decoupled crowd counting task, ViT is primarily used to extract density-related global and local features. To fully utilize its pre-training capability on large-scale datasets, its pre-trained weights are used as initialization parameters, and the model parameters are frozen to reduce training computation and improve stability. The input image I has a size of (W, H). After passing through the ViT encoder, the output image features have a dimension of [missing information]. In this model, C, W', and H' represent the number of feature channels, width, and height, respectively. ViT efficiently extracts global and local features through residual connections and MHSA, providing rich semantic information for density level classification. The image is divided into a series of image patches, and then the image patch sequence is processed through a multi-layer Transformer to extract feature representations containing global contextual information, thus obtaining encoded features.

[0047] In this embodiment, a Gaussian distribution mask generator is constructed and combined with point annotation information to generate a density heatmap, thereby modeling the density distribution of crowds in an image. This enables more accurate identification of the spatial distribution characteristics of different density regions in an image, possesses stronger region perception capabilities, adapts to complex scenes under different viewpoints, scales, and occlusion conditions, and exhibits good cross-scene generalization capabilities, making it suitable for various practical application environments. By introducing a segmentation mask generator, pixel-level density perception of the target image is achieved, automatically dividing it into three semantic regions: low density, medium density, and high density. Together, these constitute a structured analysis of the spatial distribution of crowds in the image, providing more accurate spatial guidance for subsequent counting paradigm matching.

[0048] In some embodiments, the step of inputting the encoded features into the density level segmentation module to perform multi-stage feature enhancement on the encoded features to obtain a density level mask includes: The encoded features are input into the transposed convolution module for upsampling to restore the spatial size of the encoded features and obtain the preliminary upsampled features; The initial upsampled features are input into the activation layer for further extraction and refinement of local contextual features to obtain local enhanced features; The local enhancement features are input into the multi-head self-attention module to model the global dependency relationship of the two-dimensional spatial dimension of the local enhancement features, capture contextual information, and obtain global perception features; The globally perceived features are input into the two-dimensional convolutional feature mapping module to perform channel dimension mapping and density level prediction, thereby obtaining the density level mask.

[0049] Figure 3 This is a schematic diagram of the density level segmentation module provided in an embodiment of this application, as shown below. Figure 3 As shown, the density-level segmentation module employs a progressive upsampling strategy. By fusing transposed convolution, convolution operations, and MHSA, it enhances features through multi-stage enhancement to more accurately distinguish between high-density and low-density regions and adapts to density variations in complex scenes, achieving refined feature processing. Upsampling via transposed convolution progressively restores the spatial details of low-resolution features, enhancing the segmentation capability for regions of different densities. The calculation formula is shown below: in, These are the parameters of the transposed convolutional layer. The output features are obtained by upsampling through transposed convolution. This is a transpose convolution operation used to upsample the feature map. These are the output features of the image encoder.

[0050] Furthermore, local features are extracted by combining convolution operations with the ReLU activation function. The formula for calculating local features is shown below: in, These are the parameters of the convolutional layer. These are local features.

[0051] To fully utilize global context information, an MHSA mechanism is introduced at each upsampling stage to enhance the model's ability to perceive global features. After multiple transposed convolutions, convolutions, and MHSA feature enhancements, the mapping convolutional layer transforms high-level features into density-level masks, calculated as follows: in, The parameters of the mapping convolutional layer, This layer maps features to a density-level mask, enabling density-level classification for each pixel.

[0052] In this embodiment, by introducing a density level segmentation mechanism, the image is divided into low-density, medium-density, and high-density regions, and the most suitable counting paradigm is matched for each region. This effectively reduces the limitations of a single method in complex scenarios and significantly improves the overall counting accuracy and robustness.

[0053] Step 120: Input the target image into the multivariate example counting module to obtain the bounding box location information map, the point location information map, and the density location information map; In some embodiments, inputting the target image into the multivariate instance counting module to obtain a bounding box location information map, a point location information map, and a density location information map includes: The target image is input into the bounding box detection module, and features are extracted by the first feature extractor to obtain multiple feature maps of different scales. The multiple feature maps of different scales are input into the top-down feature modulation module respectively, and the multiple feature maps of different scales are integrated through the scale feedback mechanism to obtain the bounding box position information map. The target image is input into the point localization module, and the point localization feature map is obtained by the second feature extractor. The point localization feature map is upsampled and then input into the convolution module for depth convolution to obtain the point position information map. The target image is input into the density map generation module, where features are extracted by the third feature extractor and upsampling and focus localization are performed by the focus density map regression head to obtain a density location information map. The multivariate example counting module includes a bounding box detection module, a point localization module, and a density map generation module. The density map generation module includes a third feature extractor and a focal density map regression head.

[0054] It is easy to understand that the multi-paradigm counting module's process includes a parallel counting part and a directional mask fusion part.

[0055] The parallel counting section feeds the target image into three independent processing modules of the multivariate example counting module in parallel, including the bounding box detection module, the point localization module, and the density map generation module. Each module performs a preliminary analysis of the entire image once.

[0056] Figure 4 This is a model structure diagram of the bounding box detection module provided in the embodiments of this application, as shown below. Figure 4 As shown, the bounding box detection module is specifically responsible for processing the low-density region divided by the segmentation mask generator. This region has sparse crowds and clear individual outlines, so using bounding box detection can more accurately locate the bounding box of each head.

[0057] The feature extractor of the bounding box detection module uses VGG16 as the backbone network and is initialized with ImageNet pre-trained weights, generating feature maps at different scales such as 1 / 2, 1 / 4, 1 / 8, and 1 / 16. The top-down feature modulation module is a key part of the bounding box detection module. It integrates feature maps of different resolutions through a dynamic scale feedback mechanism. Specifically, feature information from low-resolution branches (such as 1 / 8 and 1 / 16 scales) is fed back and integrated into high-resolution branches (such as 1 / 4 scale), thereby achieving gradual enhancement of semantic information and details. In subsequent processing, after the bounding box detection module outputs a large number of candidate boxes, non-maximum suppression (NMS) is used to filter the final results. NMS sorts the boxes according to their confidence level, retaining the boxes with the highest confidence and removing other boxes whose overlap with the highest-confidence bounding box exceeds a set threshold.

[0058] During training, the bounding box detection module only uses low-density region annotations for learning, while high-density region annotations are ignored to reduce noise interference. The loss function consists of two parts: a classification loss (cross-entropy loss) for optimizing class prediction and a regression loss (SmoothL1 loss) for optimizing bounding box coordinates. The calculation formula is shown below: in, For real category labels, To predict probabilities, To balance the weights for each category, and These are the coordinates of the predicted and ground truth bounding boxes, respectively. For classification loss function, For regression loss function, This is the loss function for bounding box detection.

[0059] Regression loss is adopted The loss function optimizes the bounding box coordinates. Losses can effectively reduce the impact of outliers. Used to balance the effects of regression loss.

[0060] Figure 5 This is a model structure diagram of the point positioning module provided in the embodiments of this application, such as... Figure 5As shown, the point localization module is specifically designed for medium-density regions, where the population is relatively dense, yet the center point of a person's head can still be located. The feature extractor for this module uses VGG-16bn as its backbone network. During the feature fusion stage, the point localization module upsamples the feature map output from the backbone network and then adds it element-wise with feature maps from earlier layers in the backbone network. This approach allows the point localization module to combine high-level semantic information and richer spatial details, ultimately outputting a 1 / 8 scale depth feature map. The point localization module employs two parallel regression head modules. The point coordinate regression head predicts the offset of each reference point based on a set of fixed reference points. The final predicted point coordinates are obtained through offsetting, while the confidence score regression head predicts in parallel the confidence score of each proposed point as a real human head. .

[0061] During training, the point localization module uses only ground data from medium-density regions to avoid introducing noise in low-density and high-density areas. The optimization objective of the point localization module is determined by the classification loss. The localization loss consists of two parts. The classification loss uses the cross-entropy loss function to optimize the target category prediction, and the calculation formula is shown below: in, For negative sample reweighting factor, For point localization and classification loss function, The confidence score for the predicted point represents the probability that the model predicts a point to be a real human head. The total number of predicted point proposals. This represents the number of real data points.

[0062] The localization loss is calculated using supervised regression with Euclidean distance, as shown in the following formula: in, For point localization, the localization loss function is... Let i be the coordinates of the i-th real head center. These are the coordinates of the predicted point.

[0063] The total loss function for point localization is Classification loss and positioning loss The composition and calculation formula are as follows: in, Used to balance the impact of regression loss, This is the total loss function for point localization.

[0064] In some embodiments, the step of inputting the target image into the density map generation module, performing feature extraction through a third feature extractor, and upsampling and focus localization through a focus density map regression head to obtain a density location information map includes: The target image is input into the density map generation module, and features are extracted by the third feature extractor to obtain the first multi-scale feature map, the second multi-scale feature map, the third multi-scale feature map and the fourth multi-scale feature map; The first multi-scale feature map, the second multi-scale feature map, the third multi-scale feature map, and the fourth multi-scale feature map are input into the transition layer for feature fusion to obtain a multi-scale fused feature map. The multi-scale fused feature map is input into the convolutional layer and the normalization layer for depth convolution and normalization to obtain the first convolutional feature map; The first convolutional feature map is input into the activation layer for activation, resulting in the second convolutional feature map. The second convolutional feature map is input into a two-dimensional transposed convolutional layer for transposition, and candidate local maxima are extracted to obtain a density location information map.

[0065] Figure 6 This is a model structure diagram of the density map generation module provided in the embodiments of this application, such as... Figure 6 As shown, the density map generation module is specifically designed for high-density areas where the population is extremely dense and individual boundaries and center points are difficult to distinguish. Therefore, density map generation is used to estimate the total number of individuals. The density map generation module consists of three core parts: a feature extraction module, a focal density map generation head, and a post-processing module.

[0066] The feature extraction module of the density map generation module uses HRNet as its backbone network. It maintains high-resolution feature representation through parallel multi-resolution branches, supporting accurate generation of the focal density map. The feature extraction module includes convolutional layers, normalization layers, two activation layers, and two 2D transposed convolutional layers, reducing the input image resolution to 1 / 4. Subsequently, multi-resolution branches are gradually introduced, ultimately outputting four feature maps at different scales. Each stage fuses multi-resolution features through transition layers to generate a comprehensive multi-scale feature map. The focal density map generation head is responsible for integrating the multi-scale features output by the feature extraction module and upsampling them to generate a single-channel focal density map, where the local maximum corresponds to the center of the crowd's head. Its structure includes convolutional layers, normalization and activation layers, and two transposed convolutional layers, ultimately outputting a single-channel focal density map. The specific pixel value calculation formula is shown below: in, For pixels Euclidean distance to the nearest center of the head To adjust the parameters of the decay rate, This is a constant term used to control the baseline value. These are pixel values.

[0067] The post-processing module extracts candidate local maxima from the focal density map using 3×3 max pooling and applies an adaptive threshold. Filter positive examples. If the maximum value is below a fixed threshold, it is determined to be an unmanned scene and the count is set to 0, thereby converting the focus density map into an interpretable counting result.

[0068] During training, the density map generation module uses only data from high-density regions to avoid introducing noise in low- and medium-density regions. Input images are trained by cropping only those conforming to high-density regions. The optimization objective of the density map generation module is determined by the overall mean squared error loss. It consists of local structural similarity loss. The calculation formula is shown below: in, These are the predicted pixel values ​​generated by the focus density map generator. It is the true value of the focal density map. , The height and width of the density map, This is the overall mean squared error loss function.

[0069] in, The population count in high-density areas and The first Predicted and actual values ​​of the focal density map in the 30-pixel region surrounding an individual's head. It is a structural similarity index. This is the local structural similarity loss function.

[0070] The calculation formula is as follows: in, and These are the means of the predicted density map and the actual density map, respectively. and Its standard deviation, Let the covariance of the two be , and It is a stabilizing factor.

[0071] Through overall mean square error loss and local structural similarity loss It can monitor pixel-level differences, enabling the predicted focal density map to closely approximate the true value, while enhancing the structural information of local maxima and suppressing interference from background regions.

[0072] In this embodiment, a decoupled training strategy is employed, training the bounding box detection module, point localization module, and density map generation module separately for different density regions. This reduces gradient conflicts and inconsistencies in optimization objectives during multi-task joint training. Simultaneously, each module can focus on learning its optimal feature representation within a specific density region, improving training efficiency and convergence stability while reducing overall training costs.

[0073] In this embodiment, by introducing a non-overlapping focal density map structure into the density map generation module, the individual position is directly represented by local maxima, reducing the overlap and ambiguity problem of traditional Gaussian density maps in dense regions and improving the counting accuracy and interpretability in high-density scenes. By fusing the bounding box detection and point localization results, it is possible to count in high-density areas while preserving the individual bounding boxes or center point positions in low-density areas, thus balancing the dual requirements of counting accuracy and rich positional information, and improving counting accuracy and robustness.

[0074] Step 130: Input the density level mask, bounding box location information map, point location information map, and density location information map into the result fusion module to obtain the composite result map; In some embodiments, inputting the density level mask, bounding box location information map, point location information map, and density location map into the result fusion module to obtain a composite result map includes: The density level mask is represented as a box binary mask, a dot binary mask, and a density binary mask according to the size of the density region; Element-wise multiplication of the binary mask and the box position information map yields the filter box position information map; Element-wise multiplication of the point binary mask and the point location information map yields the filtered point location information map; Element-wise multiplication of the density binary mask and the density location information map yields the filtered density location information map. The filter box location information map, filter point location information map, and filter density location information map are summed pixel by pixel to obtain the composite result map.

[0075] It's easy to understand that the three technical paradigms—boundary detection, point localization, and density map generation—with their different advantages and applicable boundaries, are no longer in a competitive or substitutive relationship. Instead, they are decoupled into three independent expert processing units. Using a segmentation mask as a spatial switch, the output of each counting paradigm is controlled to only take effect within its authorized region. This allows these three expert units to work collaboratively within a unified framework, performing tasks in the areas where they can best leverage their strengths based on the density characteristics of local image regions, thereby optimizing overall performance.

[0076] The result fusion module fuses the prediction results from the bounding box detection module, the point localization module, and the density map generation module to generate the final crowd count and location information for the entire image. After the prediction results from the bounding box detection module, the point localization module, and the density map generation module are converted into a location information map, a segmentation mask is used to obtain the prediction of the corresponding density regions. Then, aggregation and post-processing are performed to obtain the total number of people and individual locations in the entire image.

[0077] In the initial stage of the result fusion module, the bounding box detection module, the point localization module, and the density map generation module generate prediction results and convert them into a location information map in a unified format. For the bounding box detection module, the prediction result is a set of bounding boxes of low-density regions, from which center points are extracted to form a point set. This indicates the detected individual location. The point localization module predicts the head center point in the medium-density region, forming a point set. The density map generation model generates focal density maps of high-density regions, through... Max pooling extracts candidate local maxima to obtain a point set. These point sets are then mapped to the input image. Binary location information maps with consistent resolution are denoted as follows: , and Specifically, the location information map is defined as follows: if the pixel point If it belongs to the corresponding point set, then , or Otherwise, it is 0.

[0078] Furthermore, the density-level mask generated by the segmentation mask generator The predictions of each model are restricted to its suitable density region, which can be represented as three binary masks, denoted as box binary masks. dot binary mask And density binary mask The formula for calculating the filtered location information map using element-wise multiplication is shown below: in, This is a diagram showing the location information of the filter boxes. This is a map showing the location information of the filter points. This is a filter density location information map. This is a diagram showing the location information of the boxes. This is a point location information map. This is a density location information map. For the binary mask of the box, For point binary masks, It is a density binary mask.

[0079] The three filtered location information images are summed pixel-wise and then aggregated into a final composite image. The calculation formula is as follows: in, This is a composite result image.

[0080] Step 140: Summate the composite result image to obtain the total number of people in the target image.

[0081] Predicted total number of people The calculation formula is obtained by summing the composite result graphs, as shown below: in, To predict the total number of people.

[0082] In this embodiment, the prediction format is standardized by transforming the location information map, the adaptability of density regions is improved by using segmentation masks, and the prediction results are weighted and fused by density-level masks. The counting results are then generated through result aggregation and post-processing. This simplifies the post-processing workflow, improves real-time performance and deployment efficiency, and reduces cross-density region interference by weighting and fusing the output results of different counting paradigms at the region level, thereby improving overall counting consistency and spatial rationality.

[0083] The multi-paradigm decoupled crowd counting method based on density level segmentation provided in this application generates a density level mask through the image encoder and density level segmentation module in the segmentation mask generator, providing a structured prior for distinguishing different crowd density regions. The multi-paradigm counting module generates bounding box location information maps, point location information maps, and density location information maps in parallel, constructing a complementary supervision mechanism between localization and density estimation. The result fusion module fuses the density level mask, bounding box location information maps, point location information maps, and density location information maps into a composite result map, which is then summed to obtain the total number of people, significantly improving the accuracy, generalization ability, and application value of the counting.

[0084] The density-level segmentation-based multi-paradigm decoupled crowd counting method provided in this application can be implemented by a density-level segmentation-based multi-paradigm decoupled crowd counting system. This application uses the density-level segmentation-based multi-paradigm decoupled crowd counting system executing the density-level segmentation-based multi-paradigm decoupled crowd counting method as an example to illustrate the density-level segmentation-based multi-paradigm decoupled crowd counting system provided in this application.

[0085] This application also provides a multi-paradigm decoupled crowd counting system based on density level segmentation, such as... Figure 7 As shown, the multi-paradigm decoupled population counting system based on density level segmentation includes: an acquisition module 710, a processing module 720, a fusion module 730, and a counting module 740.

[0086] The acquisition module 710 is used to acquire the target image and input the target image into a segmentation mask generator trained by a Gaussian distribution mask generator to obtain a density level mask. The segmentation mask generator includes an image encoder and a density level segmentation module. The processing module 720 is used to input the target image into the multivariate instance counting module to obtain the bounding box location information map, the point location information map, and the density location information map; The fusion module 730 is used to input the density level mask, the box position information map, the point position information map and the density position information map into the result fusion module to obtain the composite result map; The counting module 740 is used to sum based on the composite result image to obtain the total number of people in the target image.

[0087] The multi-paradigm decoupled crowd counting system based on density level segmentation provided in this application generates a density level mask through the image encoder and density level segmentation module in the segmentation mask generator, providing a structured prior for distinguishing different crowd density regions. The multi-paradigm counting module generates bounding box location information maps, point location information maps, and density location information maps in parallel, constructing a complementary supervision mechanism between localization and density estimation. The result fusion module fuses the density level mask, bounding box location information maps, point location information maps, and density location information maps into a composite result map, which is then summed to obtain the total number of people, significantly improving the accuracy, generalization ability, and application value of the counting.

[0088] The multi-paradigm decoupled crowd counting system based on density level segmentation provided in this application can achieve... Figures 1 to 6 The various processes implemented in the multivariate paradigm decoupled crowd counting method based on density level segmentation will not be described again here to avoid repetition.

[0089] In some embodiments, such as Figure 8As shown, this application embodiment also provides an electronic device 800, including a processor 801, a memory 802, and a computer program stored in the memory 802 and executable on the processor 801. When the program is executed by the processor 801, it implements the various processes of the above-described multi-paradigm decoupled crowd counting method embodiment based on density level segmentation and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0090] It should be noted that the electronic devices in the embodiments of this application include the mobile electronic devices and non-mobile electronic devices described above.

[0091] This application also provides a non-transitory computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it implements the various processes of the above-described multi-paradigm decoupled crowd counting method embodiment based on density level segmentation and achieves the same technical effect. To avoid repetition, it will not be described again here.

[0092] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0093] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described multi-paradigm decoupled crowd counting method based on density level segmentation.

[0094] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0095] This application also provides a chip, which includes a processor and a communication interface. The communication interface and the processor are coupled. The processor is used to run programs or instructions to implement the various processes of the above-described multi-paradigm decoupled crowd counting method embodiment based on density level segmentation, and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0096] It should be understood that the chip mentioned in the embodiments of this application may also be referred to as a system-on-a-chip, system chip, chip system, or system-on-a-chip, etc.

[0097] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element. Furthermore, it should be noted that the scope of the methods and systems in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0098] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a computer software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the multi-paradigm decoupled crowd counting method based on density level segmentation of the various embodiments of this application.

[0099] In the description of this application, "first feature" and "second feature" may include one or more of the features.

[0100] In the description of this application, "multiple" means two or more.

[0101] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

[0102] In the description of this specification, the references to terms such as "one embodiment," "some embodiments," "illustrative embodiment," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0103] Although embodiments of this application have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of this application, the scope of which is defined by the claims and their equivalents.

Claims

1. A multi-paradigm decoupled population counting method based on density level segmentation, characterized in that, The method includes: The target image is acquired and input into a segmentation mask generator trained by a Gaussian distribution mask generator to obtain a density level mask. The segmentation mask generator includes an image encoder and a density level segmentation module. The target image is input into the multivariate instance counting module to obtain the bounding box location information map, the point location information map, and the density location information map; The density level mask, bounding box location information map, point location information map, and density location information map are input into the result fusion module to obtain the composite result map; The total number of people in the target image is obtained by summing the composite result images.

2. The multi-paradigm decoupled population counting method based on density level segmentation according to claim 1, characterized in that, The step of inputting the target image into a segmentation mask generator trained by a Gaussian distribution mask generator to obtain a density-level mask includes: The target image is input into an image encoder for image encoding and position encoding to obtain encoded features. The image encoder includes a block embedding Transformer encoding module and a position encoding module. The encoded features are input into the density level segmentation module to perform multi-stage feature enhancement, resulting in a density level mask.

3. The multi-paradigm decoupled population counting method based on density level segmentation according to claim 2, characterized in that, The process of inputting the target image into an image encoder for image encoding and position encoding to obtain encoded features includes: The target image is input into the block embedding Transformer encoding module, and the image is divided into multiple image blocks through block embedding. Flatten multiple image patches to obtain multiple image patch vectors; Multiple image patch vectors are positionally encoded to obtain an image patch sequence; The image patch sequence is input into multiple Transformer layers, and global and local features of the image patch sequence are extracted through MHSA and feedforward neural networks to obtain encoded features.

4. The multi-paradigm decoupled population counting method based on density level segmentation according to claim 2, characterized in that, The step of inputting the encoded features into the density level segmentation module to perform multi-stage feature enhancement on the encoded features to obtain a density level mask includes: The encoded features are input into the transposed convolution module for upsampling to restore the spatial size of the encoded features and obtain the preliminary upsampled features; The initial upsampled features are input into the activation layer for further extraction and refinement of local contextual features to obtain local enhanced features; The local enhancement features are input into the multi-head self-attention module to model the global dependency relationship of the two-dimensional spatial dimension of the local enhancement features, capture contextual information, and obtain global perception features; The globally perceived features are input into the two-dimensional convolutional feature mapping module to perform channel dimension mapping and density level prediction, thereby obtaining the density level mask.

5. The multi-paradigm decoupled population counting method based on density level segmentation according to claim 1, characterized in that, The step of inputting the target image into the multivariate instance counting module to obtain a bounding box location information map, a point location information map, and a density location information map includes: The target image is input into the bounding box detection module, and features are extracted by the first feature extractor to obtain multiple feature maps of different scales. The multiple feature maps of different scales are input into the top-down feature modulation module respectively, and the multiple feature maps of different scales are integrated through the scale feedback mechanism to obtain the bounding box position information map. The target image is input into the point localization module, and the point localization feature map is obtained by the second feature extractor. The point localization feature map is upsampled and then input into the convolution module for depth convolution to obtain the point position information map. The target image is input into the density map generation module, where features are extracted by the third feature extractor and upsampling and focus localization are performed by the focus density map regression head to obtain a density location information map. The multivariate example counting module includes a bounding box detection module, a point localization module, and a density map generation module. The density map generation module includes a third feature extractor and a focal density map regression head.

6. The multi-paradigm decoupled population counting method based on density level segmentation according to claim 5, characterized in that, The process involves inputting the target image into the density map generation module, performing feature extraction through a third feature extractor, and upsampling and focus localization through a focus density map regression head to obtain a density location information map, including: The target image is input into the density map generation module, and features are extracted by the third feature extractor to obtain the first multi-scale feature map, the second multi-scale feature map, the third multi-scale feature map and the fourth multi-scale feature map; The first multi-scale feature map, the second multi-scale feature map, the third multi-scale feature map, and the fourth multi-scale feature map are input into the transition layer for feature fusion to obtain a multi-scale fused feature map. The multi-scale fused feature map is input into the convolutional layer and the normalization layer for depth convolution and normalization to obtain the first convolutional feature map; The first convolutional feature map is input into the activation layer for activation, resulting in the second convolutional feature map. The second convolutional feature map is input into a two-dimensional transposed convolutional layer for transposition, and candidate local maxima are extracted to obtain a density location information map.

7. The multi-paradigm decoupled population counting method based on density level segmentation according to claim 1, characterized in that, The process involves inputting the density level mask, bounding box location information map, point location information map, and density location information map into the result fusion module to obtain a composite result image, including: The density level mask is represented as a box binary mask, a dot binary mask, and a density binary mask according to the size of the density region; Element-wise multiplication of the binary mask and the box position information map yields the filter box position information map; Element-wise multiplication of the point binary mask and the point location information map yields the filtered point location information map; Element-wise multiplication of the density binary mask and the density location information map yields the filtered density location information map. The filter box location information map, filter point location information map, and filter density location information map are summed pixel by pixel to obtain the composite result map.

8. A multi-paradigm decoupled population counting system based on density level segmentation, implemented using the multi-paradigm decoupled population counting method based on density level segmentation as described in any one of claims 1 to 7, characterized in that, The system includes: The acquisition module is used to acquire the target image and input the target image into a segmentation mask generator trained by a Gaussian distribution mask generator to obtain a density level mask. The segmentation mask generator includes an image encoder and a density level segmentation module. The processing module is used to input the target image into the multivariate instance counting module to obtain the bounding box location information map, the point location information map, and the density location information map; The fusion module is used to input the density level mask, the bounding box location information map, the point location information map, and the density location information map into the result fusion module to obtain the composite result map; The counting module is used to sum the results based on the composite image to obtain the total number of people in the target image.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the multi-paradigm decoupled crowd counting method based on density level segmentation as described in any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the multi-paradigm decoupled crowd counting method based on density level segmentation as described in any one of claims 1 to 7.

Images