A few-shot counting method fusing density map and target detection double branches

Abstract

This invention discloses a few-shot counting method that integrates density map and object detection in two branches, belonging to the field of computer vision and image processing technology. This method constructs object detection and density map estimation branches in parallel on a unified feature base, achieving deep fusion of deep semantics and shallow details through multi-scale cross-attention interaction. During the training phase, a density loss function based on near-end mapping is used to construct non-overlapping jurisdictions through nearest-neighbor assignment, forcing the density map to form sharp extreme responses at the object center, thus alleviating feature adhesion in dense regions. During the inference phase, the local crowding is determined by calculating the inclusion rate between detection candidate boxes. In crowded regions, mask-guided density map integration counting is automatically used to achieve adaptive fusion of sparse and dense scenes. This invention improves counting accuracy and robustness in extremely crowded and occluded scenes while retaining instance-level localization capabilities.

Description

Technical Field

[0001] This invention belongs to the field of computer vision and image processing technology, specifically relating to a few-sample counting method that integrates density maps and object detection in two branches. Background Technology

[0002] The goal of few-shot object counting is to locate and count all object instances belonging to the same category in an image based on a small number of bounding boxes provided by the user. This task has wide applications in scenarios such as crowd counting, cell counting, and remote sensing object statistics. Unlike general object detection, few-shot object counting often faces the challenges of drastic changes in object scale and dense occlusion.

[0003] Early work primarily employed the density map regression paradigm. This approach transforms discrete target point labels into a continuous density map using a Gaussian kernel, then trains a deep convolutional network to regress the density map, finally obtaining the global count through spatial integration of the density map. While density map regression exhibits good robustness for estimating the overall number of dense targets, it also has significant limitations. Firstly, the density map itself does not provide boundary information for individual targets, making it difficult to directly output instance-level location and scale. Secondly, in regions with highly dense targets, the labels of different targets are very close together, causing their corresponding Gaussian response kernels to overlap, resulting in a spatially contiguous and blurred distribution of the network's output response. In this case, the local maxima of the density map can no longer reliably correspond to the center of a single target, leading to difficulties in distinguishing adjacent independent individuals and making local counting prone to errors.

[0004] Another type of work borrows the idea of ​​general object detection, directly predicting the bounding box and corresponding confidence score of each object instance in the network. These methods can provide high localization accuracy in scenarios with sparse targets and light occlusion. However, in areas of severe occlusion and dense overlap, the high degree of feature overlap makes it difficult for the classification and regression branches to provide clear individual separations, and the non-maximum suppression algorithm (NMS) also struggles to correctly filter out redundant boxes, resulting in a large number of missed and false detections. Furthermore, conventional detection loss functions mainly rely on bounding box annotations for optimization, making it difficult to obtain accurate spatial regression gradients under weak supervision with only center point annotations, further limiting the counting accuracy of the detection paradigm in dense scenes.

[0005] In recent years, some works have attempted to unify density regression and instance detection into a single network framework. Some methods, through dense query mechanisms, combine object detection and segmentation branches in small-sample counting, achieving performance superior to pure density regression methods on datasets such as FSC147. However, these methods primarily rely on centrality classification loss and bounding box regression loss when handling object localization. When training annotations are limited to center points and objects are extremely dense, the centrality loss cannot impose sufficiently strict spatial constraints on the global density distribution. The network's centrality response in dense regions tends to flatten and become blurred, local peaks no longer stand out, leading to difficulties in accurately distinguishing adjacent objects and significantly impacting local counting and localization accuracy.

[0006] Furthermore, in density map estimation tasks, traditional methods commonly employ the "intersection hypothesis" to construct supervisory signals, meaning that the density value at the same pixel location is allowed to be influenced by the combined effects of Gaussian kernels from multiple neighboring labeled points. This approach can provide relatively smooth training targets in sparse or moderately dense scenes. However, in extremely dense scenes, the mixing of multiple overlapping Gaussian kernels weakens the density difference between the target center and non-center regions, resulting in unclear response boundaries between adjacent targets. This problem further exacerbates feature adhesion between instances, hindering the detection branch's perception of target boundaries in the joint architecture.

[0007] Therefore, how to construct a density branch to impose macroscopic constraints on global quantity estimation while retaining the advantages of instance-level localization in the detection branch, and how to alleviate response ambiguity and feature adhesion in dense scenes through more effective spatial supervision signals, so as to improve the accuracy and robustness of small sample counting in complex scenes, is a technical problem that urgently needs to be solved in this field. Summary of the Invention

[0008] To address the aforementioned problems, this invention provides a few-sample counting method that integrates density mapping and target detection in a dual-branch manner.

[0009] The technical solution of this invention is as follows:

[0010] A few-sample counting method fusing density map and object detection dual branches includes the following steps:

[0011] S1. Extracting Multi-Scale Feature Pyramids

[0012] The query image is input into the backbone network to obtain a multi-scale feature pyramid. Deep features Capturing global semantics, shallow features Preserve high-resolution spatial details.

[0013] S2. Construct a target prototype that integrates appearance and geometric information.

[0014] Extracting appearance features from multi-scale features using example bounding boxes At the same time, adjust the frame width and height. Mapped to shape embedding via a multilayer perceptron (MLP) The target prototype is obtained by splicing. .

[0015] S3. Prototype Feature Fusion and Query Vector Generation

[0016] Initialize query vector This represents the global characteristics of the backbone network. (Through...) Step-by-step cross-attention iteration:

[0017]

[0018] in This represents a standard cross-attention operation, with three parameters corresponding to the query, key, and value, respectively. Each step of the cross-attention calculation includes residual connections and layer normalization operations. For the first The query vector after each step is gradually infused with prototype information, and finally the target-aware query vector is output. .

[0019] S4, Cross-scale deformable attention feature interaction

[0020] For each query vector, aggregate multi-scale features:

[0021]

[0022] in For the number of levels, For query vector, As the reference point coordinates, For the first Layer feature map, The number of sampling points per level. For attention weights, This is the sampling offset. This indicates that eigenvalues ​​for continuous coordinate positions are obtained through bilinear interpolation.

[0023] S5, Cross-scale Feature Aggregation and Spatial Reconstruction

[0024] The interactive features are decoupled according to resolution and reshaped into two-dimensional feature maps. A top-down progressive fusion is then used to output a target-aware feature map and extract auxiliary features. Used to assist in supervision.

[0025] S6, Dual-branch parallel prediction

[0026] The target-aware feature map is fed into both the target detection branch and the density map estimation branch.

[0027] The object detection branch consists of a centrality classification head and a bounding box regression head. The classification head converts features into centrality confidence scores using a linear mapping. The bounding box regression head predicts the bounding box coordinates of the object at each location using a multilayer perceptron, and uses a sigmoid function to ensure that the output values ​​are confined to the (0,1) interval.

[0028] The density map estimation branch consists of a fully convolutional density subnetwork and a density mapping layer. The density subnetwork is composed of two cascaded 3×3 convolutional layers, and the output is guaranteed to be non-negative through the softplus activation function, generating a continuous density map.

[0029] S7. Calculate the joint loss and optimize the network.

[0030] The joint loss is a weighted sum of the target detection branch loss and the density branch loss.

[0031] Object detection branch loss: First, using the labeled center points as anchors, the prediction network dynamically generates pseudo-ground truth bounding boxes, providing a geometric supervision benchmark for point labeling. Then, local maxima points on the predicted response map are extracted as candidates. The Hungarian algorithm is used for optimal bipartite graph matching between candidate boxes and pseudo-ground truth boxes, classifying candidate points into three categories: true positives, false positives, and false negatives. Based on this classification, the centerness classification loss based on the effective mask and the bounding box regression loss for true positive samples are calculated. The bounding box regression loss uses L1 loss and generalized intersection-union ratio (IoU). A linear combination of losses.

[0032] Density branch loss: A density loss based on near-end mapping (PMLoss) is introduced. Non-overlapping jurisdictions are constructed through a nearest-neighbor assignment mechanism, and near-end gradient descent is used to dynamically find reachable targets, forcing the network to form sharp extrema at the target center. The optimization objective of the density estimation branch is expressed as:

[0033]

[0034] in, This is the predicted density distribution vector within the local jurisdiction. P is the centered distance cost vector; P is the nearest neighbor matching matrix; For hyperparameters; To adopt The norm's counting constraint requires that the predictive density integral within each local jurisdiction strictly approximate 1.

[0035] The spatial pixel coordinate tensor is divided into multiple fixed-size chunks. Local distances are calculated independently within each chunk, and nearest neighbor indices are extracted. Once completed, the intermediate tensor memory is released immediately to reduce space complexity.

[0036] In addition, the auxiliary features of the intermediate hidden layer introduced in step S5 are fed into the auxiliary prediction head to calculate the auxiliary loss.

[0037] S8, Adaptive Fusion Counting in the Inference Phase

[0038] During the inference phase, the raw outputs of the two branches of the network are first obtained. For the initial bounding boxes output by the detection branch, a fine-grained segmentation mask is generated using the frozen SegmentAnythingModel2 (SAM2) mask decoder as a cue, and this mask is used to refine the bounding boxes. Subsequently, the bounding boxes are sorted in descending order of the updated confidence scores, and a non-maximum suppression algorithm (nms) is performed to remove redundant boxes and filter out out-of-bounds boxes, resulting in a set of candidate detection boxes.

[0039] Next, for any two candidate boxes, calculate their one-way containment ratio, which is the ratio of their intersection area to their own area. Count the number of highly contained bounding boxes. When this number exceeds a preset threshold, the current local region is determined to be a crowded region, triggering density branch intervention; otherwise, the number of candidate boxes is used as the count result.

[0040] The one-way inclusion rate Defined as:

[0041]

[0042] in, and For any two candidate boxes, Indicates the area of ​​the box. This represents the intersection area of ​​the two boxes. When Exceeding the preset threshold At that time, see quilt Highly inclusive.

[0043] After triggering the crowding check, all candidate bounding box coordinates are mapped to the spatial resolution of the density map, generating a binary joint mask. The bounding box coverage area is set to 1, and the background is set to 0. The final fused count value... Calculate using the following formula:

[0044]

[0045] in, Density branch in spatial location The predicted density value at that location, This is the binary joint mask generated from all candidate boxes. This indicates element-wise multiplication.

[0046] The present invention has the following beneficial effects:

[0047] Combining the micro-instance-level localization advantage of the detection branch with the macro-robustness of the density branch, this architecture effectively solves the problems of missed detections and boundary adhesion under dense occlusion. An innovative block-based pipelined computation mechanism addresses the memory overflow challenge of PMLoss in high-resolution images, enabling stable training in extremely dense scenes. Through an adaptive fusion strategy, the model dynamically switches prediction modes based on local crowding, ensuring good counting accuracy in both sparse and extremely dense scenes. Utilizing the general features and prototype feature extraction mechanisms of large-scale visual models enhances the model's ability to perceive unseen categories. Attached Figure Description

[0048] Figure 1 This is the overall flowchart of the present invention;

[0049] Figure 2 This is a schematic diagram of cross-scale feature aggregation and target perception reconstruction; Detailed Implementation

[0050] This embodiment provides a few-sample counting method that integrates density mapping and target detection in a dual-branch manner. The overall process is as follows: Figure 1 As shown, this method constructs object detection and density map estimation branches in parallel on a unified feature base. Through multi-scale cross-attention interaction, density loss based on proximal mapping, and an adaptive inference fusion strategy, it achieves collaborative optimization of macroscopic counting statistics and microscopic instance localization. The following sections explain each step in detail.

[0051] S1. Extract the multi-scale feature pyramid of the query image.

[0052] In this embodiment, the backbone network uses a pre-trained visual baseline model with feature pyramid output capability, the Segmentation-All Model (SAM2). The spatial resolution is... Query Image The input is passed to the backbone network for forward propagation. The network outputs a multi-scale feature pyramid, specifically including: deep global semantic features. It has a lower spatial resolution but the largest receptive field of context; and two high-resolution feature maps from a shallower layer. and These three layers of features constitute a multi-scale feature pyramid set. The shallow features retain rich edge and texture information, while the deep features capture the global semantic context. This multi-scale representation strategy provides the necessary data foundation for subsequent cross-scale interactions and bi-branch predictions.

[0053] S2. Construct a target prototype that integrates appearance and geometric information.

[0054] The user provides a small number of example bounding boxes, each annotating the location of an example target in the query image. The target prototype built in this step contains both appearance and geometric information of the examples.

[0055] Apparent feature extraction: For each example bounding box, RoI-Align technique is used to extract features from multi-scale feature maps. Local feature tensors for corresponding regions are pooled and flattened into fixed-dimensional appearance features. These multi-scale appearance features are then concatenated to obtain the appearance feature vector. .

[0056] Shape embedding generation: Extracting the width of the example bounding box and height , forming a two-dimensional shape vector The vector is fed into a lightweight MLP and mapped to... Generate shape embedding vectors with the same embedding dimension. .

[0057] Prototype concatenation: The appearance features and shape embedding vectors are concatenated along the channel dimension to obtain the target prototype.

[0058]

[0059] S3. Prototype Feature Fusion and Query Vector Generation

[0060] Specifically, firstly, the global image spatial features extracted by the backbone network are... Initialize as query vector , Indicates the feature hierarchy. The target prototype constructed in step S2... As keys and values, iterative interaction is then performed through a multi-step cascaded cross-attention mechanism. In this embodiment, the number of cross-attention iteration steps is... Set to 3, the first The iterative process of step 1 is as follows:

[0061]

[0062] in This represents a cross-attention operation, with three parameters corresponding to the query, key, and value, respectively. Each step of the cross-attention calculation includes residual connections and layer normalization operations, ultimately outputting a target-aware query vector. .

[0063] After multiple iterations, the image query vector is injected with target perception capability, outputting a set of prototype-guided feature representations as input signals for subsequent cross-scale matching.

[0064] S4, Cross-Attention Feature Interaction

[0065] This step involves interacting the target-aware query vector generated in step S3 with the multi-scale feature pyramid extracted in step S1.

[0066] Reference point prediction: for each query vector A reference point is predicted in a normalized two-dimensional image space through a linear mapping layer. (range of values) This reference point can be considered as the initial anchoring position of the query vector on the image.

[0067] Sampling offset and weight prediction: for each feature level ( The query vector is used to predict two sets of parameters through parallel fully connected layers:

[0068] Dynamic sampling offset ,common Each sampling point has an offset of continuous floating-point values;

[0069] Attention weight Normalization using the softmax function satisfies .

[0070] Feature aggregation: using bilinear interpolation to aggregate features from feature maps at various levels. Local features are extracted from sub-pixel locations and summed using attention weights:

[0071]

[0072] in This indicates that eigenvalues ​​for continuous coordinate positions are obtained through bilinear interpolation.

[0073] S5, Cross-scale Feature Aggregation and Spatial Reconstruction

[0074] This step aggregates and reconstructs multi-scale features, with the specific structure as follows: Figure 2 As shown.

[0075] First, the multi-scale feature sequences after interaction are decoupled according to a preset three-level spatial resolution. The three levels of spatial resolution correspond to three scales: high-level... Middle layer bottom layer By using tensor transpose and reshaping operations, features at various scales are mapped back to a two-dimensional spatial feature map.

[0076] A top-down progressive fusion pipeline is employed. The first step involves upsampling the high-level features: this upsampling layer consists of bilinear interpolation (scaled up by 2x), 3×3 convolutions, and a cascaded GELU activation function. After aligning the amplified high-level and mid-level features in spatial resolution, they are concatenated along the channel dimension. The number of channels is doubled after concatenation and fed into the fusion module: this module consists of 3×3 convolutions, group normalization, and a ReLU activation function.

[0077] The second step involves upsampling and amplifying the fusion result from the first step by a factor of 2, then stitching and fusing it with the highest resolution underlying features, following the same procedure as the first step.

[0078] After a separate additional upsampling operation, a set of high-resolution target-aware feature maps is output. In this feature space, regions similar to the target prototype are highly activated, while background regions are suppressed, providing a clean and unified representational basis for subsequent bi-branch prediction.

[0079] Furthermore, to accelerate the training convergence of deep networks, an auxiliary feature is extracted by upsampling the intermediate layers during the aforementioned fusion process. This represents the feature representation of the network during the intermediate stage of training and will be used for auxiliary supervision in step S7.

[0080] S6, Dual-branch parallel prediction

[0081] The target-aware feature map output from step S5 is simultaneously fed into the target detection branch and the density map estimation branch. The two branches share the underlying features but each has its own independent prediction head.

[0082] S6.1, Target Detection Branch

[0083] This branch consists of a centrality classification head and a bounding box regression head.

[0084] The centrality classification head flattens the target-aware feature map and then maps the high-dimensional embedded features into single-channel confidence scores through a fully connected linear layer, followed by a LeakyReLU activation function to introduce non-linearity. The output is a two-dimensional centrality probability map, where the value at each spatial location represents the confidence that the point belongs to a certain target center region. Pixels closer to the true target center should have higher prediction scores.

[0085] Bounding box regression head: Consists of a three-layer multilayer perceptron, with each layer alternately using linear mapping and ReLU activation functions to progressively reduce dimensionality. The final output is a feature tensor with four channels, each corresponding to the normalized distance of the four edges of the target bounding box relative to its center point. This refers to the offset from the target center to the top, left, right, and bottom boundaries. A sigmoid activation function is introduced at the end of the network to force the output value to be normalized. Within the range.

[0086] S6.2, Density Map Estimation Branch

[0087] This branch first performs local spatial smoothing of the features using a fully convolutional density subnet. The density subnet consists of two cascaded 3×3 convolutional layers, with convolution padding set to 1 to strictly maintain the spatial resolution of the feature map, and the GELU activation function is used between layers.

[0088] The smoothed features after density subnetting are compressed to a single channel through a density mapping layer to generate the final 2D density map. A softplus activation function is then used after the mapping layer to ensure the output is strictly non-negative, where x is the input value.

[0089]

[0090] Ultimately, the sum of pixel values ​​within any spatial region on the density map is an estimate of the total number of targets within that region.

[0091] S7. Calculate the joint loss and optimize the network.

[0092] This step constructs a joint loss function that includes both object detection branch loss and density branch loss, and optimizes it through end-to-end backpropagation using weighted summation. Deep auxiliary supervision is also introduced during training to accelerate convergence.

[0093] S7.1 Generation of pseudo-truth value bounding boxes

[0094] In this embodiment, the coordinates of the labeled center point are used as anchor points, and a multilayer perceptron adaptively regresses the offset of the target corresponding to that point. After converting the offset into bounding box coordinates, each labeled point is expanded into a complete pseudo-ground truth box.

[0095] S7.2, Optimal matching of bipartite graphs

[0096] After detecting dense predicted responses from the branch outputs, the boxes corresponding to local maxima on the response map are extracted as candidate predicted boxes. The Hungarian algorithm is used to perform optimal bipartite graph matching between the candidate boxes and the set of pseudo-true boxes to construct a global cost matrix that includes classification cost and regression cost.

[0097] After matching, all candidate points are divided into three mutually exclusive states: true positive samples (TP) that have been successfully matched, false positive samples (FP) that have not matched any real targets, and false negative samples (FN) that have not been assigned to any predicted bounding boxes corresponding to real targets. For FN samples, the center coordinates of their real bounding boxes are actively extracted, forcibly added back to the candidate point list and marked as FN, guiding the network to refocus on these missed regions in subsequent training.

[0098] S7.3, Target Detection Branch Loss

[0099] The object detection branch loss is composed of the centrality classification loss. and bounding box regression loss composition.

[0100] The centrality classification loss employs a squared error loss based on an effective mask. For TP and FN samples, the centrality pseudo-labels are... Set to 1; for FP samples and absolute background regions, Set to 0.

[0101]

[0102] in For the network in spatial location The predicted centrality score, This represents the total number of targets in the current batch, used for normalization.

[0103] The bounding box regression loss is calculated only on the TP samples, using a linear combination of L1 loss and generalized intersection-over-union loss:

[0104]

[0105] in For the prediction box, For the corresponding pseudo-truth boxes, For generalized intersection-union ratio functions, A true positive sample that has been successfully matched. and This is the balance coefficient between the two losses.

[0106] The total loss of the object detection branch is:

[0107]

[0108] S7.4, Density Branching Loss

[0109] The density branch loss employs a density loss function that avoids the Gaussian kernel intersection assumption. A dynamically reachable target distribution is constructed using a proximal mapping method, thereby forming a sharp extreme response at the target center.

[0110] The loss function consists of the following two items:

[0111] Background penalty term: Within the jurisdiction block, using the distance cost from the centered pixel to the target point, negative gradients are automatically applied to pixels very close to the target to increase density values, while positive penalties are applied to background pixels far from the target to suppress density output.

[0112] Counting constraint: This forces the sum of the density pixels within each jurisdiction block to be close to 1, meaning each jurisdiction block represents exactly one target. In this embodiment, the L1 norm is used to calculate this constraint to enhance robustness against manual annotation offsets.

[0113] The high-dimensional spatial pixel coordinate tensor is divided into multiple fixed-size data blocks along the sequence dimension; computation is performed only within the current data block using a broadcast mechanism to calculate the local L2 distance and extract the nearest neighbor index; the corresponding intermediate tensor memory is released immediately after each data block is processed. This mechanism reduces the space complexity of distance computation from... Compress to ,in The block size can be decoupled from the overall size of the density map.

[0114] S7.5, Auxiliary Supervision

[0115] The auxiliary features of the intermediate hidden layer extracted in step S5 The auxiliary dual-branch prediction head, which shares weights with the main branch, is fed in, and the auxiliary classification loss, auxiliary bounding box regression loss, and auxiliary density loss are calculated independently. The auxiliary loss is calculated in the same way as the main loss, only the source of its input features is different.

[0116] S7.6, Joint Optimization

[0117] The final joint loss is a weighted sum of the target detection branch loss, density branch loss, and auxiliary loss:

[0118]

[0119] For the combined total loss, For the loss of the object detection branch, To employ density branch loss based on a density loss function with proximal mapping, To assist in predicting the head's auxiliary loss, To assist in determining the loss weight, this embodiment sets it to 0.3. In this embodiment, the weight for density loss is set to 0.3.

[0120] S8, Adaptive Fusion Counting in the Inference Phase

[0121] The goal of the inference phase is to effectively combine the discrete predictions of the detection branch and the continuous predictions of the density branch. This embodiment employs a strategy of calibration first, then decision-making, and finally selective fusion.

[0122] S8.1 Post-processing calibration of the detection frame

[0123] First, the initial bounding box is used as a spatial cue and input into the SAM2 mask decoder with frozen weights. The mask decoder generates a fine-grained segmentation mask of the target and simultaneously predicts the intersection-over-union (IoU) score of the mask. Then, calibration is performed, extracting the minimum bounding rectangle of the mask as the corrected bounding box, and replacing the original classification confidence of the detector head with the IoU score.

[0124] After calibration, the boxes are sorted in descending order of the updated confidence level. Standard non-maximum suppression is then performed to remove highly overlapping redundant boxes and to filter out invalid boxes that exceed the physical boundaries of the image, ultimately obtaining a set of candidate detection boxes.

[0125] S8.2, Crowding Determination and Adaptive Fusion

[0126] In crowded, occluded scenes, aliasing of target features often leads to incorrect nesting of "large boxes containing smaller boxes" in the detection branch output. To identify this situation, define any two candidate boxes... and One-way inclusion rate between:

[0127]

[0128] in Indicates the area of ​​the box. This indicates the intersection area of ​​the two frames.

[0129] Count the number of images that satisfy the following conditions in the current image. The number of bounding boxes that are highly contained within the bounding box. In this embodiment... The threshold is set to 0.9, and the trigger threshold is set to 1. When the number of boxes contained in a height exceeds the threshold, in this embodiment the threshold is set to 5, indicating that severe feature aliasing has occurred in the current local region, and the detection branch can no longer provide a reliable count, so the density branch is used to intervene; otherwise, the number of candidate detection boxes is used directly as the final count result.

[0130] S8.3, Mask-guided density integral fusion

[0131] Once it is determined that density branch intervention is needed, density integration based on detection mask is used for counting. Specifically, the coordinates of all candidate boxes are mapped back to the density map spatial resolution, pixels inside the box are set to 1, and pixels outside the box are set to 0, generating a binary joint mask. The final fusion count value is predicted by the probability plot of the density branch. The result is obtained by spatial integration within the masked region, as shown in the formula below. Element-wise multiplication:

[0132]

[0133] Technical effect verification

[0134] The experimental results are shown in Tables 1 and 2, illustrating the counting and detection performance of different methods on various datasets. It can be seen that the method of this invention achieves superior results in all cases.

[0135] Specifically, in terms of counting performance, this embodiment achieves a mean absolute error (MAE) of 7.82 and a root mean square error (RMSE) of 48.33 on the test set. Compared to the GeCo method, which also employs a detection-counting joint architecture, the MAE decreases by approximately 1.1%, and the RMSE decreases by approximately 11.0%. In terms of detection performance, this embodiment achieves a mean accuracy (AP) of 45.21 on the test set, an improvement of approximately 4.1% compared to GeCo (43.42); the AP50 is 74.84, at the same level as GeCo (75.06), indicating that the introduction of density branching did not sacrifice detection accuracy.

[0136] In the cross-domain generalization test, the model trained on FSCD-147 was directly applied to the unseen categories of FSCD-LVIS. In this embodiment, the MAE was 14.83 and the RMSE was 26.59, which verified the adaptability of the method to unseen categories.

[0137] Table 1. Counting and detection results of different methods in FSCD 147

[0138]

[0139] Table 2. Count and detection results of different methods in the FSCD LVIS unseen category.

[0140]

[0141] Through the above steps, this embodiment achieves collaborative operation of the detection and density branches. In sparse scenarios, the detection branch provides accurate instance-level localization and reliable direct counting; in dense and crowded scenarios, the density branch takes over the counting function to ensure counting stability; the adaptive switching between the two during the inference phase achieves robust few-sample counting across all scenarios.

[0142] Of course, the above description is not intended to limit the present invention, and the present invention is not limited to the examples given above. Any changes, modifications, additions or substitutions made by those skilled in the art within the scope of the present invention should also fall within the protection scope of the present invention.

Claims

1. A few-shot counting method of fusing density map and target detection double branch, characterized in that, Includes the following steps: S1, input the query image into the backbone network to obtain a multi-scale feature pyramid wherein the deep features capture global semantics, and the shallow features retain high-resolution spatial details; S2, extracting appearance features from multi-scale features using example bounding boxes , while the box width height mapped to shape embeddings via a multi-layer perceptron (MLP) , and concatenating to obtain target prototypes ; S3, initialize query vector For the global feature of the backbone network, through Step cross-attention iteration: ， in This represents a standard cross-attention operation, with three parameters corresponding to the query, key, and value, respectively. Each step of the cross-attention calculation includes residual connections and layer normalization operations. For the first The query vector after each step is gradually infused with prototype information, and finally the target-aware query vector is output. ; S4. For each query vector, aggregate multi-scale features: ， in For the number of levels, For query vector, As the reference point coordinates, For the first Layer feature map, The number of sampling points per level. For attention weights, This is the sampling offset. This indicates that eigenvalues ​​of continuous coordinate positions are obtained through bilinear interpolation; S5. Decouple the interactive features according to resolution and reshape them into two-dimensional feature maps. Use a top-down progressive fusion method to output a target-aware feature map and extract auxiliary features. Used to assist in supervision; S6. Simultaneously feed the target-aware feature map into the target detection branch and the density map estimation branch: The object detection branch includes a centrality classification head and a bounding box regression head. The classification head converts features into centrality confidence scores through a linear mapping. The bounding box regression head predicts the bounding box coordinate parameters of the target at each location using a multilayer perceptron and uses the sigmoid function to ensure that the output value is limited to the (0,1) interval. The density map estimation branch includes a fully convolutional density subnetwork and a density mapping layer. The density subnetwork is composed of two cascaded 3×3 convolutional layers and the output is guaranteed to be non-negative through the softplus activation function, generating a continuous density map. S7. Calculate the joint loss and optimize the network. Define the joint loss as a weighted sum of the target detection branch loss and the density branch loss. Object detection branch loss: First, using the labeled center point as the anchor point, the prediction network dynamically generates pseudo-ground truth bounding boxes, providing a geometric supervision benchmark for point labeling; then, local maxima points on the predicted response map are extracted as candidates, and the Hungarian algorithm is used to perform optimal bipartite graph matching between candidate boxes and pseudo-ground truth boxes, classifying candidate points into three categories: true positives, false positives, and false negatives; based on the three-class classification, the centerness classification loss based on the effective mask and the bounding box regression loss for true positive samples are calculated; the bounding box regression loss uses L1 loss and generalized intersection-union ratio (IoU). A linear combination of losses; Density branch loss: A density loss based on near-end mapping (PMLoss) is introduced. Non-overlapping jurisdictions are constructed through a nearest-neighbor assignment mechanism, and near-end gradient descent is used to dynamically find reachable targets, forcing the network to form sharp extrema at the target center. The optimization objective of the density estimation branch is expressed as: ， in, This is the predicted density distribution vector within the local jurisdiction. P is the centered distance cost vector; P is the nearest neighbor matching matrix; For hyperparameters; To adopt The norm's counting constraint requires that the predictive density integral within each local jurisdiction be strictly approximated to 1; The network is optimized for end-to-end backpropagation by weighted summation; S8, Reasoning Stage: First, the original outputs of the two branches of the network are obtained. For the initial bounding boxes of the detection branch output, the frozen segmentation model mask decoder is used to generate a fine segmentation mask with the bounding boxes as cues. The bounding boxes are then corrected using this mask. Subsequently, the boxes are sorted in descending order of the updated confidence scores, and the non-maximum suppression algorithm (NMS) is executed to remove redundant boxes and filter out out-of-bounds boxes, resulting in a set of candidate detection boxes. Next, for any two candidate boxes, calculate their one-way containment ratio, which is the ratio of the intersection area to the area of ​​the box itself; count the number of bounding boxes that are highly contained. When the number exceeds a preset threshold, the current local region is determined to be a crowded region, and density branch intervention is triggered; otherwise, the number of candidate boxes is used as the counting result. The one-way inclusion rate is defined as: ， in, and For any two candidate boxes, Indicates the area of ​​the box. This represents the intersection area of ​​the two boxes, when Exceeding the preset threshold At that time, see quilt Highly inclusive; After triggering the crowding check, all candidate box coordinates are mapped to the spatial resolution of the density map, generating a binary joint mask. The box coverage area is set to 1, and the background is set to 0. Finally, the count value is fused. Calculate using the following formula: ， in, Density branch in spatial location The predicted density value at that location, This is the binary joint mask generated from all candidate boxes. This indicates element-wise multiplication.

Images