A small sample target counting method, system, device and storage medium

Abstract

This invention relates to the fields of computer vision and artificial intelligence, and discloses a method, system, device, and storage medium for small-sample target counting. The method includes: extracting deep features from a query image and reference samples, and constructing a relevance response feature map; extracting background priors through multi-size average pooling, performing difference operations and nonlinear mapping on the background priors and the relevance response feature map to generate gating weights and modulate the relevance response features, and then obtaining an initial density map through upsampling decoding; extracting candidate points and generating pseudo-boundary boxes based on the initial density map, and inputting the query image, candidate points, pseudo-boundary boxes, and reference sample boxes into a pre-trained segmentation model to obtain a mask; extracting semantic features of the candidates and references and clustering them, and setting interference regions to zero according to the clustering results to obtain a filtered final density map, which is used as the counting result. This invention can achieve adaptive sharpening and cross-class anti-interference matching for dense targets in images.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and artificial intelligence, and in particular to a method, system, device and storage medium for counting small samples of targets. Background Technology

[0002] Visual object counting, a fundamental perceptual task in computer vision, aims to accurately estimate the number of specific objects within a region of interest in an image or video. This technology has profound practical significance in areas such as smart supermarket shelf inventory management, crowd control in densely populated areas, and industrial warehousing and logistics. Early counting models typically followed a closed-set learning paradigm, training independently for a single specific category, requiring expensive manual annotation and failing to generalize to unknown categories. To overcome this limitation, Few-Shot Class-Agnostic Object Counting (FAO) technology emerged. This technology only requires the user to select a very small number of reference objects as visual cues on the query image, enabling generalized recognition and counting of objects of any unknown category. However, in real-world, unconstrained open environments, due to significant variations in object scale, dense physical occlusion and adhesion, and complex multi-class background interference, existing counting systems still face significant technical bottlenecks in terms of accuracy and robustness.

[0003] In recent years, class-agnostic counting has gradually become an important research direction in the field of visual perception due to its advantage of being able to generalize to unknown categories without retraining. The core paradigm of this type of method is to use a very small number of reference targets given by the user as visual cues to guide the network to perform similarity measurement and regress density maps in the query image, thereby achieving statistical results.

[0004] However, when such baseline paradigms are deployed in unconstrained real-world open environments, their accuracy and robustness often suffer significant losses. This is because existing mainstream architectures have multi-dimensional limitations when dealing with complex scenarios: First, due to the drastic spatial scale abrupt changes and disordered arrangement of objects in real-world scenes, the single fixed receptive field used by conventional models is difficult to adaptively adjust. This not only leads to feature adhesion of small targets in extremely dense areas but also causes response splitting of large-scale targets in feature maps, resulting in severe missed detections and local overcounting. Second, existing solutions typically rely solely on low-level local visual features for cross-correlation matching, lacking high-dimensional instance-level semantic gating mechanisms. When the image is filled with background interference that is highly similar to the local texture of the reference sample, purely low-level feature points are prone to semantic matching ambiguity, causing the system to generate dense, falsely high responses in non-target areas. Finally, at the network optimization level, existing end-to-end counting models generally rely on global mean squared error (MSE) as the loss function. This indiscriminate pixel-level penalty mechanism makes the gradient update direction of the network easily dominated by the "simple zero-value background" that occupies the majority of the image. As a result, the model lacks sufficient penalty for the "hard negative samples (such as complex shadows and messy textures)" that actually cause false positives, ultimately resulting in a significant accumulation of background noise. Summary of the Invention

[0005] To address the aforementioned issues, this invention proposes a small-sample target counting method, system, device, and storage medium. This method breaks the limitations of a single receptive field to achieve adaptive sharpening of multi-scale features, introduces strong visual priors for instance-level semantic decoupling, and applies targeted suppression to difficult backgrounds during the optimization phase. This enables adaptive sharpening of dense targets in images and cross-class anti-interference matching.

[0006] The technical solution adopted in this invention is as follows: A small sample target counting method, comprising: Extract deep features from the query image and the reference sample, and construct a relevance response feature map; Background priors are extracted by average pooling of various sizes. Differential operations and nonlinear mapping are performed on the background priors and the correlation response feature map to generate gated weights and modulate the correlation response features. Then, the initial density map is obtained by upsampling and decoding. Candidate points are extracted from the initial density map and pseudo-boundary boxes are generated. The query image, candidate points, pseudo-boundary boxes, and reference sample boxes are input into the pre-trained segmentation model to obtain a mask. Semantic features of the candidates and references are extracted and clustered. The interference regions are set to zero according to the clustering results to obtain the filtered final density map, which is used as the counting result.

[0007] Furthermore, the step of extracting deep features from the query image and the reference sample, and constructing a relevance response feature map, includes: The query image and the reference sample box are input into the basic counting network, and the encoder of the basic counting network extracts the deep features of the query image and the deep features of the reference sample, respectively. The deep features of the query image and the deep features of the reference samples are input into the object prototype extraction decoder. The object prototype extraction decoder generates the correlation response feature map by calculating the similarity between the features of the query image and the prototype features of each reference sample.

[0008] It should be noted that this method extracts deep semantic information from the query image and the reference sample through the encoder, and then uses the decoder to establish a similarity mapping between the query features and the reference prototype. This helps to effectively transfer the target prior of the reference sample to the query image, providing accurate relevance guidance for subsequent target localization and counting.

[0009] Furthermore, the step of extracting background priors through average pooling of multiple sizes, and performing difference operations and nonlinear mapping on the background priors and the relevant response feature maps, includes: The correlation response feature map is processed using a first-size average pooling method to obtain a fine-grained background prior; the correlation response feature map is then processed using a second-size average pooling method to obtain a coarse-grained background prior, wherein the first size is smaller than the second size. The fine-grained background prior and the coarse-grained background prior are subtracted from the correlation response feature map, respectively, to obtain the fine-grained differential response features and the coarse-grained differential response features; Channel-independent nonlinear mappings are performed on the fine-grained differential response features and the coarse-grained differential response features, respectively. The nonlinear mappings include depthwise convolution, batch normalization, and activation functions.

[0010] It should be noted that small-size pooling can retain more refined local background information, which helps to enhance subtle response differences between dense small targets and alleviate the problem of target adhesion; large-size pooling can capture a wider range of contextual constraints, which is beneficial to maintain the consistency of responses within large targets and avoid feature splitting; the combination of the two can differentiate and enhance target responses at different scales, thereby improving the quality of density maps.

[0011] Further, the process of generating gated weights and modulating the correlation response features, followed by upsampling and decoding to obtain the initial density map, includes: The correlation response feature map, the fine-grained differential response feature after nonlinear mapping, and the coarse-grained differential response feature after nonlinear mapping are concatenated along the channel dimension. The concatenated features are then convolved and spatial adaptive gating weights are generated through an activation function. The gating weights are then multiplied element-wise with the correlation response feature map to obtain the enhanced correlation response feature. The enhanced correlation response features are upsampled and decoded. Spatial resolution is restored and the number of channels is compressed by upsampling at each level to obtain the initial density map of a single channel.

[0012] It should be noted that by generating spatially adaptive gating weights through channel concatenation and convolution operations, the response intensity of the original features can be adaptively adjusted at different spatial locations, effectively highlighting the target region and suppressing background noise; stepwise upsampling decoding helps to gradually restore the spatial resolution of the feature map, thereby generating a high-quality initial density map, providing a reliable distribution of candidate targets for subsequent instance-level verification.

[0013] Further, the step of extracting candidate points and generating pseudo-boundary boxes based on the initial density map, and inputting the query image, candidate points, pseudo-boundary boxes, and reference sample boxes into a pre-trained segmentation model to obtain a mask includes: Local maxima extraction is performed on the initial density map, and peak points with confidence scores higher than preset conditions are selected as candidate cue points; pseudo bounding boxes are generated by proportionally expanding the bounding boxes of the reference sample bounding boxes according to the average size of the reference sample bounding boxes. The original query image, all candidate cue points, all pseudo bounding boxes, and reference sample boxes are input into the pre-trained segmentation model. Guided by both cue points and bounding box cue points, the segmentation model outputs the instance mask corresponding to each candidate target.

[0014] It should be noted that extracting local maxima from the initial density map can effectively locate the position of candidate targets. Generating pseudo bounding boxes by combining the scale of the reference samples helps to constrain the target search range of the segmentation model and avoid missegmentation in irrelevant areas. Point hints and box hints together guide the segmentation model, which can improve the boundary accuracy and target integrity of the instance mask and provide high-quality instance representations for subsequent semantic feature extraction and clustering verification.

[0015] Further, the step of extracting and clustering the semantic features of candidates and references, and setting the interference regions to zero according to the clustering results to obtain the filtered final density map includes: After removing invalid masks with confidence levels below the preset conditions, the masks of each instance are spatially weighted and aggregated with the deep coding features of the query image to obtain the semantic features of each candidate instance; the image regions within each reference sample box are cropped and enlarged and then input into the segmentation model to obtain the reference prototype mask; the semantic features of the reference sample are extracted based on the reference prototype mask and the deep coding features of the query image. After normalizing the semantic features of all candidate instances and the semantic features of the reference samples, they are concatenated into a global feature set. The semantic similarity between any two features is calculated and negative values ​​are truncated to obtain a non-negative semantic affinity matrix. Based on the semantic affinity matrix, a normalized graph Laplacian matrix is ​​constructed, and the feature vectors are solved to divide the candidate instances into foreground and background clusters. The distribution of semantic features of each reference sample in the foreground and background clusters is counted. The cluster containing the most reference samples is identified as the foreground target cluster, and the other cluster is identified as the potential interference cluster. The filtering gate condition is determined based on the proportion of reference samples in the foreground cluster and the inter-cluster similarity between the foreground and background clusters. If the condition is met, the region corresponding to the candidate instance belonging to the potential interference cluster is set to zero in the weight map. The weight map is multiplied pixel by pixel with the initial density map to obtain the final filtered density map.

[0016] It should be noted that spatial weighted aggregation based on instance masks can obtain semantic features with clear target boundary constraints, reducing the interference of background and outlier targets on semantic representation; dividing candidate instances into foreground and background clusters through spectral clustering can effectively distinguish between the real query target and visually similar non-target interference objects; setting gating conditions composed of the proportion of reference samples and inter-cluster similarity helps to avoid erroneous deletion of real targets, maintaining the recall ability of real targets while suppressing false activation, and ultimately improving counting accuracy.

[0017] Furthermore, the model training phase also includes difficult negative sample mining and joint loss optimization: A nonnegative constraint is applied to the output final density map to obtain a nonnegative prediction density map. The nonnegative prediction density map is compared pixel by pixel with the pre-constructed ground truth density map to calculate the pixel-by-pixel error. Based on the value of each pixel in the ground truth density map, positive sample regions and negative sample regions are divided. All pixel errors in the positive sample regions are retained to participate in gradient update. For the negative sample regions, pixels with errors higher than a preset quantile are selected as difficult negative samples to participate in gradient update, and the remaining negative sample pixels do not participate in gradient update. During training, the loss of the main output layer and the loss of at least one auxiliary output layer are calculated simultaneously. The weighted sum of the losses of the main output layer and the auxiliary output layer is used as the total loss, and the model parameters are jointly optimized through backpropagation.

[0018] It should be noted that applying non-negativity constraints to the network output can ensure that the predicted density map is physically reasonable and avoid generating negative density values. By distinguishing between positive and negative sample regions and selectively retaining only difficult negative samples for optimization, the gradient contribution of difficult samples can be avoided by the errors of a large number of simple background pixels, making the model pay more attention to easily confused background regions, thereby improving the ability to distinguish between foreground and background. The joint supervision of the main output layer and the auxiliary output layer helps to alleviate the gradient vanishing problem and accelerate network convergence, further improving the model's counting accuracy and generalization performance.

[0019] A small sample target counting system, comprising: The preprocessing module is configured to extract deep features from the query image and reference samples, and construct a relevance response feature map; The multi-scale local contrast enhancement module is configured to extract background priors through average pooling of multiple sizes, perform difference operations and nonlinear mapping on the background priors and the correlation response feature map, generate gate weights and modulate the correlation response features, and then obtain the initial density map through upsampling decoding. The semantic verification post-processing module is configured to extract candidate points and generate pseudo bounding boxes based on the initial density map, input the query image, candidate points, pseudo bounding boxes and reference sample boxes into the pre-trained segmentation model to obtain a mask; extract the semantic features of the candidates and references and cluster them, and set the interference regions to zero according to the clustering results to obtain the filtered final density map, which is used as the counting result.

[0020] A computer device includes a memory and a processor, the memory storing a computer program, the processor executing the computer program to implement the small sample target counting method.

[0021] A computer-readable storage medium storing a computer program that, when executed by a processor, implements the small sample target counting method.

[0022] The beneficial effects of this invention are as follows: 1. Enhanced local feature representation. This invention enhances local contrast at multiple scales by explicitly modeling the difference between the target and the surrounding background before upsampling regression. This enhances the local response differences of dense small targets while maintaining the structural integrity of large targets, thereby effectively alleviating the problems of target adhesion, boundary blurring, and response fragmentation.

[0023] 2. Improve cross-category interference suppression capability. This invention utilizes an instance-level semantic verification mechanism to transform candidate targets in the initial density map into instance masks, and extracts the semantic features of candidate instances and reference samples for similarity judgment, thereby effectively distinguishing real targets from visually similar interference objects and reducing false matching and overcounting phenomena in complex multi-class scenarios.

[0024] 3. Improved stability of density regression training. This invention introduces a hard negative sample mining strategy (HNM), dynamically selecting high-error background pixels to participate in optimization, avoiding the dilution of hard sample gradients by a large number of simple background regions. Simultaneously, combined with joint supervision from the main output layer and the auxiliary output layer, the network can learn clearer foreground-background discrimination boundaries.

[0025] 4. Improve overall counting accuracy and generalization ability. Experiments show that this invention, through the synergistic effect of multi-scale local contrast enhancement, instance-level semantic verification, and joint loss optimization, achieves the effects of reducing undercounting, false detection, and overcounting on public benchmark datasets (such as FSC-147 and CARPK), and has higher counting accuracy and generalization ability in open categories, dense small targets, and complex background scenes.

[0026] In summary, this invention utilizes deep learning networks and visual fundamental models to construct a multi-scale local feature enhancement and instance-level semantic verification mechanism, achieving adaptive sharpening and cross-class anti-interference matching for dense targets in images. It is particularly suitable for accurate counting of class-agnostic targets in unconstrained and complex scenarios such as highly mixed multi-class targets, drastic scale changes, and severely cluttered backgrounds. It can be widely applied in fields such as smart supermarket shelf inventory, public safety and dense crowd management, industrial warehouse cargo statistics, and traffic network monitoring. Attached Figure Description

[0027] Figure 1 This is a schematic diagram of a small sample target counting method according to Embodiment 2 of the present invention.

[0028] Figure 2 This is a schematic diagram of the multi-scale local contrast enhancement module structure in Embodiment 2 of the present invention. Detailed Implementation

[0029] To provide a clearer understanding of the technical features, objectives, and effects of the present invention, specific embodiments are now described. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention; that is, the described embodiments are only a part of the embodiments of the invention, not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0030] Example 1 This embodiment provides a small sample target counting method. It utilizes deep learning networks and visual basic models to construct a multi-scale local feature enhancement and instance-level semantic verification mechanism, which can achieve adaptive sharpening and cross-class anti-interference matching for dense targets in images. It is suitable for accurate counting of class-agnostic targets in unconstrained complex scenes such as highly mixed multi-class targets, drastic scale changes, and severely cluttered backgrounds. The specific description is as follows.

[0031] First, existing counting models struggle with drastic changes in target scale in complex scenes. This is due to the difficulty of adaptively adjusting a single, fixed receptive field, leading to feature adhesion in densely packed small targets and feature fragmentation in large targets, resulting in missed detections and local overcounting. To overcome the feature perception imbalance caused by a single receptive field and improve the adaptive representation capability of multi-scale targets, this embodiment proposes a multi-scale local contrast enhancement module (MS-LCE). This module innovatively constructs a multi-scale (e.g., 3×3 and 7×7) center-periphery difference mechanism. Through local spatial difference operations, it adaptively sharpens the response peaks of small targets in dense regions while effectively aggregating local features of large targets. This successfully avoids the adhesion of small targets and the feature fragmentation of large targets, significantly improving the localization accuracy of cross-scale objects.

[0032] Secondly, existing matching mechanisms rely solely on low-level visual features for dot product calculations, lacking verification and constraints of high-dimensional instance-level semantic boundaries. This leads to semantic ambiguity when highly similar-looking interfering objects are present, resulting in serious false detections. To address matching ambiguity in complex, unconstrained environments and achieve accurate decoupling between the target and background interference, this embodiment introduces an instance-level semantic verification mechanism (HQ-SAPP) based on a visual foundation model. This mechanism breaks away from the traditional pure low-level matching framework, utilizing dot-box dual cues to generate high-quality candidate masks. Combined with unsupervised feature spectrum clustering, it adaptively filters the initial response through dual gating constraints of purity and inter-cluster similarity. This effectively aggregates genuine reference samples while significantly filtering out false activation noise caused by similar backgrounds, ensuring the purity and robustness of the count.

[0033] Furthermore, existing counting models generally rely on globally indiscriminate mean squared error (MSE) as the loss function, leading to network gradient updates being dominated by a large number of simple zero-valued backgrounds and insufficient penalty for truly confusing difficult backgrounds, resulting in significant noise accumulation. To compensate for the shortcomings of the globally uniform loss in suppressing complex backgrounds and enhance the model's ability to identify difficult interference regions, this embodiment proposes a difficult negative sample mining and joint loss optimization mechanism (HNM). During the model training and optimization phase, this mechanism dynamically captures erroneous background pixels (i.e., difficult negative samples) that generate abnormally high responses in the prediction results and applies a high proportion of dynamic penalty weights to them. This effectively breaks the dominance of simple backgrounds on the network gradient, strongly suppresses background noise caused by messy textures or shadows, and further widens the decision boundary between positive and negative samples.

[0034] Specifically, the small sample target counting method in this embodiment can be implemented in the following way.

[0035] I. Overall Plan Given an RGB query image Given K reference bounding boxes provided by users, this method first uses the basic counting network FSDENet to extract deep features of the query image and the reference samples, and then generates a relevance response feature map through an object prototype extraction decoder. Based on this, this embodiment designs three key mechanisms to address issues such as dense target adhesion, cross-class mismatch, and false background activation in complex scenarios.

[0036] First, a multi-scale local contrast enhancement module (MS-LCE) is introduced. Based on the "center-periphery difference" concept, this module uses 3×3 and 7×7 average pooling to extract local background priors at different scales, and performs difference operations with the original relevant features to enhance the local response differences between dense small targets. Simultaneously, the module modulates the original features through nonlinear mapping and adaptive gating mechanisms, thereby improving the independence and structural integrity of the target responses while maintaining the feature size.

[0037] Secondly, an instance-level semantic verification post-processing mechanism guided by HQ-SAM is constructed. This mechanism extracts candidate peak points from the initial density map G and generates candidate pseudo-boundary boxes by combining the scale of the reference sample. Then, the original image, cue points, pseudo-boundary boxes, and reference boxes are input into HQ-SAM to obtain candidate target instance masks. Subsequently, semantic features of candidate instances and reference samples are extracted through Mask Pooling, the semantic affinity matrix is ​​calculated, and spectral clustering is used to distinguish foreground target clusters from potential interference clusters. When the clustering results meet the gating condition, the regions corresponding to the interference clusters are set to zero in the weight map W, thereby suppressing erroneous responses caused by visual similarity to non-target categories.

[0038] Finally, a Hard Negative Mining (HNM) mechanism is proposed. This mechanism imposes a non-negative constraint on the network output and divides the positive sample region and negative sample background region based on the ground truth density map. All errors are retained for the positive sample region, while only hard negative samples with higher errors are selected for optimization in the negative sample region, thus avoiding the dilution of the hard sample gradient by a large number of simple background pixels. Simultaneously, joint supervision from the main output layer and the auxiliary output layer further enhances the model's ability to distinguish between foreground objects and complex backgrounds.

[0039] In summary, this embodiment establishes a complete few-sample counting framework consisting of relevance feature enhancement, instance-level semantic verification, and hard negative sample optimization. Finally, the enhanced and verified features are fed into an upsampling regression decoder to generate a high-resolution density map, and the target counting result is obtained by integrating the density map.

[0040] II. Multi-scale Local Contrast Enhancement Module (MS-LCE) To improve the local separability of feature responses during density regression, this embodiment designs a multi-scale local contrast enhancement module (MS-LCE). This module draws on the "center-periphery difference" mechanism in biological vision systems to explicitly enhance the local response differences of relevant features before they enter the upsampling decoding stage.

[0041] Suppose the relevance response feature map output by the object prototype extraction decoder in the basic counting network FSDENet is as follows:

[0042] Where C, H, and W represent the number of channels, height, and width of the feature map, respectively. To simultaneously obtain local background information at different scales, MS-LCE employs 3×3 and 7×7 average pooling operations to extract fine-grained and coarse-grained background priors, respectively, calculated as follows:

[0043]

[0044] Subsequently, the module will use the original correlation features Subtracting these subtractions from the aforementioned local background priors yields the differential response features at different scales. To further enhance the expressive power of the differential features with lower parameter overhead, this embodiment introduces a nonlinear mapping function. The difference results are processed independently for each channel, specifically as follows:

[0045]

[0046] in, and These represent nonlinear mapping blocks at the corresponding scales. Each mapping block consists of depthwise convolution, batch normalization, and the ReLU activation function in sequence. It is mainly used to enhance subtle response differences between dense small targets, thereby improving the local sharpening effect; F_{large} is used to provide a wider range of contextual constraints to alleviate the response splitting problem caused by texture changes inside large targets.

[0047] To further achieve adaptive fusion of features at different scales, this module will integrate the original correlation features. Multi-scale difference features The layers are concatenated along the channel dimension, and spatially adaptive gated weights G are generated through 1×1 convolution and sigmoid activation. The calculation method is as follows:

[0048] in, This indicates a concatenation operation along the channel dimension. This represents the weights of a 1×1 convolution. Let G represent the Sigmoid activation function. Finally, the original correlated features are modulated element-wise using the gate weights G to obtain the enhanced feature representation:

[0049] in, This represents the Hadamard product. This process weights and enhances the effective target response without changing the feature map spatial resolution and channel dimension. The enhanced features... The data is then fed into the upsampling decoding module, where spatial resolution is restored step by step and channels are compressed to ultimately generate a single-channel high-quality density map for target counting. Through the aforementioned multi-scale differential modeling and dynamic gating modulation, MS-LCE can effectively improve the independence of responses from dense small targets and enhance the structural integrity of large target regions, thereby alleviating the problems of target adhesion and feature splitting in complex scenes.

[0050] III. Post-processing of instance-level semantic verification guided by HQ-SAM To reduce erroneous responses caused by visually similar distractors in complex and multi-class scenarios, this embodiment further proposes an instance-level semantic verification post-processing mechanism guided by HQ-SAM. This mechanism establishes a high-dimensional semantic association between candidate instances and reference samples, performs instance-level consistency judgment on candidate targets, and thus distinguishes between the true query target and potential distracting targets.

[0051] Specifically, candidate targets are first extracted from the initial density map G output by the regression network. Then, a local maximum extraction algorithm is used to filter targets with confidence scores higher than a threshold from the normalized density map. The two-dimensional peak points are identified, and a set of candidate cue points is constructed:

[0052] To constrain the segmentation range of HQ-SAM in complex backgrounds, this embodiment uses a scaling factor based on the average width and height of the reference sample bounding box. Expand the scope to generate a set of pseudo-boundary boxes corresponding to the candidate targets. Subsequently, the original image I and the candidate cue points were... pseudo-boundary box and real reference sample frame A pre-trained HQ-SAM decoder is used as the common input. Guided by both point and bounding box cues, HQ-SAM generates a high-quality instance mask for each candidate object.

[0053] At the same time, those with confidence levels below the threshold were excluded. Invalid masks are used to ensure the reliability of subsequent semantic verification. After obtaining valid instance masks, this embodiment employs mask pooling to combine the instance mask regions with deep image coding features. Spatial weighted aggregation is performed to obtain the semantic features of candidate instances. The calculation method is as follows:

[0054] in, To prevent extremely small constants with a denominator of zero, this embodiment uses a "cropping-enlarging-segmentation" method to obtain the reference prototype mask for the reference sample, and extracts the corresponding semantic features, thereby constructing candidate feature libraries. and reference feature library .

[0055] Existing few-sample counting methods typically rely on local relevance responses for target matching. When non-target categories with similar textures or local structures to the query target exist in the image, false activations are easily generated. For example, in a scene where strawberries and blueberries appear simultaneously, the blueberry region might be mistakenly identified as the query target due to its similar local appearance. Compared to ordinary global feature pooling, feature aggregation based on instance mask regions provides more explicit target boundary constraints, thereby reducing the interference of background regions and outlier targets on semantic representation. To further measure the semantic relationship between candidate instances and reference samples, this embodiment performs L2 normalization on both the candidate feature library and the reference feature library, and concatenates them into a global feature set. Subsequently, the semantic correlation between features of any two instances is calculated using cosine similarity, and negative values ​​are truncated to obtain the non-negative semantic affinity matrix A:

[0056] Based on the semantic affinity matrix A, this embodiment uses an unsupervised spectral clustering algorithm to partition the instance feature manifold, with the number of clusters set to K=2. Specifically, the degree matrix is ​​first calculated:

[0057] Then construct the symmetric normalized graph Laplacian matrix:

[0058] Where I is the identity matrix. By solving... The feature vectors corresponding to the K smallest eigenvalues ​​can map candidate instances to a low-dimensional space and perform clustering. The cluster labels of the reference prototype features are then used to define the cluster containing the most reference prototypes as the foreground target cluster. The other cluster is defined as the potential interference cluster. .

[0059] To avoid mistakenly deleting real targets during semantic filtering, this embodiment further designs a gating condition composed of cluster purity and inter-cluster similarity. Let the proportion of reference prototypes in the foreground cluster to the total number of reference prototypes be the cluster purity. The cosine similarity between the central features of the foreground cluster and the background cluster is The gated filtering function is then defined as:

[0060] in, Indicates the purity threshold. This represents the inter-cluster similarity threshold. Only when the clustering result simultaneously satisfies the conditions of high purity and low inter-cluster similarity, i.e. Only then is the filtering operation initiated. At this point, it belongs to the potential interference cluster. Candidate targets are considered mismatched targets, and their corresponding regions are set to zero in the weight map W. Finally, the weight map W is multiplied pixel-wise by the initial density map G to obtain the filtered final density map. Through this instance-level semantic verification and spectral clustering filtering mechanism, this embodiment can effectively suppress cross-class mismatches caused by local texture similarity and reduce spurious activation and overcounting problems in complex multi-class scenes.

[0061] IV. Difficult Negative Sample Mining and Joint Loss Optimization Mechanism To improve the training stability and foreground-background discrimination ability of high-resolution density regression models in complex backgrounds, this embodiment proposes a Hard Negative Mining (HNM) strategy and combines it with a joint loss optimization mechanism.

[0062] First, to avoid the network generating physically unreasonable negative density values, this embodiment adds a ReLU nonnegativity constraint to the model output to obtain a nonnegative prediction density map:

[0063] Subsequently, the predicted density map Y and the ground truth density map Y_{gt} are flattened into one-dimensional vectors:

[0064] And calculate the pixel-by-pixel squared error matrix: .

[0065] Example 2 This embodiment is based on embodiment 1: like Figure 1 As shown, this embodiment provides a few-sample target counting method that, through the synergistic effect of three core components, specifically alleviates the perceptual limitations of the baseline model in extreme scenarios. First, this embodiment introduces a Hard Negative Mining (HNM) strategy, dynamically capturing and increasing the penalty weight for difficult backgrounds during the training phase, effectively suppressing false noise caused by complex textures in images like envelopes. Second, at the feature level, a Multi-Scale Local Contrast Enhancement (MS-LCE) module is designed, using a local difference mechanism to achieve adaptive sharpening of target response peaks and global structure aggregation, improving large-scale feature splitting and feature adhesion in dense regions, similar to watch straps. Finally, a high-quality segmentation model, HQ-SAM, is introduced to construct an instance-level semantic verification mechanism (High-QualitySAM-Guided Post-Processing, HQ-SAPP). This mechanism performs instance-level filtering on the initial density response through high-dimensional semantic consistency analysis between candidate instance masks and reference samples, effectively suppressing false matches and false activations caused by visually similar but semantically inconsistent targets.

[0066] like Figure 2 As shown, the Multi-Scale Local Contrast Enhancement (MS-LCE) module is used to enhance the local difference between the target and the surrounding background before the relevant features are incorporated into the upsampling regression. This module includes a local background prior extraction unit, a multi-scale difference unit, a nonlinear mapping unit, a dynamic gating unit, and a feature modulation unit.

[0067] Specifically, this module uses 3×3 and 7×7 average pooling for the correlation features, respectively. Processing is performed to obtain small-scale background priors. and large-scale background prior Subsequently, respectively with , Subtraction yields multi-scale difference features, which are then enhanced using a DWConv-BN-ReLU structure. Small-scale branches improve the local separability of dense small targets, while large-scale branches maintain the structural integrity of large target regions. Further, this module concatenates the original correlation features with the multi-scale difference features, generating a gating weight G through 1×1 convolution and sigmoid activation. G is then used to modulate Xcorr element-wise, resulting in the enhanced feature Xenhanced. The essential structure of this module includes multi-scale background prior extraction, difference modeling, and dynamic gating modulation; the pooling kernel size, convolution form, and activation function can be replaced or adjusted according to task requirements. This module effectively alleviates problems such as dense target adhesion, boundary blurring, and response splitting.

[0068] In this embodiment, the instance-level semantic verification post-processing module guided by HQ-SAM is used to filter erroneous responses in the initial density map and reduce false matches caused by visually similar interfering objects. This module includes a candidate point extraction unit, a pseudo-boundary box generation unit, an instance segmentation unit, a semantic feature extraction unit, a semantic clustering unit, and a weight map generation unit.

[0069] Specifically, firstly, local maxima are extracted from the initial density map G to form a candidate cue point set PC; then, candidate pseudo-boundary boxes BC are generated based on the average size of the reference sample boxes. Subsequently, the original image I, candidate points PC, pseudo-boundary boxes BC, and the ground truth reference box BE are input into HQ-SAM to generate candidate instance masks Mi, and low-confidence masks are removed. After obtaining effective masks, semantic features between candidate instances and reference samples are extracted using Mask Pooling, and cosine similarity is calculated to construct a semantic affinity matrix A. Then, spectral clustering is used to divide candidate instances into foreground target clusters and potential interference clusters, and filtering is enabled based on cluster purity and inter-cluster similarity. When the filtering conditions are met, the regions corresponding to the interference clusters are set to zero in the weight map W; otherwise, the original response is retained. The necessary structure of this module includes candidate point extraction, instance mask generation, semantic feature extraction, and weight map filtering; HQ-SAM can be replaced with SAM, SAM2, or other cue-based segmentation models, and spectral clustering can be replaced with K-means, hierarchical clustering, or thresholding. This module can improve the ability to judge the instance-level semantic consistency of candidate targets and reduce false detections and overcounting in complex multi-class scenarios.

[0070] In this embodiment, a difficult negative sample mining and joint loss optimization mechanism is used to enhance the model's learning ability for complex backgrounds and easily confused regions. This mechanism includes a non-negativity constraint unit, an error calculation unit, a positive / negative sample partitioning unit, a difficult negative sample selection unit, and a joint loss calculation unit.

[0071] Specifically, firstly, the original predicted density map... ReLU constraints are applied to obtain a non-negative density map Y, avoiding negative density predictions. Subsequently, Y is flattened along with the ground truth density map Ygt, and the pixel-wise squared error E is calculated. The pixel values ​​are used to divide the target area into positive sample areas and the background area into negative sample areas.

[0072] For positive sample regions, all errors are used in optimization; for negative sample regions, the optimization is based on the number of positive samples. , scaling factor α and minimum threshold Dynamically determine the number of difficult negative samples And select the one with the largest error Each background pixel participates in training. Finally, the HNM loss and MSE loss are weighted and combined, and can be applied to both the main output layer and the auxiliary output layer simultaneously. The necessary structure of this mechanism includes nonnegativity constraints, positive and negative sample partitioning, hard negative sample selection, and HNM loss calculation; the loss weights, number of auxiliary layers, and sample selection method can be adjusted according to the task. This mechanism avoids the gradient of hard background samples being diluted by a large number of simple background pixels, allowing the model to learn a clearer foreground-background discrimination boundary, improving training stability and counting accuracy.

[0073] In addition, to verify this method, an experiment was conducted in this embodiment, which is described in detail below.

[0074] (1) Dataset selection The FSC-147 dataset is a widely used and representative benchmark dataset for few-shot object counting tasks, possessing high authority and challenge. This dataset covers 147 different semantic categories, and the image content includes both common everyday objects and complex, densely distributed scenes, comprehensively reflecting the target counting needs in open environments. FSC-147 contains 6135 RGB images, each labeled with three randomly selected bounding boxes used as reference samples for queries. This embodiment strictly follows the official dataset partitioning method during experiments: 3659 images for training, 1286 images for validation, and 1190 images for testing. Notably, the test set includes unseen categories that do not overlap with the training set, thus effectively testing the model's generalization counting ability for new categories in open-world scenarios.

[0075] The CARPK dataset primarily targets dense vehicle detection and counting tasks, consisting of aerial images captured by drones in various parking lot scenarios. The dataset contains 1448 images, with 989 in the training set and 459 in the test set, and nearly 90,000 labeled vehicle instances. Although CARPK only covers a single category of vehicles, the images feature densely distributed, small-scale targets with significant occlusion and background interference, making counting challenging. This embodiment utilizes this dataset to evaluate the proposed method's feature representation ability, target discrimination ability, and accurate counting performance for densely packed small targets in complex aerial scenes.

[0076] (2) Evaluation indicators This embodiment uses Mean Absolute Error (MAE) and Root Mean Squared Error (RMSE) as metrics for evaluating counting accuracy. MAE reflects the general level of error between predicted and actual counts, while RMSE measures the degree of error fluctuation and the model's robustness to extreme samples. Lower values ​​indicate better model performance.

[0077] (3) Implementation details All experiments were conducted on an NVIDIA A6000 GPU based on the PyTorch framework.

[0078] Regarding evaluation benchmarks, this embodiment conducted experimental validation on the small sample counting public dataset FSC 147 and the drone-perspective parking lot dataset CARPK, continuing to use mean absolute error (MAE) and root mean square error (RMSE) as quantification metrics. For the basic hyperparameter settings of network training, ISANet employs a hierarchical learning rate optimization strategy to ensure the stability of the pre-trained backbone features. Specifically, the learning rate of the backbone network (e.g., ResNet-50) is set to... The learning rate for newly introduced structures (such as the MS-LCE module and density regression head) is set to... The models were all trained end-to-end using the AdamW optimizer, with the weight decay coefficient set to [value missing]. The total training period is 200 epochs, and it is dynamically adjusted using the StepLR strategy (the learning rate decays to 0.25 times its original value after a certain number of epochs).

[0079] Experiments demonstrate that this invention, through the synergistic effect of MS-LCE feature enhancement, HQ-SAM instance-level semantic verification, and HNM joint loss optimization, achieves improvements in reducing undercounting, false detection, and overcounting on public benchmark datasets (such as FSC-147 and CARPK). It also exhibits higher counting accuracy and generalization ability in open categories, dense small targets, and complex background scenarios.

[0080] In summary, the small-sample target counting method of the present invention has the following characteristics: 1. Multi-stage Few-Sample Counting Enhancement Framework: This invention proposes a multi-stage enhancement method for few-sample target counting in complex scenes. It combines multi-scale local contrast enhancement, instance-level semantic verification, and hard negative sample optimization to improve density regression counting performance from three levels: feature representation, post-processing selection, and training constraints. This framework effectively addresses issues such as dense target clustering, mismatched similar categories, and spurious responses in complex backgrounds.

[0081] 2. Multi-scale Local Contrast Enhancement Module (MS-LCE): Before upsampling and decoding the relevance features, this invention introduces a multi-scale local contrast enhancement module based on "center-periphery difference". This module uses 3×3 and 7×7 average pooling to extract local background priors at different scales, and modulates the original relevance features through difference mapping and adaptive gating, thereby enhancing the local separability of dense small targets while maintaining the structural integrity of large targets.

[0082] 3. HQ-SAM-guided instance-level semantic verification mechanism: This invention extracts candidate peak points based on the initial density map and generates candidate pseudo-boundary boxes by combining the scale of the reference sample. Then, HQ-SAM is used to generate candidate instance masks. Subsequently, semantic features of candidate instances and reference samples are extracted through mask pooling, and a semantic affinity matrix is ​​constructed to achieve instance-level semantic consistency verification between candidate targets and reference targets, thereby effectively eliminating visually similar but semantically inconsistent interfering targets.

[0083] 4. Hard Negative Sample Mining and Joint Loss Optimization Mechanism (HNM): This invention addresses the problem of excessive background pixel proportion and easy dilution of hard negative sample gradients in high-resolution density regression. It dynamically selects negative samples with higher errors to participate in optimization, and these samples, together with the MSE loss, constitute the joint loss. Simultaneously, this optimization mechanism applies to both the main output layer and the auxiliary output layer, enabling the model to learn clearer foreground-background discrimination boundaries, thus improving training stability and counting accuracy.

[0084] 5. This invention is applicable to open category counting tasks with dense targets, small scale, severe occlusion, complex background, and similar interference objects. It can be applied to scenarios such as intelligent transportation, intelligent monitoring, agricultural yield estimation, public safety, and industrial inspection, and has strong generalization ability and practical application value.

[0085] Example 3 This embodiment is based on embodiment 1: This embodiment provides a small sample target counting system, including: The preprocessing module is configured to extract deep features from the query image and reference samples, and construct a relevance response feature map; The multi-scale local contrast enhancement module is configured to extract background priors through average pooling of multiple sizes, perform difference operations and nonlinear mapping on the background priors and the correlation response feature map, generate gate weights and modulate the correlation response features, and then obtain the initial density map through upsampling decoding. The semantic verification post-processing module is configured to extract candidate points and generate pseudo bounding boxes based on the initial density map, input the query image, candidate points, pseudo bounding boxes and reference sample boxes into the pre-trained segmentation model to obtain a mask; extract the semantic features of the candidates and references and cluster them, and set the interference regions to zero according to the clustering results to obtain the filtered final density map, which is used as the counting result.

[0086] Example 4 This embodiment is based on embodiment 1: This embodiment provides a computer device, including a memory and a processor. The memory stores a computer program, and the processor executes the computer program to implement a small sample target counting method according to Embodiment 1. The computer program can be in the form of source code, object code, executable file, or some intermediate form.

[0087] Example 5 This embodiment is based on embodiment 1: This embodiment provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements a small sample target counting method according to Embodiment 1. The computer program can be in the form of source code, object code, executable file, or some intermediate form. The storage medium includes any entity or device capable of carrying computer program code, a recording medium, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electrical carrier signal, a telecommunication signal, and a software distribution medium, etc.

[0088] The above description is merely a preferred embodiment of the present invention. It should be understood that the present invention is not limited to the forms disclosed herein and should not be construed as excluding other embodiments. It can be used in various other combinations, modifications, and environments, and can be altered within the scope of the concept described herein through the above teachings or related technologies or knowledge. Modifications and variations made by those skilled in the art that do not depart from the spirit and scope of the present invention should be within the protection scope of the appended claims.

[0089] It should be noted that, for the sake of simplicity, the foregoing method embodiments are described as a series of actions. However, those skilled in the art should understand that this application is not limited to the described order of actions, as some steps may be performed in other orders or simultaneously according to this application. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions and modules involved are not necessarily essential to this application.

Claims

1. A method for counting targets in a small sample, characterized in that, include: Extract deep features from the query image and the reference sample, and construct a relevance response feature map; Background priors are extracted by average pooling of various sizes. Differential operations and nonlinear mapping are performed on the background priors and the correlation response feature map to generate gated weights and modulate the correlation response features. Then, the initial density map is obtained by upsampling and decoding. Candidate points are extracted from the initial density map and pseudo bounding boxes are generated. The query image, candidate points, pseudo bounding boxes and reference sample boxes are input into the pre-trained segmentation model to obtain the mask. Semantic features of candidates and references are extracted and clustered. Interference regions are set to zero according to the clustering results to obtain the final filtered density map, which is used as the counting result.

2. The small sample target counting method according to claim 1, characterized in that, The extraction of deep features from the query image and reference samples, and the construction of a relevance response feature map, includes: The query image and the reference sample box are input into the basic counting network, and the encoder of the basic counting network extracts the deep features of the query image and the deep features of the reference sample, respectively. The deep features of the query image and the deep features of the reference samples are input into the object prototype extraction decoder. The object prototype extraction decoder generates the correlation response feature map by calculating the similarity between the features of the query image and the prototype features of each reference sample.

3. The small sample target counting method according to claim 1, characterized in that, The process of extracting background priors through multi-size average pooling, and performing difference operations and nonlinear mappings on the background priors and the relevant response feature maps, includes: The correlation response feature map is processed using a first-size average pooling method to obtain a fine-grained background prior; the correlation response feature map is then processed using a second-size average pooling method to obtain a coarse-grained background prior, wherein the first size is smaller than the second size. The fine-grained background prior and the coarse-grained background prior are subtracted from the correlation response feature map, respectively, to obtain the fine-grained differential response features and the coarse-grained differential response features; Channel-independent nonlinear mappings are performed on the fine-grained differential response features and the coarse-grained differential response features, respectively. The nonlinear mappings include depthwise convolution, batch normalization, and activation functions.

4. The small sample target counting method according to claim 3, characterized in that, The process of generating gated weights and modulating correlation response features, followed by upsampling and decoding to obtain the initial density map, includes: The correlation response feature map, the fine-grained differential response feature after nonlinear mapping, and the coarse-grained differential response feature after nonlinear mapping are concatenated along the channel dimension. The concatenated features are then convolved and spatial adaptive gating weights are generated through an activation function. The gating weights are then multiplied element-wise with the correlation response feature map to obtain the enhanced correlation response feature. The enhanced correlation response features are upsampled and decoded, and the spatial resolution is restored and the number of channels is compressed by upsampling step by step to obtain the initial density map of a single channel.

5. The small sample target counting method according to claim 1, characterized in that, The step of extracting candidate points and generating pseudo-boundary boxes based on the initial density map, and inputting the query image, candidate points, pseudo-boundary boxes, and reference sample boxes into a pre-trained segmentation model to obtain a mask includes: Local maxima extraction is performed on the initial density map, and peak points with confidence scores higher than preset conditions are selected as candidate cue points; pseudo bounding boxes are generated by proportionally expanding the bounding boxes of the reference sample bounding boxes according to the average size of the reference sample bounding boxes. The original query image, all candidate cue points, all pseudo bounding boxes, and reference sample boxes are input into the pre-trained segmentation model. Guided by both cue points and bounding box cue points, the segmentation model outputs the instance mask corresponding to each candidate target.

6. The small sample target counting method according to claim 5, characterized in that, The process of extracting semantic features from candidates and references and clustering them, then setting the interference regions to zero according to the clustering results to obtain the filtered final density map, includes: After removing invalid masks with confidence levels below the preset conditions, the masks of each instance are spatially weighted and aggregated with the deep coding features of the query image to obtain the semantic features of each candidate instance; the image regions within each reference sample box are cropped and enlarged and then input into the segmentation model to obtain the reference prototype mask; the semantic features of the reference sample are extracted based on the reference prototype mask and the deep coding features of the query image. After normalizing the semantic features of all candidate instances and the semantic features of the reference samples, they are concatenated into a global feature set. The semantic similarity between any two features is calculated and negative values ​​are truncated to obtain a non-negative semantic affinity matrix. Based on the semantic affinity matrix, a normalized graph Laplacian matrix is ​​constructed, and the feature vectors are solved to divide the candidate instances into foreground and background clusters. The distribution of semantic features of each reference sample in the foreground and background clusters is counted. The cluster containing the most reference samples is identified as the foreground target cluster, and the other cluster is identified as the potential interference cluster. The filtering gate condition is determined based on the proportion of reference samples in the foreground cluster and the inter-cluster similarity between the foreground and background clusters. If the condition is met, the region corresponding to the candidate instance belonging to the potential interference cluster is set to zero in the weight map. The weight map is multiplied pixel by pixel with the initial density map to obtain the final filtered density map.

7. The small sample target counting method according to claim 1, characterized in that, The model training phase also includes hard negative sample mining and joint loss optimization: A nonnegative constraint is applied to the output final density map to obtain a nonnegative prediction density map. The nonnegative prediction density map is compared pixel by pixel with the pre-constructed ground truth density map to calculate the pixel-by-pixel error. Based on the value of each pixel in the ground truth density map, positive sample regions and negative sample regions are divided. All pixel errors in the positive sample regions are retained to participate in gradient update. For the negative sample regions, pixels with errors higher than a preset quantile are selected as difficult negative samples to participate in gradient update, and the remaining negative sample pixels do not participate in gradient update. During training, the loss of the main output layer and the loss of at least one auxiliary output layer are calculated simultaneously. The weighted sum of the losses of the main output layer and the auxiliary output layer is used as the total loss, and the model parameters are jointly optimized through backpropagation.

8. A small sample target counting system, characterized in that, include: The preprocessing module is configured to extract deep features from the query image and reference samples, and construct a relevance response feature map; The multi-scale local contrast enhancement module is configured to extract background priors through average pooling of multiple sizes, perform difference operations and nonlinear mapping on the background priors and the correlation response feature map, generate gate weights and modulate the correlation response features, and then obtain the initial density map through upsampling decoding. The semantic verification post-processing module is configured to extract candidate points and generate pseudo bounding boxes based on the initial density map, and input the query image, candidate points, pseudo bounding boxes and reference sample boxes into the pre-trained segmentation model to obtain a mask; Semantic features of candidates and references are extracted and clustered. Interference regions are set to zero according to the clustering results to obtain the final filtered density map, which is used as the counting result.

9. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements a small sample target counting method according to any one of claims 1-7.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements a small sample target counting method according to any one of claims 1-7.

Images