Zero sample target counting method, system and device and storage medium

Abstract

The invention relates to the technical field of computer vision, and relates to a zero sample target counting method, system and device and a storage medium. The zero sample target counting method comprises the following steps: acquiring a text cue word of a target to be recognized and an image to be recognized; extracting text cue word features to obtain text features; extracting features of the to-be-recognized image to obtain visual features; dynamically distributing feature channels of the visual features to a plurality of semantic groups through an implicit attribute channel grouping mechanism to obtain grouping features; processing the grouping features by using a cross-group attention mechanism to obtain enhanced grouping features; fusing the text features and the enhanced grouping features to obtain fusion features, and generating a target density prediction map according to the fusion features; and calculating a target number according to the target density prediction map to obtain a target counting result. The target counting precision can be improved.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and in particular to a zero-sample target counting method, system, device, and storage medium. Background Technology

[0002] Zero-shot object counting is a cutting-edge task in computer vision, aiming to enumerate instances of unseen object categories in an image using only natural language cues. Existing methods are mostly based on visual language models, but they suffer from significant shortcomings in handling the channel dimension: in existing computer vision models, feature channels are often treated as independent entities or simple sets of weights, ignoring the potential inherent semantic relationships between them. This approach overlooks a crucial fact: different channels within the network may actually correspond to different visual attributes, such as color, texture, shape, or context. When these correlations are ignored, the model's representational power and interpretability are severely limited.

[0003] Traditional channel processing methods mainly focus on three paradigms, each with its significant limitations. Channel compression techniques (such as 1x1 convolutions), while reducing the number of parameters, inevitably lose important information, performing particularly poorly in tasks requiring precise localization and counting. While the channel attention mechanisms in SENet (Squeeze-and-Excitation Network, a convolutional neural network architecture based on channel attention) and CBAM (Convolutional Block Attention Module, a hybrid attention mechanism) assign different weights to channels, they only focus on relative importance rather than intrinsic relationships, making it difficult to identify and utilize channel groups that jointly represent specific visual attributes. Methods like ShuffleNet (a lightweight convolutional neural network architecture) employ random or fixed channel mixing strategies, lacking an understanding of channel semantics and failing to ensure effective collaboration between semantically related channels. Summary of the Invention

[0004] This application aims to at least solve the technical problems existing in the prior art and provide a zero-sample target counting method, system, device and storage medium.

[0005] In a first aspect, the present invention provides a zero-sample target counting method, comprising: Obtain the text prompts and the image of the target to be identified; Text features are obtained by extracting features from text prompt words; Visual features are obtained by extracting features from the image to be recognized; The implicit attribute channel grouping mechanism dynamically allocates the feature channels of visual features to multiple semantic groups to obtain grouped features; By utilizing a cross-group attention mechanism to process group features, enhanced group features are obtained. By fusing text features and enhanced grouping features, a fused feature is obtained, and a target density prediction map is generated based on the fused feature. The number of targets is calculated based on the target density prediction map, and the target count result is obtained.

[0006] Optionally, the implicit attribute channel grouping mechanism dynamically allocates the feature channels of visual features to multiple semantic groups to obtain grouped features, including: Visual features are transformed to enhance their representational power, resulting in transformed visual features. Define a set of learnable prototype vectors, where each prototype vector represents the center of a semantic group, and calculate the similarity between the transformed visual features and the prototype vectors corresponding to each semantic group. The soft assignment probability from channel to semantic group is determined based on the similarity between the transformed visual features and the prototype vectors corresponding to each semantic group. Based on the soft assignment probability, the features corresponding to each spatial location of the visual features are softly assigned to various semantic groups to obtain grouped features.

[0007] Optionally, the steps for performing feature transformation on visual features include: The visual features are sequentially subjected to linear transformation, layer normalization, and activation processing to obtain the transformed visual features. Visual features are represented as ,in, Represents a matrix, Indicates batch size. Represents the number of spatial locations. Represents the number of channels; the transformed visual features are denoted as , ; in, Represents a linear transformation. Representation layer normalization, This represents the activation function.

[0008] Optionally, the formula for calculating the similarity between the transformed visual features and the prototype vectors corresponding to each semantic group is: ; in, This represents the similarity between the features corresponding to each spatial location of the transformed visual features and the prototype vector; This represents the feature vector corresponding to the transformed visual feature at the b-th sample, the n-th position, and the c-th channel. represents the sample batch index, n represents the spatial location index, and c represents the channel index; Represents the prototype vector. The semantic group index representing the prototype vector.

[0009] Optionally, by means of temperature parameters The softmax function is used to calculate the soft assignment probability from channel to semantic group; the formula for calculating the soft assignment probability is: ; Representation of features The soft assignment probability, Indicates temperature parameter, A semantic group index representing visual features.

[0010] Optionally, the grouping feature is represented as , For each dimension of a semantic group, the method further includes: Calculate the centroid vector for each semantic group. The expression for calculating the centroid vector is as follows: ; Indicates sample The Middle The centroid vector of each semantic group This represents the feature vector corresponding to the visual feature at the b-th sample, the n-th position, and the c-th channel; n represents the spatial location index. The intra-group consistency loss is defined as the average Euclidean distance between the visual features and the centroids of the semantic group. This intra-group consistency loss is used to optimize the semantic consistency of visual features within the same semantic group. The expression for the intra-group consistency loss is: .

[0011] Optionally, the step of processing group features using a cross-group attention mechanism to obtain enhanced group features includes: Perform layer normalization on the grouped features to obtain the normalized grouped features; The intra-group attention weights are calculated using a learnable intra-group attention mask and the query vector, key vector, and value vector of the normalized group features, and the intra-group attention features are determined based on the intra-group attention weights. The inter-group attention weights are calculated using a learnable inter-group attention mask and the query vector, key vector, and value vector of the normalized grouping features, and the inter-group attention features are determined based on the inter-group attention weights. By fusing intra-group and inter-group attention features through a gate control mechanism, a fused attention feature is obtained. Enhanced grouping features are generated based on the fusion of attention features and grouping features.

[0012] In a second aspect, the present invention provides a zero-sample target counting system, the system comprising: The acquisition module is used to acquire the text prompts and the image to be recognized for the target. The first feature extraction module is used to extract features from text prompt words to obtain text features; The second feature extraction module is used to extract features from the image to be recognized, thereby obtaining visual features; The implicit attribute channel grouping module is used to dynamically allocate the feature channels of visual features to multiple semantic groups through the implicit attribute channel grouping mechanism to obtain grouped features; The cross-group attention processing module is used to process group features using the cross-group attention mechanism to obtain enhanced group features; The fusion module is used to fuse text features and enhanced grouping features to obtain fused features, and generate a target density prediction map based on the fused features; The output module is used to calculate the number of targets based on the target density prediction map and obtain the target count result.

[0013] Thirdly, the present invention provides an electronic device, the electronic device comprising: At least one processor; and, A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the zero-sample target counting method described above.

[0014] Fourthly, the present invention also provides a computer-readable storage medium storing at least one computer program, which is executed by a processor in an electronic device to implement the zero-sample target counting method described above.

[0015] In summary, this application includes the following beneficial technical effects: This application proposes an Implicit Attribute Channel Grouping (IACG) mechanism, which dynamically assigns feature channels to different semantic groups without requiring predefined visual attributes, achieving adaptive semantic grouping of feature channels. Furthermore, a Cross-Group Attention Mechanism (CGAM) is designed, which utilizes a hierarchical attention architecture and a dual-path design to establish efficient information exchange pathways between channel groups. Simultaneously, a gating mechanism adaptively integrates intra-group and inter-group information, enhancing collaborative interaction between different attribute channels and improving counting accuracy. Our framework does not rely on predefined visual attributes but automatically discovers potential semantic relationships between feature channels through end-to-end learning. This design enables it to exhibit excellent adaptability across diverse visual tasks and data distributions. Attached Figure Description

[0016] Figure 1 This is a flowchart illustrating a zero-sample target counting method provided in an embodiment of the present invention; Figure 2 This is a schematic diagram of an image to be identified according to an embodiment of the present invention; Figure 3 This is a system architecture diagram of a zero-sample target counting method provided in an embodiment of the present invention; Figure 4 A flowchart illustrating the implicit attribute channel grouping mechanism provided in an embodiment of the present invention; Figure 5 This is an overall architecture diagram of a cross-group attention mechanism provided in an embodiment of the present invention; Figure 6 This is a schematic diagram of target counting and identification results provided in an embodiment of the present invention; Figure 7 This is a schematic table illustrating the experimental results of a zero-sample target counting method provided in an embodiment of the present invention on the FSC147 dataset; Figure 8 This is a schematic table illustrating the experimental results of a zero-sample target counting method provided in an embodiment of the present invention on the CARPK dataset; Figure 9 A schematic table illustrating the experimental results of the zero-sample target counting method provided in an embodiment of the present invention on the ShanghaiTech population counting dataset; Figure 10 This is a schematic diagram of the structure of an electronic device for implementing the zero-sample target counting method according to an embodiment of the present invention.

[0017] Reference numerals: 10, processor; 11, memory; 12, communication bus; 13, communication interface.

[0018] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0019] Embodiments of the present invention are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention.

[0020] In the description of this invention, it should be understood that the terms "longitudinal", "lateral", "up", "down", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this invention.

[0021] In the description of this invention, unless otherwise specified and limited, it should be noted that the terms "installation", "connection" and "linking" should be interpreted broadly. For example, they can refer to mechanical or electrical connections, or internal connections between two components. They can be direct connections or indirect connections through an intermediate medium. Those skilled in the art can understand the specific meaning of the above terms according to the specific circumstances.

[0022] Reference Figure 1 The diagram shown is a flowchart illustrating a zero-sample target counting method according to an embodiment of the present invention. In this embodiment, the zero-sample target counting method includes: S1. Obtain the text prompts and the image of the target to be identified.

[0023] The target to be identified can be animals, plants, people, vehicles, industrial products, daily necessities, etc., and the target to be identified can be determined according to different application scenarios. This application does not impose any restrictions. The image to be identified is an image containing the target to be identified.

[0024] S2. Extract text features from text prompt words to obtain text features.

[0025] Text features of text prompt words are extracted using a text encoder. The text encoder can be a distributed representation-based encoder such as GloVe (Global Vectors for Word Representation); or a neural network-based encoder such as Word2Vec and BERT. Word2Vec generates word vectors by predicting context or target words, while BERT uses a bidirectional Transformer architecture, which can better understand contextual dependencies. The appropriate text encoder can be selected according to the complexity of the text prompt words, and this application does not impose any restrictions.

[0026] S3. Extract features from the image to be recognized to obtain visual features.

[0027] The image to be recognized is input into the visual encoder to obtain visual features; in this embodiment, the visual encoder is an image encoder based on the Transformer architecture, for example, such as... Figure 2 As shown, the text prompt is "strawberries," and the image to be identified is a strawberry image.

[0028] S4. The feature channels of visual features are dynamically allocated to multiple semantic groups through the implicit attribute channel grouping mechanism to obtain grouped features.

[0029] Traditional feature processing methods typically treat all channels as homogeneous entities or simply apply global attention mechanisms, ignoring the potential inherent semantic relationships between channels. Meanwhile, existing channel grouping methods (such as grouped convolution) are primarily designed for computational efficiency, employing fixed and uniform grouping strategies, lacking semantic understanding and adaptability. Figure 4 As shown, we propose an Implicit Attribute Channel Grouping (IACG) mechanism. This mechanism eliminates the need for manually predefining visual attributes for each group; instead, it automatically discovers the latent semantic structure within feature channels and dynamically assigns related channels to the same group. This data-driven approach not only enhances the model's flexibility but also adaptively captures key features across different datasets and tasks. Its core concept is to utilize learnable prototype vectors to represent different feature groups and achieve dynamic grouping by calculating the similarity between features and prototypes.

[0030] In this embodiment, visual features are represented as ,in, Represents a matrix, Indicates batch size. Represents the number of spatial locations. Indicates the number of channels; Reference Figure 3 and Figure 4 Specifically, through an implicit attribute channel grouping mechanism, the feature channels of visual features are dynamically assigned to multiple semantic groups, resulting in grouped features including: S41. Perform feature transformation on the visual features to enhance their representational ability and obtain the transformed visual features.

[0031] The steps for performing feature transformation on visual features include: The visual features are sequentially subjected to linear transformation, layer normalization, and activation processing to obtain the transformed visual features. The representational ability is enhanced through feature transformation.

[0032] Visual features are represented as ,in, Represents a matrix, Indicates batch size. Represents the number of spatial locations. Represents the number of channels; the transformed visual features are denoted as , ; in, Represents a linear transformation. Representation layer normalization, This represents the activation function.

[0033] S42. Define a set of learnable prototype vectors, where each prototype vector represents the center of a semantic group, and calculate the similarity between the transformed visual features and the prototype vectors corresponding to each semantic group.

[0034] Specifically, the formula for calculating the similarity between the transformed visual features and the prototype vectors corresponding to each semantic group is as follows: ; in, This represents the similarity between the features corresponding to each spatial location of the transformed visual features and the prototype vector; This represents the feature vector corresponding to the transformed visual feature at the b-th sample, the n-th position, and the c-th channel. represents the sample batch index, n represents the spatial location index, and c represents the channel index; Represents the prototype vector. This represents the semantic group index of the prototype vector. Here, we use negative Euclidean distance as the similarity metric.

[0035] S43. Determine the soft assignment probability of channels to semantic groups based on the similarity between the transformed visual features and the prototype vectors corresponding to each semantic group.

[0036] By having temperature parameters The softmax function is used to calculate the soft assignment probability from channel to semantic group; the formula for calculating the soft assignment probability is: ; Representation of features The soft assignment probability, Indicates temperature parameter, Semantic group index representing visual features. Temperature parameter. Control the "softness" of the allocation: a larger amount Values ​​will produce a softer allocation, while smaller values ​​will result in a softer allocation. The value will then generate a result that is closer to hard assignments.

[0037] S44. Based on the soft assignment probability, the features corresponding to each spatial location of the visual features are softly assigned to each semantic group to obtain grouped features.

[0038] Based on the soft-assignment probability, we soft-assign visual features to various semantic groups, and the calculation formula is as follows: ; in, It is the number of channels in each group. , The number of semantic groups; Representing the The sample, the first The position and the first The visual feature vectors corresponding to each semantic group. This soft assignment mechanism has several advantages: (1) it allows channels to belong to multiple groups to different degrees, thereby capturing the multifaceted characteristics of the channels; (2) the soft assignment probability is differentiable, supporting end-to-end training; (3) the "softness" of the assignment can be flexibly controlled by adjusting the temperature parameter.

[0039] In a preferred embodiment of this application, to adapt to input features of different dimensions, we designed a dynamic group size adjustment mechanism to ensure that the product of the number of groups and the group size equals the total number of channels. When input dimension When changes occur, we automatically adjust using the following strategies: ; That is, if Cannot be Divisibility, we will base it on Divide by The remainder is used to adjust the grouping, ensuring dimensionality compatibility and computational efficiency. This dynamic adjustment mechanism significantly enhances the model's adaptability and robustness in different scenarios, eliminating the need to manually configure parameters for different input feature dimensions.

[0040] To optimize the semantic coherence and discriminative capacity of channel grouping, we propose a novel Intra-Group Consistency Loss (IGCL) mechanism. This specialized loss function aims to encourage feature channels within the same semantic group to exhibit similar representations, thereby forming compact clusters with clear semantic boundaries in the feature space.

[0041] Unlike traditional regularization techniques that indiscriminately apply to the entire feature space, IGCL specifically targets the internal geometric structure of each channel group, guiding the model towards a more semantically meaningful feature organization. This approach significantly enhances the interpretability of the learned representations and the overall discriminative power of the model.

[0042] The final grouping features are represented as follows: , The dimension of each semantic group; in a preferred embodiment of this example, the method further includes calculating the centroid vector of each semantic group.

[0043] The expression for calculating the centroid vector is: ; Indicates sample The Middle The centroid vector of each semantic group This represents the feature vector corresponding to the visual feature at the b-th sample, the n-th position, and the c-th channel; n represents the spatial index; this centroid vector... In a high-dimensional feature space, it acts as a semantic anchor point, representing the sample. middle The average characteristic of all features in the group.

[0044] Then, we define IGCL as the normalized average Euclidean distance between each feature vector and its corresponding group centroid: We utilize intra-group consistency loss to optimize the semantic consistency of visual features within the same semantic group. The expression for the intra-group consistency loss is: .

[0045] By minimizing this loss function, the model is incentivized to generate compact and semantically coherent feature sets. Each channel within a set tends to converge toward encoding similar visual attributes, while the absence of explicit inter-group repulsion terms allows for natural semantic overlaps between different attribute sets—a crucial consideration because visual attributes are rarely completely independent in real-world scenarios.

[0046] Intra-group consistency loss can significantly improve the quality of feature grouping, thereby leading to more discriminative representations and improved performance on downstream tasks. This loss function enhances intra-group semantic cohesion, improves model interpretability through clearer group semantics, enhances feature representation quality, and benefits the entire zero-shot target counting framework.

[0047] S5. Use cross-group attention mechanism to process group features to obtain enhanced group features.

[0048] Reference Figure 5 Specifically, cross-group attention mechanisms are used to process group features, resulting in enhanced group features, including: S51. Perform layer normalization on the grouped features to obtain the normalized grouped features.

[0049] The expression for layer normalization of grouped features is: ; The grouped features after the representation layer normalization process, Presentation layer normalization operation; S52. Calculate the intra-group attention weights using the learnable intra-group attention mask and the query vector, key vector, and value vector of the normalized grouping features, and determine the intra-group attention features based on the intra-group attention weights.

[0050] S53. Calculate the inter-group attention weights using the learnable inter-group attention mask and the query vector, key vector, and value vector of the normalized grouping features, and determine the inter-group attention features based on the inter-group attention weights.

[0051] Basic attention calculations follow this form: ; The query vector key vector key vector It is through a learnable linear projection matrix. The obtained query, key, and value matrix.

[0052] To capture both intra-group and inter-group relationships simultaneously, we designed a dual-path attention mechanism: an intra-group attention mechanism and an inter-group attention mechanism.

[0053] The expression for calculating the intra-group attention weight is: These are learnable intra-group attention masks used to enhance intra-group interactions; they generate enhanced intra-group attention features by weighting and summing the grouped features of the semantic group using intra-group attention weights. .

[0054] The expression for calculating the inter-group attention weight is: ; These are learnable inter-group attention masks used to enhance inter-group interactions; the weighted sum of the grouped features of semantic groups is applied using inter-group attention weights to generate enhanced inter-group attention features. .

[0055] S54. By fusing intra-group attention features and inter-group attention features through a gate control mechanism, a fused attention feature is obtained.

[0056] The expression for fusing the outputs of the two paths using a gating mechanism is as follows: , in It is a learnable gating weight vector. Indicates intra-group attention characteristics. This represents the inter-group attention characteristics. This represents the sigmoid function. This represents element-wise multiplication. The final output is calculated through residual connections and a feed-forward network. ; ; in , and This represents the weight matrix of the two-layer feedforward network. and These are the corresponding bias vectors. GELU stands for Gaussian Error Linear Unit activation function, which introduces nonlinearity to enhance the model's representational power.

[0057] S55. Generate enhanced grouping features based on the fused attention features and grouping features.

[0058] S6. Combine text features and enhanced grouping features to obtain fused features, and generate a target density prediction map based on the fused features.

[0059] Enhanced grouped features are fused with textual features across modalities (e.g., through attention mechanisms or splicing) to generate density map prediction features. The fused features are then mapped to the target density prediction map through a lightweight regression head (e.g., a convolutional layer).

[0060] S7. Calculate the number of targets based on the target density prediction map to obtain the target count result.

[0061] First, the Implicit Attribute Channel Grouping (IACG) mechanism dynamically assigns feature channels to different semantic groups using a prototype vector-based soft allocation strategy, without requiring predefined visual attributes. This data-driven approach automatically discovers latent channel structures and adapts to visual characteristics in different scenarios. Second, we designed a Cross-Group Attention Mechanism (CGAM), which establishes an efficient information exchange pathway between groups through a hierarchical attention architecture and a dual-path design. Finally, Intra-Group Consistency Loss (IGCL) optimizes the consistency of features within groups, guiding the model to form semantically distinguishable feature groups, thereby improving representation quality and interpretability.

[0062] The density map visualization results predicted using the zero-shot target counting method of this application are as follows: Figure 6 As shown, our method uses the visual language model CLIP (Contrastive Language–Image Pre-training) as the counting backbone network, which significantly improves the model's performance under different object categories, scales, and densities while maintaining zero-shot capability.

[0063] To verify the effectiveness of the zero-shot target counting method in this application, we evaluated the results using multiple datasets, including FSC-147, CARPK, and ShanghaiTech.

[0064] FSC-147 is a comprehensive benchmark designed specifically for class-independent object counting. It contains 6135 images covering 147 distinct object classes with no overlap. The dataset provides each training image with a class label, point-level annotations, and three randomly selected example image patches. In our experimental protocol, we utilize only the class names as textual cues for counting, without using the example image patches. This better simulates real-world deployment constraints and aligns with the zero-shot evaluation paradigm.

[0065] CARPK is a dedicated counting benchmark focused on vehicle enumeration, containing 1448 high-resolution aerial views of parking lots and precise annotations for 89,777 individual vehicles. The dataset is characterized by high object density and complex spatial distribution, making it particularly suitable for evaluating fine-grained counting capabilities. We leverage CARPK to evaluate the cross-domain generalization performance of our model, validating its adaptability to specialized counting scenarios without domain-specific optimization.

[0066] The ShanghaiTech dataset is an authoritative benchmark in the field of crowd density estimation. It is constructed as two complementary partitions (A and B), totaling 1198 labeled images. Part A contains 482 images (400 for training and 82 for testing), primarily sourced from internet media; Part B contains 716 images (400 for training and 316 for testing), captured through fixed surveillance infrastructure. These partitions exhibit significant differences in crowd density and scene composition, providing a rigorous testing ground for evaluating the robustness of models under different environmental conditions and crowd configurations.

[0067] The comprehensive utilization of these three datasets provides a rigorous evaluation protocol that covers multiple technical challenges: scale variations in FSC-147, density extremes in CARPK, and complex environmental factors in ShanghaiTech. This multi-benchmark strategy enables us to quantitatively evaluate the algorithm's generalization ability on class-independent and class-specific counting tasks, providing statistical validity for performance comparisons across different data distributions.

[0068] Our model underwent a systematic training process on the FSC-147 training dataset. During training, we integrated Implicit Attribute Channel Grouping (IACG), Cross-Group Attention Mechanism (CGAM), Intra-Group Consistency Loss (IGCL), and Mean Squared Error (MSE) loss on the original images, performing 200 epochs of fine-tuning. The entire training pipeline used a batch size of 16 and employed the AdamW optimizer with a fixed learning rate. The learning rate decays by a factor of 0.33 after 100 epochs. To optimize training efficiency, we implement a hierarchical learning rate decay strategy, which applies different scaling factors to the parameters of different network layers.

[0069] Following the previous class-agnostic counting method, we evaluate the performance by mean absolute error (MAE) and root mean square error (RMSE) in the following way.

[0070] in, The total number of images in the test set, For the model to the first The predicted count values ​​for the images, For the first The ground truth value of each image.

[0071] In experiments on the FSC147 dataset, we compared our proposed method with state-of-the-art generalized counting methods. Quantitative results are summarized in [the table / incomplete]. Figure 7 This table presents the results for reference-less and zero-shot counting methods. Our method achieves state-of-the-art performance in zero-shot counting. On the test set, our Mean Absolute Error (MAE) is 16.55 and the Root Mean Square Error (RMSE) is 101.27, outperforming methods such as ZSOC, CLIP-Count, VA-Count, and VLCounter. On the validation set, our method also shows competitive results with an MAE of 17.15 and an RMSE of 58.82. This clearly demonstrates that on the FSC147 dataset, our method significantly outperforms existing state-of-the-art zero-shot target counting methods. Visualization results on the FSC147 dataset are shown below. Figure 6 As shown.

[0072] In our testing on the CARPK dataset, we followed the setup of previous class-agnostic counting methods to evaluate the cross-dataset generalization ability of our proposed method. Specifically, the model was trained on FSC147 and then evaluated on the CARPK test set. Figure 8The results demonstrate the performance of few-shot (3-shot) methods, including FamNet, BMNet, and BMNet+, as well as zero-shot methods, including RCC, CLIP-Count, and our method. Among the zero-shot methods, our method exhibits superior performance. With an MAE of 11.37 and an RMSE of 16.32, our method outperforms RCC and achieves comparable results with CLIP-Count. Figure 8 The results show that our method significantly outperforms previous few-sample counting methods in cross-dataset evaluations on the CARPK dataset and is highly competitive in zero-sample counting methods.

[0073] In our experiments on the ShanghaiTech crowd counting dataset, we compared our method with both specific and generic target counting methods. We evaluated cross-dataset performance by training on one partition of the dataset and testing on another. Figure 9 The results show that our method significantly outperforms traditional class-specific counting methods such as MCNN and CrowdCLIP. Even without specialized training, it surpasses generic counting methods such as RCC and CLIP-Count. This demonstrates its superior generalization ability across datasets.

[0074] We introduce a novel framework for zero-shot object counting that overcomes the limitations of existing visual language models in channel handling. Our framework has three key components: Implicit Attribute Channel Grouping (IACG), Cross-Group Attention (CGAM), and Intra-Group Consistency Loss (IGCL). IACG dynamically assigns feature channels to different semantic groups without relying on predefined visual attributes. It uses learnable prototype vectors and a soft assignment strategy, which not only enhances the model's flexibility but also adapts to different visual scenarios. This allows the model to capture the latent semantic structure within feature channels, thereby improving its representational ability. CGAM, with its dual-path architecture, effectively models the complex interdependencies between different channel groups. By integrating intra-group and inter-group attention and using an adaptive gating fusion mechanism, it enriches the model's ability to exchange information between groups, resulting in better feature representations. IGCL optimizes the semantic consistency within channel groups. By minimizing the Euclidean distance between intra-group features and their centroid representations, it guides the model to form compact and semantically distinguishable feature clusters. This improves the model's interpretability and enhances its discriminative ability. We applied our framework to zero-shot object counting tasks and evaluated it on several challenging benchmarks, including the FSC-147, CARPK, and ShanghaiTech crowd counting datasets. Experimental results show that our method outperforms existing methods in handling multi-scale, multi-class object counting scenarios. It demonstrates excellent generalization ability across different object types, densities, and scales, providing a novel solution for the zero-shot object counting domain.

[0075] Based on the same inventive concept, an embodiment of the present invention provides a zero-sample target counting system.

[0076] The zero-sample target counting system of the present invention can be installed in an electronic device. Depending on the functions implemented, the zero-sample target counting system includes: The acquisition module is used to acquire the text prompts and the image to be recognized for the target. The first feature extraction module is used to extract features from text prompt words to obtain text features; The second feature extraction module is used to extract features from the image to be recognized, thereby obtaining visual features; The implicit attribute channel grouping module is used to dynamically allocate the feature channels of visual features to multiple semantic groups through the implicit attribute channel grouping mechanism to obtain grouped features; The cross-group attention processing module is used to process group features using the cross-group attention mechanism to obtain enhanced group features; The fusion module is used to fuse text features and enhanced grouping features to obtain fused features, and generate a target density prediction map based on the fused features; The output module is used to calculate the number of targets based on the target density prediction map and obtain the target count result.

[0077] The module described in this invention can also be called a unit, which refers to a series of computer program segments that can be executed by the processor of an electronic device and can perform a fixed function, and are stored in the memory of the electronic device.

[0078] The various variations and specific examples of the zero-sample target counting method provided in the above embodiments are also applicable to the zero-sample target counting system of this embodiment. Through the foregoing detailed description of the zero-sample target counting method, those skilled in the art can clearly understand the implementation method of the zero-sample target counting system in this embodiment. For the sake of brevity, they will not be described in detail here.

[0079] This application also discloses an electronic device, such as Figure 10 The diagram shown is a schematic representation of an electronic device for a zero-sample target counting method according to an embodiment of the present invention. The electronic device may include at least one processor 10, a memory 11 communicatively connected to the at least one processor, a communication bus 12, and a communication interface 13. It may also include a computer program, such as a method program for the zero-sample target counting method, stored in the memory 11 and executable on the processor 10.

[0080] In some embodiments, the processor 10 may be composed of integrated circuits, such as a single packaged integrated circuit or multiple integrated circuits with the same or different functions, including combinations of one or more central processing units (CPUs), microprocessors, digital processing chips, graphics processors, and various control chips. The processor 10 is the control unit of the electronic device, connecting various components of the entire electronic device through various interfaces and lines. It executes programs or modules stored in the memory 11 (e.g., methods for zero-sample target counting) and calls data stored in the memory 11 to perform various functions of the electronic device and process data.

[0081] The memory 11 includes at least one type of readable storage medium, including flash memory, portable hard drive, multimedia card, card-type memory (e.g., SD or DX memory), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 11 can be an internal storage unit of an electronic device, such as a portable hard drive. In other embodiments, the memory 11 can be an external storage device of the electronic device, such as a plug-in portable hard drive, smart media card (SMC), secure digital (SD) card, flash card, etc. Furthermore, the memory 11 can include both internal and external storage units of the electronic device. The memory 11 can be used not only to store application software and various types of data installed on the electronic device, such as the code of a zero-sample target counting method, but also to temporarily store data that has been output or will be output.

[0082] The communication bus 12 can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This bus can be divided into an address bus, a data bus, a control bus, etc. The bus is configured to enable communication between the memory 11 and at least one processor 10, etc.

[0083] Communication interface 13 is used for communication between the aforementioned electronic device and other devices, including a network interface and a user interface. Optionally, the network interface may include a wired interface and / or a wireless interface (such as a Wi-Fi interface, Bluetooth interface, etc.), typically used to establish communication connections between the electronic device and other electronic devices. The user interface may be a display, an input unit (such as a keyboard), and optionally, a standard wired or wireless interface. Optionally, in some embodiments, the display may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or an OLED (Organic Light-Emitting Diode) touchscreen, etc. The display may also be appropriately referred to as a screen or display unit, used to display information processed in the electronic device and to display a visual user interface.

[0084] Figure 10 Only electronic devices with components are shown; it will be understood by those skilled in the art that... Figure 10The structure shown does not constitute a limitation on the electronic device and may include fewer or more components than shown, or combine certain components, or have different component arrangements.

[0085] For example, although not shown, the electronic device may also include a power supply (such as a battery) to power various components. Preferably, the power supply can be logically connected to at least one processor 10 via a power management device, thereby enabling functions such as charging management, discharging management, and power consumption management. The power supply may also include one or more DC or AC power supplies, recharging devices, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components. The electronic device may also include various sensors, Bluetooth modules, Wi-Fi modules, etc., which will not be elaborated further here.

[0086] It should be understood that the embodiments are for illustrative purposes only and are not limited to this structure in the scope of the patent application.

[0087] Furthermore, if the modules / units integrated into the electronic device are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. The computer-readable storage medium can be volatile or non-volatile.

[0088] This application provides a computer-readable storage medium, including, for example, any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive, a portable hard drive, a magnetic disk, an optical disk, a computer memory, or a read-only memory (ROM). The computer-readable storage medium stores a computer program that can be loaded by a processor and execute the zero-sample target counting method of the above embodiments.

[0089] In the description of this specification, the references to terms such as "an embodiment," "some embodiments," "example," "specific example," "a implementation," "a preferred implementation," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the present invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0090] Although embodiments of the invention have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the claims and their equivalents.

Claims

1. A zero-sample target counting method, characterized in that, The method includes: Obtain the text prompts and the image of the target to be identified; Text features are obtained by extracting features from text prompt words; Visual features are obtained by extracting features from the image to be recognized; The implicit attribute channel grouping mechanism dynamically allocates the feature channels of visual features to multiple semantic groups to obtain grouped features; By utilizing a cross-group attention mechanism to process group features, enhanced group features are obtained. By fusing text features and enhanced grouping features, a fused feature is obtained, and a target density prediction map is generated based on the fused feature. The number of targets is calculated based on the target density prediction map, and the target count result is obtained.

2. The zero-sample target counting method as described in claim 1, characterized in that, The implicit attribute channel grouping mechanism dynamically allocates the feature channels of visual features to multiple semantic groups, resulting in grouped features including: Visual features are transformed to enhance their representational power, resulting in transformed visual features. Define a set of learnable prototype vectors, where each prototype vector represents the center of a semantic group, and calculate the similarity between the transformed visual features and the prototype vectors corresponding to each semantic group. The soft assignment probability from channel to semantic group is determined based on the similarity between the transformed visual features and the prototype vectors corresponding to each semantic group. Based on the soft assignment probability, the features corresponding to each spatial location of the visual features are softly assigned to various semantic groups to obtain grouped features.

3. The zero-sample target counting method as described in claim 2, characterized in that, The steps for performing feature transformation on visual features include: The visual features are sequentially subjected to linear transformation, layer normalization, and activation processing to obtain the transformed visual features. Visual features are represented as ,in, Represents a matrix. Indicates batch size. Represents the number of spatial locations. Represents the number of channels; the transformed visual features are denoted as , ; in, Represents a linear transformation. Representation layer normalization, This represents the activation function.

4. The zero-sample target counting method as described in claim 3, characterized in that, The formula for calculating the similarity between the transformed visual features and the prototype vectors corresponding to each semantic group is as follows: ； in, This represents the similarity between the features corresponding to each spatial location of the transformed visual features and the prototype vector; This represents the feature vector corresponding to the transformed visual feature at the b-th sample, the n-th position, and the c-th channel. represents the sample batch index, n represents the spatial location index, and c represents the channel index; Represents the prototype vector. The semantic group index representing the prototype vector.

5. The zero-sample target counting method as described in claim 4, characterized in that, By having temperature parameters The softmax function is used to calculate the soft assignment probability from channel to semantic group; the formula for calculating the soft assignment probability is: ； Representation of features The soft assignment probability, Indicates temperature parameter, A semantic group index representing visual features.

6. The zero-sample target counting method as described in claim 3, characterized in that, Grouping features are represented as , For each dimension of a semantic group, the method further includes: Calculate the centroid vector for each semantic group. The expression for calculating the centroid vector is as follows: ； Indicates sample The Middle The centroid vector of each semantic group This represents the feature vector corresponding to the visual feature at the b-th sample, the n-th position, and the c-th channel; n represents the spatial location index. The intra-group consistency loss is defined as the average Euclidean distance between the visual features and the centroids of the semantic group. This intra-group consistency loss is used to optimize the semantic consistency of visual features within the same semantic group. The expression for the intra-group consistency loss is: 。 7. The zero-sample target counting method according to any one of claims 1 to 6, characterized in that, The method of using cross-group attention mechanism to process group features to obtain enhanced group features includes: Perform layer normalization on the grouped features to obtain the normalized grouped features; The intra-group attention weights are calculated using a learnable intra-group attention mask and the query vector, key vector, and value vector of the normalized group features, and the intra-group attention features are determined based on the intra-group attention weights. The inter-group attention weights are calculated using a learnable inter-group attention mask and the query vector, key vector, and value vector of the normalized grouping features, and the inter-group attention features are determined based on the inter-group attention weights. By fusing intra-group and inter-group attention features through a gate control mechanism, a fused attention feature is obtained. Enhanced grouping features are generated based on the fusion of attention features and grouping features.

8. A zero-sample target counting system for implementing the zero-sample target counting method according to any one of claims 1 to 7, characterized in that, include: The acquisition module is used to acquire the text prompts and the image to be recognized for the target. The first feature extraction module is used to extract features from text prompt words to obtain text features; The second feature extraction module is used to extract features from the image to be recognized, thereby obtaining visual features; The implicit attribute channel grouping module is used to dynamically allocate the feature channels of visual features to multiple semantic groups through the implicit attribute channel grouping mechanism to obtain grouped features; The cross-group attention processing module is used to process group features using the cross-group attention mechanism to obtain enhanced group features; The fusion module is used to fuse text features and enhanced grouping features to obtain fused features, and generate a target density prediction map based on the fused features; The output module is used to calculate the number of targets based on the target density prediction map and obtain the target count result.

9. An electronic device, characterized in that, The electronic device includes: At least one processor (10); and, A memory (11) communicatively connected to the at least one processor (10); The memory (11) stores a computer program that can be executed by the at least one processor (10) to enable the at least one processor (10) to perform the zero-sample target counting method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program; when executed by a processor, the computer program implements the zero-sample target counting method as described in any one of claims 1 to 7.

Images