Automatic counting method based on attention mechanism

Abstract

The invention discloses an automatic counting method based on an attention mechanism. The automatic counting method comprises the steps of generating a prediction frame based on a Transfomer attention mechanism; the generated prediction frames are screened; carrying out clustering processing on the screened prediction frames; the class with the maximum number is selected for further screening, and an example object is obtained; objects are automatically counted based on bounding box position information of an example object and feature information in a box. The method provided by the invention has relatively high universality, can be applied to positioning and counting of common types of objects, accurately counts new types of objects under the condition that only a small number of annotation instances exist, and provides a more universal and efficient solution for a counting task in computer vision.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, and in particular provides an automatic counting method based on an attention mechanism. Background Art

[0002] Object counting is an important task in computer vision, with important applications in many fields such as surveillance, medical imaging, and autonomous driving. Traditional counting methods usually focus on specific categories of objects, such as humans, animals, or cells, and require a large amount of labeled data. However, these methods face the problems of high annotation costs, limited scalability, and inability to generalize across different object categories.

[0003] In recent years, the development of object counting has introduced few-shot learning techniques. These systems aim to count objects of arbitrary categories with a small number of annotated examples, thereby reducing the reliance on large annotated datasets. By combining few-shot regression tasks and novel adaptation strategies, these methods enhance the flexibility and intelligence of counting systems. Developing such systems requires creating robust algorithms for object recognition, density map prediction, and real-time adaptation to new categories.

[0004] The current counting method based on the Few-shot Adaptive and Matching Network (FamNet) first requires an input image and a small number of example objects annotated by bounding boxes, and uses a pre-trained model to extract multi-scale features of the image. At the same time, features are extracted from each example object through ROI pooling and matched with image features. Then, the correlation between image features and example features is calculated at different scales to generate multiple correlation maps. The correlation map is passed as input to the density prediction module, which generates a density map through convolution and upsampling operations. Each pixel value of the density map represents the probability of the existence of the target object at that position. By summing all the pixel values ​​on the density map, the total number of target objects in the image can be obtained. However, this method has the following disadvantages, that is, at the input image stage, it is necessary to manually annotate the bounding boxes of the instance objects, which is not only costly, but also time-consuming and labor-intensive. In addition, due to the reliance on manual annotation, it means that the counting process cannot be fully automated, which limits the efficiency and operability of this method in large-scale applications.

[0005] In summary, an automatic counting method is proposed to solve the problem of manual labeling in the few-sample counting task in the prior art. Summary of the invention

[0006] The purpose of this invention is to propose an automatic counting method based on the attention mechanism, so as to accurately count objects of new categories with only a small number of labeled instances.

[0007] In order to achieve the above object, the present invention provides an automatic counting method based on an attention mechanism, comprising:

[0008] S1: Generate prediction box based on Transformer attention mechanism;

[0009] S2: Screening the generated prediction boxes; and performing clustering on the screened prediction boxes;

[0010] S3: Select the class with the largest number for further screening to obtain example objects;

[0011] S4: Automatically count objects based on the bounding box position information of the example object and the feature information within the box.

[0012] Preferably, S1 generates a prediction box based on the Transformer attention mechanism, including:

[0013] S11: In the prediction box generation stage, a convolutional neural network (CNN) is used to extract a feature map from the input image; the feature map contains spatial and semantic information of the image;

[0014] S12: The extracted feature map is input into a Transformer encoder composed of multiple self-attention layers and feed-forward neural network layers. At the same time, the position encoding information is introduced. The encoder generates a more expressive feature representation through the self-attention mechanism.

[0015] S13: The feature representation output by the encoder is input into the Transformer decoder composed of multiple self-attention layers and feed-forward neural network layers. The decoder generates a fixed number of prediction boxes through the self-attention mechanism and the encoder-decoder attention mechanism, where each prediction box contains position information.

[0016] Preferably, S2 screens the generated prediction boxes; and performs clustering processing on the screened prediction boxes, including:

[0017] S21: defining a screening method for the prediction box generated for the input image, determining whether the prediction box is within the image boundary, and ensuring that the area of ​​the prediction box does not exceed one quarter of the image area; selecting a valid prediction box and corresponding features through the screening method;

[0018] S22: evaluating the effects of different numbers of clusters through the silhouette coefficient and selecting the optimal number of clusters; based on the optimal number of clusters, clustering the valid prediction boxes according to the features using the Kmeans clustering algorithm, and grouping the prediction boxes according to the cluster labels;

[0019] Preferably, S3 selects the class with the largest number for the next step of screening;

[0020] S31: Calculate the aspect ratio and area of ​​each prediction box, and set filtering thresholds based on the median and standard deviation of the area and aspect ratio, and retain the prediction boxes within the threshold range;

[0021] S32: randomly select several prediction boxes as the bounding boxes of the example object, calculate the intersection over union (IoU) between two prediction boxes, and verify whether their IoUs meet the requirements, ensuring that the maximum IoU between the selected prediction boxes does not exceed the set threshold;

[0022] S33: If a box that meets the conditions is found, the result is output, otherwise a predicted box is randomly selected again until the conditions are met.

[0023] Preferably, S4 uses the FamNet counting method based on the bounding box of the example object to automatically count the objects, including:

[0024] S41: Use the generated bounding box coordinates as the input of FamNet;

[0025] S42: The FamNet network extracts multi-scale features of the image and features of the sample object through the feature extraction module;

[0026] S43: The density prediction module generates a density map based on these features, and obtains the final object count by adding all pixel values ​​in the density map.

[0027] The automatic counting method based on the attention mechanism provided by the present invention solves the problem of manually labeling the bounding boxes of instance objects in the prior art; the method does not require category information when obtaining instances of objects to be counted, can accurately screen out valid prediction boxes, and can avoid overlapping when generating prediction boxes;

[0028] The new method proposed in the present invention has strong versatility and can be applied to the positioning and counting of common types of objects, such as people, animals, vehicles, etc. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] The present invention is further described in detail below in conjunction with the accompanying drawings and embodiments:

[0030] Figure 1 A schematic diagram of a feature extraction backbone network provided by the present invention;

[0031] Figure 2 A schematic diagram of a transformer encoder-decoder provided by the present invention;

[0032] Figure 3 This is a schematic diagram of overlapping prediction boxes generated without any IoU restriction in the embodiment of the present invention;

[0033] Figure 4A schematic diagram showing setting of an IoU threshold and non-overlapping prediction boxes for an embodiment of the present invention;

[0034] Figure 5 This is a schematic diagram of the FamNet counting principle of the present invention;

[0035] Figure 6 This is a dense scene - bottle cap image bounding box generation and counting effect diagram of an embodiment of the present invention;

[0036] Figure 7 This is a dense scene-car image bounding box generation and counting effect diagram of an embodiment of the present invention;

[0037] Figure 8 This is a diagram showing the generation and counting effects of a dense scene-bird image bounding box according to an embodiment of the present invention. DETAILED DESCRIPTION

[0038] The present invention will be further explained below in conjunction with specific implementation schemes, but the present invention is not limited thereto.

[0039] The method provided by the present invention can allow accurate counting of objects of new categories with only a small number of labeled instances, providing a more general and efficient solution for counting tasks in computer vision; the automatic counting method based on the attention mechanism of the present invention first introduces the attention mechanism in the generation of the annotation box, then screens the generated annotation box, and clusters the screened annotation box, and finally selects the object category with the highest frequency in the image as the example object, thereby realizing automatic counting of the object.

[0040] Specifically, S1: Generate prediction boxes based on Transformer attention mechanism; Use Transformer's self-attention mechanism and encoder-decoder structure to automatically generate prediction boxes for instance objects;

[0041] like Figure 1 As shown in the figure, in the prediction box generation stage, a convolutional neural network (CNN) is first used to extract feature maps from the input image. These feature maps contain the spatial and semantic information of the image and provide input for the subsequent Transformer encoder. Next, the extracted feature maps are input into the Transformer encoder composed of multiple self-attention layers and feedforward neural network layers. At the same time, position encoding information is introduced. Through the self-attention mechanism, the encoder can capture the global relationship between different positions in the feature map, thereby generating a more expressive feature representation. Then, the feature representation output by the encoder is input into the Transformer decoder, which is also composed of multiple self-attention layers and feedforward neural network layers. The decoder generates a set of fixed number of prediction boxes (1000 in this embodiment) through the self-attention mechanism and the encoder-decoder attention mechanism. Figure 2As shown in , each prediction box output by the decoder contains position information. In order to be more general, the category information is not used for classification, so the prediction box without category information output by the decoder is directly used as the result of this stage.

[0042] S2: Filter the generated prediction boxes; perform clustering on the filtered prediction boxes; combine the Kmeans clustering method to generate several bounding boxes of instance objects without requiring category information;

[0043] S3: Select the class with the largest number for further screening to obtain sample objects; use a multiple screening mechanism to set the filtering threshold based on the median and standard deviation of the area and aspect ratio, and verify through IoU to ensure that the screened bounding boxes meet the requirements

[0044] For the prediction box generated in the previous stage, first, for the prediction box generated by the input image, a method is defined to determine whether the prediction box is within the image boundary and ensure that the area of ​​the prediction box does not exceed one-quarter of the image area. This method is used to screen out valid prediction boxes and corresponding features. Next, the Kmeans clustering algorithm is used to cluster the prediction boxes according to the features. In order to determine the optimal number of clusters, the silhouette score is used as shown in the following formula to evaluate the effects of different numbers of clusters, and finally the optimal number of clusters is selected.

[0045]

[0046] After determining the optimal number of clusters, use the clustering algorithm to cluster the valid features and group the prediction boxes according to the cluster labels. Select the class with the largest number for the next screening.

[0047] Since the counting scene is a scene with highly dense and repeated objects, the shapes and sizes of the objects to be counted are similar. In order to further filter the prediction boxes, the aspect ratio and area of ​​each prediction box are calculated, and the filtering thresholds are set based on the median and standard deviation of the area and aspect ratio, and the prediction boxes within the threshold range are retained.

[0048] Finally, we randomly select several prediction boxes as the bounding boxes of the example objects, calculate the intersection over union (IoU) between the two prediction boxes, and verify whether their IoU meets the requirements, ensuring that the maximum IoU between the selected boxes does not exceed the set threshold. Figure 3 and Figure 4 The generated results are before and after IoU restriction. If a box that meets the conditions is found, the result is output, otherwise the predicted box is randomly selected again until it meets the conditions.

[0049] Through the above steps, the present invention realizes the effective screening of prediction boxes and the selection of example objects, and is suitable for various target detection tasks.

[0050] S4: Automatically count objects based on the bounding box position information of the example object and the feature information within the box.

[0051] The counting method of the present invention is based on the existing FamNet counting method. The bounding box coordinates generated by the present invention can be directly used as the input of FamNet to perform object counting operations. The counting principle of FamNet is a few-sample regression task. The input is a query image and its corresponding small number of example objects, which are represented by bounding boxes in the image. The network first extracts the multi-scale features of the image and the features of the sample objects through the feature extraction module. Then, the density prediction module generates a density map based on these features, and the final object count is obtained by adding all the pixel values ​​in the density map. FamNet has good generalization ability. During the testing phase, the network is adjusted to adapt to new categories through a small number of gradient descent steps. This adaptation mechanism utilizes the position information of the example objects provided during the test, thereby improving the counting accuracy. The principle of the FamNet counting method is as follows Figure 5 shown.

[0052] Since the FamNet counting method still requires manual operation when obtaining the bounding box of the example object, the present invention solves this problem by introducing an automatic prediction box generation method. The entire counting process does not require manual intervention, achieving fully automated counting.

[0053] In order to verify the feasibility and accuracy of the proposed method, we selected dense scene images of different types for experiments. The experimental data included target objects of multiple categories and different densities. The experimental results are as follows: Figures 6 to 8 As shown, the complexity of the three selected scenarios ranges from low to high. Figure 6 are bottle cap images with regular distribution and the same shape and size. Figure 7 For vehicle images with relatively regular distribution and similar features, Figure 8 The images are irregularly distributed and varied bird images. It can be seen from the figure that the bounding boxes of the sample objects generated by the method of the present invention are all accurate, and after the bounding box coordinates are input into the counting method based on the few-sample adaptive matching network (FamNet), the counting can be completed smoothly.

[0054] It can be seen from the experimental results that the proposed method can well solve the problem that the FamNet counting method requires manual labeling of bounding boxes of instance objects, achieves complete automation, and greatly saves economic and time costs. The proposed method does not require category information when generating prediction boxes and has very high versatility. In addition, through the multiple filtering mechanism, the screening accuracy of the prediction box is further improved, ensuring the reliability and accuracy of the detection results.

[0055] The detailed description of the present invention is written in a progressive manner, emphasizing the differences between the various implementations, and similar parts thereof can be referred to each other.

[0056] The embodiments of the present invention are described in detail above with reference to the accompanying drawings, but the present invention is not limited to the above embodiments, and various changes can be made within the knowledge scope of ordinary technicians in this field without departing from the purpose of the present invention.

Claims

1. The automatic counting method based on the attention mechanism is characterized by: include: S1: Generate prediction box based on Transformer attention mechanism; S2: Screening the generated prediction boxes; and performing clustering on the screened prediction boxes; S3: Select the class with the largest number for further screening to obtain example objects; S4: Automatically count objects based on the bounding box position information of the example object and the feature information within the box.

2. The automatic counting method based on the attention mechanism according to claim 1, characterized in that: S1 generates prediction boxes based on the Transformer attention mechanism, including: S11: In the prediction box generation stage, a convolutional neural network (CNN) is used to extract a feature map from the input image; the feature map contains spatial and semantic information of the image; S12: The extracted feature map is input into a Transformer encoder composed of multiple self-attention layers and feed-forward neural network layers. At the same time, the position encoding information is introduced. The encoder generates a more expressive feature representation through the self-attention mechanism. S13: The feature representation output by the encoder is input into the Transformer decoder composed of multiple self-attention layers and feed-forward neural network layers. The decoder generates a fixed number of prediction boxes through the self-attention mechanism and the encoder-decoder attention mechanism, where each prediction box contains position information.

3. The automatic counting method based on the attention mechanism according to claim 1, characterized in that: S2 screens the generated prediction box; The filtered prediction boxes are clustered, including: S21: defining a screening method for the prediction box generated for the input image, determining whether the prediction box is within the image boundary, and ensuring that the area of ​​the prediction box does not exceed one quarter of the image area; selecting a valid prediction box and corresponding features through the screening method; S22: Evaluate the effects of different numbers of clusters through the silhouette coefficient and select the optimal number of clusters; based on the optimal number of clusters, use the Kmeans clustering algorithm to cluster the valid prediction boxes according to the features, and group the prediction boxes according to the cluster labels.

4. The automatic counting method based on the attention mechanism according to claim 1, characterized in that: S3 selects the class with the largest number for the next step of screening: S31: Calculate the aspect ratio and area of ​​each prediction box, and set filtering thresholds based on the median and standard deviation of the area and aspect ratio, and retain the prediction boxes within the threshold range; S32: Randomly select several prediction boxes as the bounding boxes of the example object, calculate the intersection over union (IoU) between two prediction boxes, and verify whether their IoUs meet the requirements, ensuring that the maximum IoU between the selected prediction boxes does not exceed the set threshold. S33: If a box that meets the conditions is found, the result is output, otherwise a predicted box is randomly selected again until the conditions are met.

5. The automatic counting method based on the attention mechanism according to claim 1, characterized in that: S4 uses the FamNet counting method based on the bounding box of the example object to automatically count the objects, including: S41: Use the generated bounding box coordinates as the input of FamNet; S42: The FamNet network extracts multi-scale features of the image and features of the sample object through the feature extraction module; S43: The density prediction module generates a density map based on these features, and obtains the final object count by adding all pixel values ​​in the density map.

Images