Multi-scale detection and example box selection fused few-sample learning target counting method

Abstract

The invention discloses a few-sample learning target counting method fusing multi-scale detection and example box selection, and the method comprises the steps: 1, carrying out the preprocessing of a query image and an example image with scale information, and carrying out the splicing, and obtaining a spliced image; 2, inputting the spliced image into a visual transformer model to obtain query image features; and 3, inputting the query image features into a regression decoder, and completing target counting in the query image. According to the method, the problem of scale change in the image is effectively solved, and the accuracy and robustness of target detection are remarkably improved.

Description

Technical Field

[0001] The present invention relates to the technical field of few-shot learning object counting, and in particular to a few-shot learning object counting method that integrates multi-scale detection and example box selection. Background Art

[0002] The core objective of object counting is to predict the number of objects in a given image. This task is widely used in various computer vision problems, particularly when dealing with multiple similar or highly overlapping objects, such as crowd counting, vehicle counting, and cell counting. However, a key challenge in object counting is the significant variation in object type, form, posture, and distribution, making accurate object counting a complex task. Accurately estimating the number of objects is particularly challenging in situations with densely populated objects, complex backgrounds, varying lighting, or occlusion.

[0003] Traditional object counting methods rely primarily on large-scale annotated datasets, using deep learning models (such as convolutional neural networks (CNNs)) to learn the spatial distribution characteristics of objects. These methods work well when data is abundant, but when data is scarce, the models are prone to overfitting, resulting in decreased counting accuracy. In recent years, with the development of computer vision, few-shot learning (FSL) has become an emerging research area that has attracted increasing attention. Few-shot learning aims to solve the problem of how to enable models to learn techniques that generalize well and handle new tasks when the number of samples is extremely limited. This is particularly important for object counting tasks, as in many scenarios, large amounts of annotated data are unavailable or the annotation cost is prohibitive. In some specific fields (such as medical imaging and ecological monitoring), annotated data is even extremely scarce.

[0004] Current state-of-the-art few-shot learning object counting methods, such as the CACViT model, utilize VisionTransformer to improve detection capabilities. While this has advanced the field to some extent, some pressing issues remain, particularly when dealing with objects with significant scale variations. In many practical applications, factors such as the size, morphology, and distribution of target objects often change. For example, in urban traffic monitoring, vehicles vary greatly in scale; in ecological conservation, species may be distributed across a range of scales. While visual transformers have advantages in handling complex contextual relationships, they are prone to errors when dealing with large-scale or small-scale objects due to their lack of multi-scale feature representation, affecting overall counting accuracy and robustness.

[0005] Furthermore, all current few-shot learning target technology approaches typically require users to provide example boxes (bounding boxes of labeled objects) during the application phase. However, drawing accurate example boxes is not easy, especially when dealing with complex scenes or small objects. First, users need to accurately identify the location and size of the target object, which is particularly difficult when objects are densely packed, overlapping, or occluded. Summary of the Invention

[0006] To at least partially address the existing issues of low accuracy and robustness in counting and inaccurate example box drawing when the target object scale varies significantly, the present invention provides a few-shot learning target counting method that integrates multi-scale detection and example box selection, achieving accurate counting through a visual transformer model and a regression decoder. Targeting scenarios with scale variations, the present invention designs a multi-scale dilated convolution feature extraction module and a global channel-spatial attention module within the visual transformer model to improve the performance of the visual transformer model in detecting objects of different scales. The global channel-spatial attention module further improves the effective utilization of feature information by enhancing the model's global understanding and expression capabilities of the input feature map. The combination of these two effectively addresses the issue of scale variation within the image and significantly improves the accuracy and robustness of target detection. Furthermore, by setting up the example box drawing process, the accuracy of the example box drawing is guaranteed.

[0007] In order to achieve the above object, the technical solution of the present invention is:

[0008] The present invention proposes a few-shot learning object counting method that integrates multi-scale detection and example box selection, including:

[0009] Step 1: Preprocess the query image and the sample image with scale information and then splice them to obtain a spliced ​​image to facilitate image feature extraction;

[0010] Step 2: Input the spliced ​​image into the visual transformer model to obtain the query image features, so as to capture the relationship between image features;

[0011] Step 3: Input the query image features into the regression decoder to complete the object counting in the query image.

[0012] Furthermore, the step 1 specifically includes:

[0013] The query image and the sample image with scale information are divided into multiple small blocks, and the multiple small blocks are converted into tokens;

[0014] The tokens are flattened and then stitched together to obtain a stitched image.

[0015] Furthermore, the visual transformer model includes a plurality of transformer units and reshaping units connected in sequence, wherein the transformer unit includes a multi-head attention, a first-layer normalization, a multi-layer perceptron, a multi-scale dilated convolution feature extraction module, a global channel-spatial attention module, and a second-layer normalization;

[0016] The multi-head attention is used to process the spliced ​​images to obtain a similarity measure, so as to capture different semantic information;

[0017] The first layer normalization is used to perform layer normalization processing on the features after element-by-element addition of the output of the multi-head attention and the spliced ​​image;

[0018] The multi-layer perceptron is used to process the normalized output of the first layer to extract important features;

[0019] The multi-scale dilated convolution feature extraction module is used to extract features from the output of the multi-layer perceptron;

[0020] The global channel-spatial attention module is used to extract features after element-by-element addition of the output of the multi-scale dilated convolution feature extraction module and the output of the first layer normalization;

[0021] The second layer normalization is used to perform layer normalization processing on the output of the global channel-spatial attention module;

[0022] The reshaping unit is used to splice the output of the last transformer unit and the output of the multi-head attention in the first transformer unit, and then reshape the spliced ​​output into image blocks to facilitate subsequent counting.

[0023] Furthermore, the multi-scale dilated convolution feature extraction module includes multiple parallel convolution branches, splicing layers, spatial attention mechanisms, channel attention mechanisms, and addition units;

[0024] The multiple parallel convolution branches are used to extract features of different scales input by the multi-scale dilated convolution feature extraction module;

[0025] The splicing layer is used to splice the outputs of multiple parallel convolution branches to obtain a comprehensive feature map;

[0026] The spatial attention mechanism is used to extract features from the comprehensive feature map to obtain an output feature map of the spatial attention mechanism, which facilitates feature extraction at multiple spatial scales;

[0027] The channel attention mechanism is used to extract features from the comprehensive feature map to obtain the output feature map of the channel attention mechanism, so as to enhance the interdependence between channels;

[0028] The adding unit is used to add the output feature map of the spatial attention mechanism and the output feature map of the channel attention mechanism element by element, and input the added feature map into the convolution layer for dimensionality reduction and integration to obtain the final output feature map.

[0029] Furthermore, the channel attention mechanism is used to extract features from the comprehensive feature map to obtain an output feature map of the channel attention mechanism, specifically including:

[0030] Perform global average pooling on the comprehensive feature map to obtain the global features of each channel;

[0031] A fully connected layer that sequentially connects two layers of global features of each channel; the first fully connected layer includes a ReLU activation function, and the second fully connected layer includes a Sigmoid activation function;

[0032] The features processed by the two fully connected layers are multiplied with the comprehensive feature map to obtain the output feature map of the channel attention mechanism.

[0033] Furthermore, the spatial attention mechanism is used to extract features from the comprehensive feature map to obtain an output feature map of the spatial attention mechanism, specifically including:

[0034] Perform global pooling operation on the comprehensive feature map to obtain the spatial feature map;

[0035] The importance weight of each spatial position is calculated based on a 1×1 convolutional layer with a sigmoid activation function;

[0036] The importance weight of each spatial position is multiplied element-wise with the spatial feature map to obtain the output feature map of the spatial attention mechanism.

[0037] Furthermore, the global channel-spatial attention module includes a channel attention submodule, a channel shuffling submodule, a spatial attention submodule and a connection submodule;

[0038] The channel attention submodule is used to perform dimensionality transformation on the input feature map of the global channel-spatial attention module to highlight key features;

[0039] The channel shuffling submodule divides the features obtained by element-wise multiplication of the output of the channel attention submodule and the input feature map of the global channel-spatial attention module into multiple groups, performs a transposition operation on the grouped feature maps, and finally restores the transposed feature maps to their original shape to facilitate mixing and sharing of information;

[0040] The spatial attention submodule is used to perform multiple convolution operations on the output of the channel shuffling submodule;

[0041] The connection submodule is used to perform element-by-element multiplication of the output of the channel shuffling submodule and the output of the spatial attention submodule.

[0042] Furthermore, the channel attention submodule is used to perform dimensionality transformation processing on the input feature map of the global channel-spatial attention module, specifically including:

[0043] Rearrange the dimensions of the input feature map of the global channel-spatial attention module so that the channel dimension is at the end;

[0044] The feature map after dimension rearrangement is input into two layers of multi-layer perceptrons connected sequentially for processing to better capture the global dependency between channels;

[0045] The feature map processed by the two-layer multi-layer perceptron is reversely rearranged to restore the dimension of the feature map to the dimension before dimension rearrangement.

[0046] Furthermore, the spatial attention submodule is used to perform multiple convolution operations on the output of the channel shuffling submodule, specifically including:

[0047] The output of the channel shuffling submodule is input into the first 7x7 convolutional layer, and the output of the first 7x7 convolutional layer is batch normalized and ReLU activated.

[0048] The processed feature map is input into the second 7x7 convolutional layer, and the output of the second 7x7 convolutional layer is batch normalized and processed with the Sigmoid activation function.

[0049] Furthermore, the visual transformer model needs to be trained. During training, instance boxes need to be drawn. The instance box drawing process specifically includes:

[0050] The target contour is sampled by manually drawing lines, and accurate segmentation is performed based on the sampling points and the segmentation model to generate multiple mask proposal boxes;

[0051] The mask proposal boxes from the same curve sampling point are retained, and all areas outside the mask area in the mask proposal box are set to zero to complete the example box construction, which facilitates clear separation of the background and greatly improves the quality and efficiency of subsequent feature extraction.

[0052] Beneficial effects of the present invention:

[0053] (1) Aiming at scenarios with scale changes, the present invention designs a multi-scale dilated convolution feature extraction module and a global channel-spatial attention module to improve the performance of the visual transformer model in detecting objects of different scales. Specifically, the multi-scale dilated convolution feature extraction module sets different convolution kernel sampling rates, thereby enhancing the model's adaptability to targets of different sizes and effectively dealing with the challenges brought by scale changes in images; while the global channel-spatial attention module further improves the effective use of feature information by enhancing the model's global understanding and expression capabilities of the input feature map. The combination of these two effectively solves the problem of scale changes within images and significantly improves the accuracy and robustness of target detection.

[0054] (2) The present invention proposes a method for drawing example frames. This method can not only accurately locate the target but also clearly separate the background, greatly improving the quality and efficiency of subsequent feature extraction. In addition, this method is flexible in design and does not conflict with the existing network model structure. It can be used as a plug-and-play data enhancement method. This innovative selection method changes the traditional few-sample example frame selection method, allowing users to quickly and easily obtain example frames without background interference and with accurate positioning, effectively optimizing the overall processing flow of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Figure 1 Flowchart of a method for object counting using few-shot learning that integrates multi-scale detection and example box selection, provided in an embodiment of the present invention.

[0056] Figure 2 Schematic diagram of a few-shot learning object counting method that integrates multi-scale detection and example box selection, provided in an embodiment of the present invention.

[0057] Figure 3 A schematic diagram of a multi-scale dilated convolution feature extraction module provided in an embodiment of the present invention.

[0058] Figure 4 Schematic diagram of a global channel-spatial attention module provided by an embodiment of the present invention.

[0059] Figure 5 A schematic diagram of example frame selection provided in an embodiment of the present invention.

[0060] Figure 6 A schematic diagram of an example frame effect provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0061] To make the objectives, technical solutions, and advantages of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly described below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0062] Example 1

[0063] like Figure 1 and Figure 2 As shown in Figure 2, the few-shot learning object counting method that integrates multi-scale detection and example box selection includes:

[0064] S101: Pre-process the query image and the example image with scale information and then stitch them together to obtain a stitched image.

[0065] Specifically, the overall network process is to first split the query image and the example image with scale information into small blocks, convert these small blocks into tokens, and then flatten these tokens and splice them together.

[0066] S102: Input the spliced ​​image into the visual transformer model to obtain query image features.

[0067] S103: Input the query image features into the regression decoder to complete the object counting in the query image.

[0068] The present invention uses the Vision Transformer model as the backbone network, and introduces the constructed Multi-Scale Dilated Convolution Feature Extraction Module (MSDCFE) and Global Channel-Spatial Attention Module (GCSA) internally. The process of the present invention is to first pre-process and splice the query image and the example image with scale information, and then input the spliced ​​image into the Vision Transformer for processing to capture the relationship between image features. The output of the last transformer unit and the output of the multi-head attention in the first transformer unit are spliced, and then the spliced ​​output is reshaped into image blocks. Finally, through the regression decoder, the model predicts the density map of the target object and completes the target counting in the query image. The present invention effectively solves the problem of scale variation within the image and significantly improves the accuracy and robustness of target detection.

[0069] Example 2

[0070] Based on the above embodiments, the present invention proposes a structure of a visual transformer model, which specifically includes:

[0071] The visual transformer model includes multiple transformer units and reshaping units connected in sequence. The transformer unit includes multi-head attention, first-layer normalization, multi-layer perceptron, multi-scale hole convolution feature extraction module, global channel-spatial attention module and second-layer normalization.

[0072] Multi-head attention is used to process the spliced ​​images to obtain a similarity measure. The first layer normalization is used to perform layer normalization on the features after element-by-element addition of the output of the multi-head attention and the spliced ​​image. The multi-layer perceptron is used to process the output of the first layer normalization to extract important features. The multi-scale dilated convolution feature extraction module is used to extract features from the output of the multi-layer perceptron. The global channel-spatial attention module is used to extract features after element-by-element addition of the output of the multi-scale dilated convolution feature extraction module and the output of the first layer normalization. The second layer normalization is used to perform layer normalization on the output of the global channel-spatial attention module. The reshaping unit is used to splice the output of the last transformer unit with the output of the multi-head attention in the first transformer unit, and then reshape the spliced ​​output into image blocks.

[0073] The multi-scale dilated convolution feature extraction module aims to enhance feature representation by utilizing different dilation rates and combining channel and spatial attention mechanisms. It consists of two parts: the first is the multi-scale dilated convolution component, and the second focuses on the fusion of channel and spatial attention.

[0074] Specifically, if Figure 3 As shown in the figure, the multi-scale dilated convolution feature extraction module includes multiple parallel convolution branches, splicing layers, spatial attention mechanisms, channel attention mechanisms and addition units.

[0075] Specifically, multiple parallel convolution branches adopt five parallel convolution branches to achieve feature extraction at different scales. Each branch is configured with a different void rate to expand the receptive field and capture spatial information across different ranges. The first branch uses a 1x1 convolution kernel to directly extract features while maintaining the spatial scale. The second branch uses a 3x3 convolution kernel and sets the void rate to 6 to moderately increase the receptive field. The third branch also uses a 3x3 convolution kernel, but with a void rate of 12, further expanding the receptive field to capture a wider range of contextual information. The fourth branch uses a 3x3 convolution kernel with a void rate of 18, providing the largest receptive field. Finally, the fifth branch is an additional global average pooling branch that aims to extract global contextual features and enhance the model's ability to understand the overall layout.

[0076] The concatenation layer is used to fuse and calibrate the channel and spatial features of the features extracted by the five different branches mentioned above. Along the channel dimension, these features are connected into a comprehensive feature map.

[0077] The merged feature maps are then calibrated through two parallel attention mechanisms: spatial attention on the top and channel attention on the bottom.

[0078] Channel attention mechanism: First, perform global average pooling on the merged comprehensive feature map to obtain the global features of each channel:

[0079] z=GlobalAvgPool(F)

[0080] Where z∈R C represents the global features of each channel, C is the number of channels, and F is the comprehensive feature map.

[0081] Next, two fully connected layers are used to learn the importance weights of each channel. The first fully connected layer uses the ReLU activation function:

[0082] y=ReLU(W1z+b1)

[0083] Among them, W1 and b1 are the weight and bias of the first fully connected layer, and y is the output of the first fully connected layer.

[0084] The second fully connected layer uses the Sigmoid activation function to generate the weight of each channel:

[0085] a=Sigmoid(W2y+b2)

[0086] Among them, W2 and b2 are the weight and bias of the second fully connected layer, a∈R C Represents the importance weight of each channel.

[0087] Finally, these weights are multiplied with the original feature map channel by channel to achieve channel weighting:

[0088] F channel =F⊙a

[0089] Among them, ⊙ represents element-by-element multiplication, F channel It is the output feature map of the channel attention mechanism.

[0090] Spatial attention mechanism: for the comprehensive feature map F∈R C×H×W Perform a global pooling operation along the channel dimension to obtain a spatial feature map. Where H and W are the height and width of the comprehensive feature map respectively.

[0091] The importance weight of each spatial position is learned through a 1×1 convolutional layer with a sigmoid activation function:

[0092] W spatial =σ(K*F+b)

[0093] where K∈R 1×1×C×1 represents the convolution kernel, b is the bias term, σ represents the Sigmoid activation function, W spatial ∈R H×W represents the importance weight of each spatial position, and F is the comprehensive feature map.

[0094] These weights are then element-wise multiplied with the original feature map to achieve spatial weighting:

[0095] F out =F⊙W spatial

[0096] Among them, ⊙ represents element-by-element multiplication, F out is the output feature map of the spatial attention mechanism, F out ∈R C×H×W .

[0097] Finally, the summing unit is used to add the output feature map F of the channel attention mechanism channel And the output feature map F of the spatial attention mechanism out The fusion is performed by element-wise addition:

[0098] F enchanted =F channel +F out

[0099] Among them, F enchanted To enhance the feature map, F enchanted ∈R C×H×W , the feature map integrates and amplifies relevant features.

[0100] The enhanced feature map is reduced in dimension and integrated through a 1×1 convolutional layer to generate the final output feature map:

[0101] F final =σ(K final *F channel +b final )

[0102] Among them, K final ∈R C′×H×W is the final output feature map, C′ represents the number of channels after reduction, K final is the convolution kernel, b final is the bias term and σ is the activation function.

[0103] The multi-scale atrous convolution feature extraction module is able to extract features at multiple spatial scales by introducing multi-scale atrous convolutions. This approach has two significant advantages. First, it effectively expands the receptive field: Traditional convolutional networks expand the receptive field by stacking multiple convolutional layers. However, this approach often leads to a sharp increase in computational cost and may cause information dilution. In contrast, the use of dilated convolutions, especially dilated convolutions with different dilation rates, can significantly expand the receptive field without losing resolution, which is crucial for capturing a wider range of contextual information in the image. Second, it is able to capture multi-scale information: In image processing tasks, target objects may appear in various sizes and shapes. By simultaneously utilizing multiple atrous rates, the network is able to capture features at different scales, enhancing the model's adaptability and recognition capabilities for features of various scales within the image.

[0104] Example 3

[0105] Based on the above embodiments, Figure 4 As shown in FIG, the present invention proposes a structure of a global channel-spatial attention module, which specifically includes:

[0106] To enhance interdependence between channels, the global channel-spatial attention module constructed in this paper is used to extract features by element-by-element addition of the output of the multi-scale dilated convolutional feature extraction module and the output of the first layer normalization. The global channel-spatial attention module includes a channel attention submodule, a channel shuffling submodule, a spatial attention submodule, and a connection submodule.

[0107] The channel attention submodule uses a multi-layer perceptron (MLP) to implement channel attention, which can effectively reduce the redundancy between channels and highlight important features. First, the input feature map F′∈R of the global channel-spatial attention module is (H ×W×C) The dimensions of the feature map (the output of the multi-scale dilated convolutional feature extraction module and the normalized output of the first layer are added element-by-element) are rearranged so that the channel dimension is at the last dimension.

[0108] F rearranged =Rearrange(F′)∈R H×W×C

[0109] Among them, F rearranged is the rearranged feature map, and Rearrange is the dimension rearrangement operation.

[0110] Next, the data is processed using a two-layer multi-layer perceptron (MLP). The first layer reduces the number of channels to a quarter of their original size and introduces nonlinearity through the ReLU activation function:

[0111] FMLP1 =ReLU(W1*F rearranged +b1)

[0112] in, is the weight matrix of the first layer, b1 is the bias term, F MLP1 is the output of the first layer of multi-layer perceptron.

[0113] The second layer restores the number of channels to the original dimension:

[0114] F MLP2 =W2*F MLP1 +b2

[0115] in, is the weight matrix of the second layer, b2 is the bias term, F MLP2 is the output of the second layer of multi-layer perceptron. This two-layer MLP approach helps to better capture the global dependencies between channels.

[0116] Finally, perform the inverse shuffle to restore the original dimensions:

[0117] F final =InverseRearrange(F MLP2 )∈R H×W×C

[0118] Among them, F final It is the feature map after inverse rearrangement, and InverseRearrange is the inverse rearrangement operation.

[0119] In order to further mix and share information, the channel shuffling submodule divides the features of the output of the channel attention submodule and the input feature map of the global channel-spatial attention module into 4 groups, each group contains Channels. Transpose the grouped feature maps to disrupt the order of channels within each group. Then, restore the disrupted feature maps to their original shape, C×H×W. This approach better blends feature information and enhances feature expression.

[0120] In the spatial attention submodule, the feature map output by the channel shuffling submodule passes through a 7x7 convolution layer, and the number of channels is reduced to the original times. Then, batch normalization and ReLU activation are used for nonlinear transformation. Next, a second 7x7 convolutional layer restores the number of channels to the dimension C of the feature map output by the channel shuffling submodule, followed by another batch normalization layer. Finally, a sigmoid activation function is used to generate a spatial attention map.

[0121] The connection submodule is used to perform element-wise multiplication of the output of the channel shuffling submodule and the output of the spatial attention submodule to obtain the output feature map of the global channel-spatial attention module. The output feature map of the global channel-spatial attention module contains the enhanced features after channel attention, channel shuffling, and spatial attention.

[0122] Example 4

[0123] Based on the above embodiments, Figure 5 and Figure 6 As shown in FIG, the present invention proposes a process of drawing instance boxes during visual transformer model training, which specifically includes:

[0124] During the application phase, the excellent segmentation capabilities of SAM (Segment Anything Model) are fully utilized. Specifically, the target outline is first sampled by manually drawing lines. These sampled points then guide SAM for precise segmentation, generating multiple mask proposal boxes. Next, only proposal boxes containing all sampled points from the same curve are retained. Within these proposal boxes, all regions except the confirmed mask region are set to zero to ensure that the example boxes are free of background noise. This approach not only accurately locates the target but also clearly separates the background, greatly improving the quality and efficiency of subsequent feature extraction.

[0125] In addition, this method is flexible in design and will not conflict with the existing network model structure, and can be used as a plug-and-play data enhancement method. This innovative selection method changes the traditional few-sample example box selection method, allowing users to easily and quickly obtain example boxes without background interference and with accurate positioning, effectively optimizing the overall processing flow of the model, such as Figure 5 shown.

[0126] Example 5

[0127] Based on the above embodiments, the present invention proposes an evaluation method for a few-shot learning object counting method that integrates multi-scale detection and example box selection, specifically including:

[0128] To validate the effectiveness of this invention, we conducted extensive experiments on the FSC147 dataset. We drew on the work of other scholars in this field and used mean absolute error (MAE) and root mean square error (MSE) as evaluation metrics. MAE and RMSE are two commonly used metrics for evaluating the performance of regression models. They provide insight into the accuracy of the model's predicted values ​​compared to the actual values.

[0129] The mean absolute error (MAE) is the average of the absolute differences between the predicted values ​​and the actual values ​​and is defined as follows:

[0130]

[0131] The root mean square error (RMSE) is the square root of the average of the squared differences between the predicted and actual values. It is defined as:

[0132]

[0133] The experimental environment is shown in Table 1.

[0134] Table 1 Experimental environment

[0135]

[0136] The performance indicators of several existing target counting methods and the method proposed in this paper are compared in a 3-shot setting on the FSC147 dataset. The results are shown in Table 2.

[0137] Table 2 Comparison of performance indicators of various target counting methods

[0138]

[0139] In the 3-shot scenario, the proposed method shows significant advantages in both the mean absolute error (MAE) and root mean square error (RMSE) on the validation set and the test set.

[0140] Specifically, the proposed method achieved a MAE of 10.32 and an RMSE of 29.32 on the validation set, while CACViT achieved a MAE of 10.63 and an RMSE of 37.95 on the validation set. The proposed method reduced the MAE by 0.31, an improvement of approximately 2.91%. In terms of RMSE, the proposed method reduced it by 8.63, an improvement of approximately 22.79%.

[0141] On the test set, the proposed method achieves a MAE of 8.92 and an RMSE of 31.26. Compared to CACViT's MAE of 10.63, the proposed method reduces the MAE by 1.71, an improvement of approximately 16.09%. In terms of RMSE, the proposed method reduces the MAE by 6.69, an improvement of approximately 17.68%.

[0142] Overall, the results show that the present invention exhibits superior performance in handling this task, and the accuracy and effectiveness of predictions are significantly improved.

[0143] In summary, the present invention designs a multi-scale dilated convolution feature extraction module and a global channel-spatial attention module to improve the performance of the visual transformer model in detecting objects of varying scales, targeting scenarios with varying scales. Specifically, the multi-scale dilated convolution feature extraction module sets different convolution kernel sampling rates, thereby enhancing the model's adaptability to objects of varying sizes and effectively addressing the challenges posed by scale variations in images. The global channel-spatial attention module further improves the effective utilization of feature information by enhancing the model's global understanding and representation of the input feature map. The combination of these two effectively addresses the issue of scale variation within images, significantly improving the accuracy and robustness of object detection. The present invention proposes a method for drawing example boxes that not only accurately locates the object but also clearly separates the background, significantly improving the quality and efficiency of subsequent feature extraction. Furthermore, this method is flexible and does not conflict with existing network model structures, making it a plug-and-play data augmentation method. This innovative selection method transforms traditional methods for selecting example boxes with few samples, enabling users to quickly and easily obtain accurately located example boxes free of background interference, effectively optimizing the overall model processing flow.

[0144] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present invention.

Claims

1. A few-shot learning object counting method that integrates multi-scale detection and example box selection, characterized by: include: Step 1: Preprocess the query image and the sample image with scale information and then stitch them together to obtain a stitched image; Step 2: Input the spliced ​​image into the visual transformer model to obtain the query image features; Step 3: Input the query image features into the regression decoder to complete the object counting in the query image.

2. The object counting method for few-shot learning integrating multi-scale detection and example box selection according to claim 1, characterized in that: The step 1 specifically includes: The query image and the sample image with scale information are divided into multiple small blocks, and the multiple small blocks are converted into tokens; The tokens are flattened and then stitched together to obtain a stitched image.

3. The object counting method for few-shot learning integrating multi-scale detection and example box selection according to claim 1, characterized in that: The visual transformer model includes a plurality of transformer units and reshaping units connected in sequence, wherein the transformer unit includes a multi-head attention, a first layer normalization, a multi-layer perceptron, a multi-scale dilated convolution feature extraction module, a global channel-spatial attention module and a second layer normalization; The multi-head attention is used to process the spliced ​​images to obtain a similarity measure; The first layer normalization is used to perform layer normalization processing on the features after element-by-element addition of the output of the multi-head attention and the spliced ​​image; The multi-layer perceptron is used to process the normalized output of the first layer to extract important features; The multi-scale dilated convolution feature extraction module is used to extract features from the output of the multi-layer perceptron; The global channel-spatial attention module is used to extract features after element-by-element addition of the output of the multi-scale dilated convolution feature extraction module and the output of the first layer normalization; The second layer normalization is used to perform layer normalization processing on the output of the global channel-spatial attention module; The reshaping unit is used to splice the output of the last transformer unit and the output of the multi-head attention in the first transformer unit, and then reshape the spliced ​​output into image blocks.

4. The object counting method for few-shot learning integrating multi-scale detection and example box selection according to claim 3, characterized in that: The multi-scale dilated convolution feature extraction module includes multiple parallel convolution branches, splicing layers, spatial attention mechanisms, channel attention mechanisms, and addition units; The multiple parallel convolution branches are used to extract features of different scales input by the multi-scale dilated convolution feature extraction module; The splicing layer is used to splice the outputs of multiple parallel convolution branches to obtain a comprehensive feature map; The spatial attention mechanism is used to extract features from the comprehensive feature map to obtain an output feature map of the spatial attention mechanism; The channel attention mechanism is used to extract features from the comprehensive feature map to obtain an output feature map of the channel attention mechanism; The adding unit is used to add the output feature map of the spatial attention mechanism and the output feature map of the channel attention mechanism element by element, and input the added feature map into the convolution layer for dimensionality reduction and integration to obtain the final output feature map.

5. The object counting method for few-shot learning integrating multi-scale detection and example box selection according to claim 4, characterized in that: The channel attention mechanism is used to extract features from the comprehensive feature map to obtain the output feature map of the channel attention mechanism, specifically including: Perform global average pooling on the comprehensive feature map to obtain the global features of each channel; A fully connected layer that sequentially connects two layers of global features of each channel; the first fully connected layer includes a ReLU activation function, and the second fully connected layer includes a Sigmoid activation function; The features processed by the two fully connected layers are multiplied with the comprehensive feature map to obtain the output feature map of the channel attention mechanism.

6. The object counting method for few-shot learning integrating multi-scale detection and example box selection according to claim 4, characterized in that: The spatial attention mechanism is used to extract features from the comprehensive feature map to obtain the output feature map of the spatial attention mechanism, specifically including: Perform global pooling operation on the comprehensive feature map to obtain the spatial feature map; The importance weight of each spatial position is calculated based on a 1×1 convolutional layer with a sigmoid activation function; The importance weight of each spatial position is multiplied element-wise with the spatial feature map to obtain the output feature map of the spatial attention mechanism.

7. The object counting method for few-shot learning integrating multi-scale detection and example box selection according to claim 3, characterized in that: The global channel-spatial attention module includes a channel attention submodule, a channel shuffling submodule, a spatial attention submodule and a connection submodule; The channel attention submodule is used to perform dimensionality transformation on the input feature map of the global channel-spatial attention module; The channel shuffling submodule divides the features obtained by element-wise multiplication of the output of the channel attention submodule and the input feature map of the global channel-spatial attention module into multiple groups, performs a transposition operation on the grouped feature maps, and finally restores the transposed feature maps to their original shape; The spatial attention submodule is used to perform multiple convolution operations on the output of the channel shuffling submodule; The connection submodule is used to perform element-by-element multiplication of the output of the channel shuffling submodule and the output of the spatial attention submodule.

8. The object counting method for few-shot learning integrating multi-scale detection and example box selection according to claim 7, characterized in that: The channel attention submodule is used to perform dimensionality transformation on the input feature map of the global channel-spatial attention module, specifically including: Rearrange the dimensions of the input feature map of the global channel-spatial attention module so that the channel dimension is at the end; The feature map after dimension rearrangement is input into two layers of multi-layer perceptrons connected sequentially for processing; The feature map processed by the two-layer multi-layer perceptron is reversely rearranged to restore the dimension of the feature map to the dimension before dimension rearrangement.

9. The object counting method for few-shot learning integrating multi-scale detection and example box selection according to claim 7, characterized in that: The spatial attention submodule is used to perform multiple convolution operations on the output of the channel shuffling submodule, specifically including: The output of the channel shuffling submodule is input into the first 7x7 convolutional layer, and the output of the first 7x7 convolutional layer is batch normalized and ReLU activated. The processed feature map is input into the second 7x7 convolutional layer, and the output of the second 7x7 convolutional layer is batch normalized and processed with the Sigmoid activation function.

10. The object counting method for few-shot learning integrating multi-scale detection and example box selection according to claim 1, characterized in that: The visual transformer model also needs to be trained. During training, instance boxes need to be drawn. The instance box drawing process specifically includes: The target contour is sampled by manually drawing lines, and accurate segmentation is performed based on the sampling points and the segmentation model to generate multiple mask proposal boxes; The mask proposals from the same curve sampling point are retained, and all areas outside the mask area in the mask proposal are set to zero to complete the example box construction.

Images