Few-sample target counting method based on context awareness feature enhancement

Abstract

The invention discloses a few-sample target counting method based on context perception feature enhancement, and belongs to the field of computer vision, and the method comprises the steps: inputting a query image into a feature extractor composed of CNN, and extracting the deep and shallow features of the image; the hierarchical perception joint enhancement module fuses the multi-scale features of the query image and the shape and appearance features of the visual sample; a learnable similarity matcher dynamically obtains the correlation between a query image and a visual sample, expands a receptive field and enhances the feature capturing capability of the model on different scales; the feature fusion module is used for further fusing the example features weighted by the similarity graph and the multi-scale features of the query image; the feature map output by the feature fusion module is sent into a regression head to generate a density map; and adding the density map pixel by pixel to obtain an interested target counting result in the query image. According to the method, the counting problem caused by insufficient extraction of visual sample shape information and background noise interference is solved.

Description

Technical Field

[0001] The present invention relates to the field of computer vision, and in particular to a few-sample target counting method based on context-aware feature enhancement. Background Art

[0002] With the development of computer vision technology, especially driven by deep learning, the accuracy of target detection and recognition has been greatly improved. However, in some scenarios, such as counting people in public places or counting products on a production line, it is not enough to simply locate the target, and accurate counting is also required. The target counting task aims to count the number of specific targets in the query image. Although high accuracy has been achieved in counting specific categories such as people, cells, and cars, existing methods have difficulty transferring learned knowledge to new categories that have not been seen, which limits their practical application range. Therefore, the research focus has shifted to developing methods that can learn from a small number of samples and generalize to new categories.

[0003] Few-shot object counting uses a query image and several visual example boxes to count objects of any category, thereby reducing the dependence on a large amount of training data and showing good generalization ability in different category counting tasks. At present, the most studied methods are based on density estimation. These methods are usually based on similarity. By calculating the similarity between the query image and the visual sample and taking it as the regression object, a density map is obtained, and finally the density map is added pixel by pixel to obtain the number of objects of interest in the query image. However, this direct similarity calculation method often lacks the fusion of original features, which makes it difficult for the network to identify clear contours between objects in dense scenes. In addition, during the extraction of examples, the RoI Align operation aggregates features onto a fixed-size correlation filter, resulting in the shape and position information of the object being ignored. On the other hand, most of the existing similarity metrics rely on fixed similarity rules, resulting in the loss of spatial information or limiting the matching of objects with significant intra-class differences, which in turn affects the counting accuracy of the model in complex scenes. Summary of the invention

[0004] The technical problem to be solved by the present invention is to provide a few-sample target counting method based on context-aware feature enhancement. By fusing and enhancing the multi-scale features of the query image and the shape and other information of the visual sample, a learnable similarity matcher is introduced to calculate the similarity between the query image and the visual sample, and the multi-scale features of the query image are further fused through the example features weighted by the similarity, so as to achieve the purpose of counting the targets of interest in any query image, and solve the counting problem caused by insufficient extraction of the shape information of the visual sample and background noise interference.

[0005] In order to solve the above technical problems, the technical solution adopted by the present invention is:

[0006] A few-sample target counting method based on context-aware feature enhancement includes the following steps:

[0007] Step 1: Input the query image into the feature extractor composed of CNN to extract the deep and shallow feature maps of the query image;

[0008] Step 2: The hierarchical perception joint enhancement module performs scale interaction on the features of the last three stages of the query image obtained by the feature extractor, fully integrates the deep and shallow features of the query image, and then performs Φ mapping and RoI Align operations on the query image according to the sample information to obtain the shape and appearance features of the visual sample. The sample feature enhancement module integrates the appearance and shape features of the visual sample and the multi-scale features of the query image to obtain the sample features.

[0009] Step 3: The learnable similarity matcher dynamically obtains the correlation between the query image and the visual sample, while expanding the receptive field and enhancing the model's ability to capture features at different scales to obtain a similarity graph.

[0010] Step 4: The feature fusion module performs a convolution operation with a given kernel on the similarity map obtained in step 3 to obtain similarity-weighted example features, and further fuses them with the multi-scale features of the query image in step 2 to obtain the final enhanced feature map;

[0011] Step 5: Send the enhanced feature map output from step 4 to the regression head. After a series of processing, the predicted density map is obtained by regression.

[0012] Step 6: Add the resulting predicted density maps pixel by pixel to get the total number of predicted objects of interest in the query image.

[0013] A further improvement of the technical solution of the present invention is that in the feature extraction stage, the query image is subjected to multi-scale feature extraction by a feature extractor, and multi-scale features are extracted by the second layer, the third layer and the fourth layer respectively, and the convolution layer of each network layer includes three convolution operations, and the convolution kernels are 1×1, 3×3 and 1×1 respectively.

[0014] A further improvement of the technical solution of the present invention is that in step 2, the multi-scale fusion of the query image is performed from deep features to shallow features through operations such as upsampling, channel connection, and 3×3 convolution.

[0015] A further improvement of the technical solution of the present invention is that in step 2, in order to obtain a multi-scale feature map of the query image and an example feature map that fuses the shape and appearance of the visual sample, the few-sample target counting model based on context-aware feature enhancement adopts a hierarchical perception joint enhancement module to fuse the multi-scale features of the query image and the shape and appearance information of the visual sample to enhance the feature expression, which specifically includes the following contents:

[0016] S2.1, the second, third and fourth layers of query features {F1, F2, F3} obtained by the feature extractor are subjected to a series of convolution, upsampling and channel connection operations to perform multi-scale feature fusion, and then the multi-scale features of the query image are obtained by 1×1 convolution, denoted as F q ;

[0017] S2.2, according to the example box information, a mapping Φ is used to obtain the example shape features and a RoIAlign is used to obtain the example appearance features, denoted as E s and E a ;

[0018] S2.3, the example shape features, appearance features and multi-scale features of the query image obtained in S2.2 are input into the example feature enhancement module; the example feature enhancement module iteratively uses two-level multi-head attention to fuse the appearance and shape of the visual sample, as well as the query image features and the location information used for enhancement, and finally obtains the example features after feature enhancement.

[0019] A further improvement of the technical solution of the present invention is that in S2.3, the example feature enhancement module enhances the feature expression by fusing the shape and appearance information of the visual sample. The specific process is as follows:

[0020] S2.3.1, the shape and appearance of the example are encoded separately, so that the network can learn the spatial distribution of features and enhance the network's understanding of feature locations, which is particularly useful for understanding the location of objects in images and processing dense or small-sized objects in images;

[0021] S2.3.2, the example shape feature E s Q, an example appearance feature E, is considered as a multi-head attention mechanism a K and V, which are regarded as multi-head attention mechanisms, undergo a multi-head attention, a feedforward neural network, and two element-by-element addition operations to obtain the intermediate feature F e ”1; The feedforward neural network consists of two linear layers, a Dropout layer, and a ReLU activation function;

[0022] F e '1=LN(MHA(E s +P E ,Ea +P E ,E a +P E )+E s ) (1)

[0023] F e ”1=LN(FF(F e '1)+F e '1) (2)

[0024] In the formula, E s is the shape feature of the visual sample; E a is the appearance feature of the visual sample; P E is position encoding; MHA is standard multi-head attention; LN is layer normalization; FF is feedforward neural network;

[0025] S2.3.3, similar to S2.3.2, the intermediate feature F e ” 1 is regarded as Q of the multi-head attention mechanism, and the multi-scale features F of the query image q K and V, which are regarded as multi-head attention mechanisms, undergo a multi-head attention, a feedforward neural network, and two element-by-element addition operations. The resulting feature is recorded as F e ”2;

[0026] F e '2=LN(MHA(F e ”1,F q +P E ,F q +P E )+F e ”1) (3)

[0027] F e ”2=LN(FF(F e '2)+F e 'twenty four)

[0028] S2.3.4, the F obtained in S2.3.3 e ”2After multi-layer perceptron and element-by-element addition, the final feature tensor F is obtained e ;

[0029] F e =MLP(F e ”2)+F e ”2 (5)

[0030] Where MLP is a multi-layer perceptron;

[0031] The above process can be iterated N times, and the final output is

[0032] The further improvement of the technical solution of the present invention is that in step 3, the fixed inner product similarity metric is improved, and learnable operations such as dilated convolution and linear layer are introduced to more flexibly and accurately obtain the similarity between the query image and the visual sample, which specifically includes the following contents:

[0033] S3.1, process the query features, query the multi-scale features F of the image q First, convolution operations with different expansion rates r=1, r=2, and r=3 are performed to expand the receptive field and capture information of different scales. Then, the spatial distribution of features is adjusted through 1×1 convolution and linear layer processing, making the features more suitable for the next similarity calculation.

[0034] S3.2, the obtained visual sample features F e Through the linear layer and the bilinear module composed of two linear layers and the ReLU activation function, the spatial distribution of the features is changed to make them more suitable for subsequent similarity matching; the addition of the ReLU activation function introduces nonlinear factors, enhances the expressiveness of the features, and thus improves the matching quality between features;

[0035] S3.3, processed query features F q ' and visual sample features F e 'The similarity graph is obtained by matrix multiplication; specifically, F q ' and F e The matrix product operation of ' can calculate the correlation between the query features and the example features, and the generated similarity graph reflects the degree of similarity between the two.

[0036] A further improvement of the technical solution of the present invention is that in step 4, the feature fusion module effectively fuses feature information from different sources, that is, a series of element-by-element addition, 3×3 convolution and layer normalization operations are performed on the multi-scale features of the query image obtained by the feature extractor and the example features weighted by the similarity weights, thereby improving the robustness and accuracy of the model in processing small-sample target counting tasks.

[0037] Due to the adoption of the above technical solution, the technical progress achieved by the present invention is:

[0038] 1. The present invention proposes a novel few-sample target counting method based on context-aware feature enhancement, which can enhance and fuse features, use similarity as a weighting coefficient, and integrate visual sample features into query features, thereby achieving excellent counting performance.

[0039] 2. The present invention proposes a hierarchical perception joint enhancement module, which performs feature enhancement on the multi-scale information of the query image and the shape, appearance and other information of the visual samples, effectively solving the problem of poor counting effect caused by insufficient information utilization in dense scenes.

[0040] 3. The present invention proposes a learnable similarity matcher, which expands the similarity metric of fixed inner product by adding learnable dilated convolution and linear layers, effectively alleviating the limitations of background interference and matching of objects with significant differences within the class. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art are briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative labor.

[0042] Figure 1 is a flow chart of a method for counting targets of a small number of samples based on context-aware feature enhancement provided in an embodiment of the present invention;

[0043] Figure 2 Schematic diagram of the overall structure of a few-sample target counting model based on context-aware feature enhancement in an embodiment of the present invention;

[0044] Figure 3 is a detailed schematic diagram of a hierarchical perception joint enhancement module in an embodiment of the present invention;

[0045] Figure 4 is a detailed schematic diagram of a learnable similarity matcher in an embodiment of the present invention;

[0046] Figure 5 It is a schematic diagram of a specific implementation of target counting using a few-sample target counting method based on context-aware feature enhancement provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0047] It should be noted that the terms "including" and "having" and any variations thereof in the specification and claims of the present invention and the above-mentioned drawings are intended to cover non-exclusive inclusions. For example, a process, method, system, product or apparatus comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or apparatus.

[0048] In the description of the present invention, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside" and the like indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the referred device or element must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be understood as limiting the present invention.

[0049] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of the present invention, the meaning of "several" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0050] The present invention is further described in detail below with reference to the accompanying drawings and embodiments:

[0051] like Figure 1 , 2 As shown in the figure, a few-shot target counting method based on context-aware feature enhancement is proposed, in which the core function of the few-shot target counting model based on context-aware feature enhancement is to train the counting model to learn the correlation between visual samples and counted objects given a small number of visual samples, thereby estimating the number of objects of any category in the query image. To achieve this goal, the model mainly integrates multiple components such as feature extractor, hierarchical perception joint enhancement module, learnable similarity matcher, feature enhancement module and regression head. The counting method specifically includes the following steps:

[0052] Step 1: Input the query image into the feature extractor composed of CNN to extract the deep and shallow feature maps of the query image;

[0053] In the feature extraction stage, the query image is first subjected to multi-scale feature extraction by the feature extractor, and multi-scale features are extracted through the second, third and fourth layers respectively, denoted as {F1, F2, F3}; the convolutional layer of each network layer includes three convolution operations, and the convolution kernels are 1×1, 3×3 and 1×1 respectively.

[0054] Step 2: The hierarchical perception joint enhancement module performs scale interaction on the features of the last three stages of the query image obtained by the feature extractor, fully integrates the deep and shallow features of the query image, and then performs Φ mapping and RoI Align operations on the query image according to the sample information to obtain the shape and appearance features of the visual sample. The sample feature enhancement module integrates the appearance and shape features of the visual sample and the multi-scale features of the query image to obtain the sample features.

[0055] Specifically, the layered perception joint enhancement module improves the model's perception of changes at different scales by fusing multi-scale features, and fuses query features of different scales through upsampling and channel splicing. Then, a 1x1 convolution operation is performed on the fused features to obtain enhanced query features. In addition, the shape and appearance information of the example is fused through the example feature enhancement module to provide a more comprehensive feature representation;

[0056] Combine the following Figure 3 The hierarchical perception joint enhancement module used in the present invention is described as follows:

[0057] like Figure 3 As shown in the figure, in order to obtain the multi-scale feature map of the query image and the sample feature map that integrates the shape and appearance of the visual sample, the few-shot target counting model based on context-aware feature enhancement adopts a hierarchical perception joint enhancement module to fuse the multi-scale features of the query image and the shape, appearance and other information of the visual sample to enhance the feature expression, which specifically includes the following contents:

[0058] S2.1, the second, third and fourth query features {F1, F2, F3} obtained by the feature extractor are subjected to a series of convolution, upsampling and channel connection operations to perform multi-scale feature fusion, and then the multi-scale features of the query image are obtained by 1×1 convolution, denoted as F q ;

[0059] S2.2, according to the example box information, a mapping Φ is used to obtain the example shape features and a RoIAlign is used to obtain the example appearance features, denoted as E s and E a ;

[0060] S2.3, the example shape features, appearance features and multi-scale features of the query image obtained in S2.2 are input into the example feature enhancement module. The example feature enhancement module iteratively uses two-level multi-head attention to fuse the appearance and shape of the visual sample, as well as the query image features and the location information used for enhancement, and finally obtains the example features after feature enhancement.

[0061] To facilitate understanding of the implementation process of the example feature enhancement module, a detailed description is given below:

[0062] S2.3.1, the shape and appearance of the example are encoded separately, so that the network can learn the spatial distribution of features and enhance the network's understanding of feature locations. This is particularly useful for understanding the location of objects in an image and processing dense or small-sized objects in an image;

[0063] S2.3.2, the example shape feature E s Q, an example appearance feature E, is considered as a multi-head attention mechanism a K and V, which are regarded as multi-head attention mechanisms, undergo a multi-head attention, a feedforward neural network, and two element-by-element addition operations to obtain the intermediate feature F e ”1. The feedforward neural network includes two linear layers, a Dropout layer, and a ReLU activation function;

[0064] F e '1=LN(MHA(E s +P E ,E a +P E ,E a +P E )+E s ) (1)

[0065] F e ”1=LN(FF(F e '1)+F e '1) (2)

[0066] In the formula, E s is the shape feature of the visual sample; E a is the appearance feature of the visual sample; P E is position encoding; MHA is standard multi-head attention; LN is layer normalization; FF is feedforward neural network;

[0067] S2.3.3, similar to S2.3.2, the intermediate feature F e ” 1 is regarded as Q of the multi-head attention mechanism, and the multi-scale features F of the query image q K and V, which are regarded as multi-head attention mechanisms, undergo a multi-head attention, a feedforward neural network, and two element-by-element addition operations. The resulting feature is recorded as F e ”2;

[0068] F e '2=LN(MHA(F e ”1,F q +P E ,F q +P E )+F e ”1) (3)

[0069] F e ”2=LN(FF(F e '2)+F e 'twenty four)

[0070] S2.3.4, the F obtained in S2.3.3 e ”2After multi-layer perceptron and element-by-element addition, the final feature tensor F is obtained e :

[0071] F e =MLP(F e ”2)+F e ”2 (5)

[0072] Where MLP is a multi-layer perceptron.

[0073] The above process can be iterated N times, and the final output is

[0074] Step 3: The learnable similarity matcher dynamically obtains the correlation between the query image and the visual sample, while expanding the receptive field and enhancing the model's ability to capture features at different scales to obtain a similarity graph.

[0075] Specifically, the enhanced example feature F output in step 2 is e and the multi-scale features F of the query image q The two are input into a learnable similarity matcher, which extends the fixed inner product similarity metric by introducing learnable dilated convolution and linear layers, so as to obtain the similarity between the query image and the visual sample more flexibly and accurately.

[0076] Combine the following Figure 4 The learnable similarity matcher used in the present invention is described as follows:

[0077] like Figure 4 As shown, in order to obtain the correlation between the query image and the visual sample, the present invention adopts a learnable similarity matcher, which specifically includes the following contents:

[0078] S3.1, process the query features, query the multi-scale features F of the image q First, convolution operations with different expansion rates r=1, r=2, and r=3 are performed to expand the receptive field and capture information of different scales. Then, the spatial distribution of features is adjusted through 1×1 convolution and linear layer processing, making the features more suitable for the next similarity calculation.

[0079] S3.2, the visual sample features F obtained in S2.3.4 eThrough the linear layer and the bilinear module composed of two linear layers and the ReLU activation function, the spatial distribution of the features is changed to make them more suitable for subsequent similarity matching. The addition of the ReLU activation function introduces nonlinear factors, enhances the expressiveness of the features, and thus improves the matching quality between features;

[0080] S3.3, processed query features F q ' and visual sample features F e 'The similarity graph is obtained by matrix multiplication. Specifically, F q ' and F e The matrix product operation of ' can calculate the correlation between the query features and the example features, and the generated similarity graph reflects the degree of similarity between the two.

[0081] Step 4: Perform a convolution operation with a given kernel on the similarity graph obtained in step 3 to obtain the similarity-weighted sample features, and compare them with the multi-scale features F of the query image in step 2. q Further fusion is performed to obtain the final enhanced feature map;

[0082] Combine the following Figure 2 The feature fusion module used in the present invention is further described as follows:

[0083] like Figure 2 As shown in the figure, the feature fusion module effectively fuses feature information from different sources, improving the robustness and accuracy of the model in processing object counting tasks, including the following:

[0084] S4.1, based on the similarity graph obtained by the learnable similarity matcher, the enhanced example features are used as convolution kernels to perform convolution operations on n similarity graphs, thereby obtaining similarity weighted features;

[0085] S4.2, the similarity weighted features obtained in the previous step S4.1 are combined with the query features F obtained by multi-scale fusion in step 2 q Specifically, the similarity weighted feature map and the multi-scale query feature map are added element by element to obtain the feature map F r ;

[0086] S4.3, the feature map F r and the query feature F obtained by scale fusion q Perform element-by-element summation to obtain the enhanced feature map F r ';

[0087] S4.4, Enhanced feature map F r 'Through a 3×3 convolution kernel and the introduction of a residual connection structure, the final enhanced feature F is achieved q'. This step is to further enhance the representation ability of features, so that the model has stronger ability to deal with object recognition in dense scenes.

[0088] Step 5: The final enhanced feature F output from step 4 q 'Send it to the regression head, and after a series of processing, regress to get the predicted density map;

[0089] Specifically, in the regression head, the feature map is upsampled three times and gradually restored to a feature map close to the original size. Finally, a 1×1 convolution layer is used to generate a single-channel density map, and the LeakyReLU activation function is used. The main function of the regression head is to gradually restore the feature representation close to the original image resolution from the high-level abstract feature map and generate a single-channel density map, which can be used to estimate the distribution density of the target object in the image, and then infer the number of objects.

[0090] Step 6: Add the resulting predicted density maps pixel by pixel to get the total number of predicted objects of interest in the query image.

[0091] Specifically, after the density map is predicted, each pixel in the density map represents the density of the target at the corresponding position in the image. By accumulating all pixel values ​​on the entire density map, a value can be obtained, which is an estimate of the number of all interesting targets in the query image.

[0092] Figure 5 This paper demonstrates an application example of a few-shot object counting model based on context-aware feature enhancement. Figure 5 For example, in practical applications, a group of photos of fish schools can be seen, one part of which provides three samples of fish of a specific species, and the other part is the fish school to be counted. By using the method of the present invention, the model can quickly and accurately identify and count each fish in the fish school, even if the annotation information provided for the counting object is very limited. Specifically, the model analyzes the query image and several visual samples provided, and extracts key contextual features from them, including information such as the shape, color, and texture of the fish. These features are then used to guide the counting process of the fish school photos. In the photos of the fish school, the counting model uses these features to identify the fish, thereby achieving accurate counting. Because the contextual information is taken into account, this method can better handle counting tasks in complex environments, such as occlusion, overlap, or background interference that may exist in the fish school, thereby achieving efficient and accurate counting performance.

[0093] In summary, the present invention aims to solve the problems of huge cost of training data sample annotation for specific category counting, low effect of counting transferability, and insufficient utilization of information such as the difficulty of the network in recognizing clear contours between objects and visual sample shapes in dense scenarios for existing few-sample target counting. A few-sample target counting method based on context-aware feature enhancement is proposed, which realizes efficient counting in any category with a small amount of annotation information. The query image is input into the feature extractor composed of CNN to extract the deep and shallow feature maps of the query image. The features extracted in the last three stages are scaled interactively to fully integrate the multi-scale features of the query image. Then, the query image is Φ-mapped and RoI Aligned according to the sample box information to obtain the shape and appearance features of the visual sample. The sample feature enhancement module fuses the appearance, shape features of the visual sample and the multi-scale features of the query image to obtain the enhanced sample features. Then, the correlation between the query image and the visual sample is dynamically obtained through a learnable similarity matcher, while the receptive field is expanded and the model's ability to capture features at different scales is enhanced to obtain a similarity map. On this basis, the obtained similarity map is convolved with a given kernel to obtain a similarity weighted feature, which is further fused with the multi-scale features obtained by the feature extractor to obtain the final enhanced feature map. Next, it is sent to the regression head and regressed to obtain a predicted density map after a series of processing. Finally, the obtained predicted density map is added pixel by pixel to obtain the predicted total number of objects of interest in the query image. Through the above description, this method can accurately estimate the number of objects in any category of query images with fewer example annotations.

[0094] Through effective feature enhancement and similarity matching strategies, the present invention has demonstrated excellent performance in the field of small-sample target counting, indicating that it will be expected to promote the development of target counting technology in a wider range of categories and more complex scenarios in future research.

[0095] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A few-shot object counting method based on context-aware feature enhancement, characterized in that: The steps include: Step 1: Input the query image into the feature extractor composed of CNN to extract the deep and shallow feature maps of the query image; Step 2: The hierarchical perception joint enhancement module performs scale interaction on the features of the last three stages of the query image obtained by the feature extractor, fully integrates the deep and shallow features of the query image, and then performs Φ mapping and RoI Align operations on the query image according to the sample information to obtain the shape and appearance features of the visual sample. The sample feature enhancement module integrates the appearance and shape features of the visual sample and the multi-scale features of the query image to obtain the sample features. Step 3: The learnable similarity matcher dynamically obtains the correlation between the query image and the visual sample, while expanding the receptive field and enhancing the model's ability to capture features at different scales to obtain a similarity graph. Step 4: The feature fusion module performs a convolution operation with a given kernel on the similarity map obtained in step 3 to obtain similarity-weighted example features, and further fuses them with the multi-scale features of the query image in step 2 to obtain the final enhanced feature map; Step 5: Send the enhanced feature map output from step 4 to the regression head. After a series of processing, the predicted density map is obtained by regression. Step 6: Add the resulting predicted density maps pixel by pixel to get the total number of predicted objects of interest in the query image.

2. The method for counting targets with a small number of samples based on context-aware feature enhancement according to claim 1, characterized in that: In the feature extraction stage, the query image is subjected to multi-scale feature extraction by the feature extractor, and multi-scale features are extracted through the second, third and fourth layers respectively. The convolution layer of each network includes three convolution operations, and the convolution kernels are 1×1, 3×3 and 1×1 respectively.

3. The method for counting targets with a small number of samples based on context-aware feature enhancement according to claim 1, characterized in that: In step 2, the multi-scale fusion of the query image is performed from deep features to shallow features through operations such as upsampling, channel connection, and 3×3 convolution.

4. The method for counting targets with a small number of samples based on context-aware feature enhancement according to claim 1, characterized in that: In step 2, in order to obtain the multi-scale feature map of the query image and the sample feature map that integrates the shape and appearance of the visual sample, the few-shot object counting model based on context-aware feature enhancement adopts a hierarchical perception joint enhancement module to fuse the multi-scale features of the query image and the shape and appearance information of the visual sample to enhance the feature expression, which specifically includes the following contents: S2.1, the second, third and fourth layers of query features {F1, F2, F3} obtained by the feature extractor are subjected to a series of convolution, upsampling and channel connection operations to perform multi-scale feature fusion, and then the multi-scale features of the query image are obtained by 1×1 convolution, denoted as F q ; S2.2, according to the example box information, a mapping Φ is used to obtain the example shape features and a RoIAlign is used to obtain the example appearance features, denoted as E s and E a ; S2.3, the example shape features, appearance features and multi-scale features of the query image obtained in S2.2 are input into the example feature enhancement module; the example feature enhancement module iteratively uses two-level multi-head attention to fuse the appearance and shape of the visual sample, as well as the query image features and the location information used for enhancement, and finally obtains the example features after feature enhancement.

5. The method for counting targets with a small number of samples based on context-aware feature enhancement according to claim 4, characterized in that: In S2.3, the sample feature enhancement module fuses the shape and appearance information of the visual sample to enhance the feature expression. The specific process is as follows: S2.3.1, the shape and appearance of the example are encoded separately, so that the network can learn the spatial distribution of features and enhance the network's understanding of feature locations, which is particularly useful for understanding the location of objects in images and processing dense or small-sized objects in images; S2.3.2, the example shape feature E s Q, an example appearance feature E, is considered as a multi-head attention mechanism a K and V, which are regarded as multi-head attention mechanisms, undergo a multi-head attention, a feedforward neural network, and two element-by-element addition operations to obtain the intermediate feature F e ”1; The feedforward neural network consists of two linear layers, a Dropout layer, and a ReLU activation function; F e '1=LN(MHA(E s +P E ,HAVE BEEN a +P E ,HAVE BEEN a +P E )+E s ) (1) F e ”1=LN(FF(F e '1)+F e '1) (2) In the formula, E s is the shape feature of the visual sample; E a is the appearance feature of the visual sample; P E is position encoding; MHA is standard multi-head attention; LN is layer normalization; FF is feedforward neural network; S2.3.3, similar to S2.3.2, the intermediate feature F e ” 1 is regarded as Q of the multi-head attention mechanism, and the multi-scale features F of the query image q K and V, which are regarded as multi-head attention mechanisms, undergo a multi-head attention, a feedforward neural network, and two element-by-element addition operations. The resulting feature is recorded as F e ”2; F e '2=LN(MHA(F e ”1,F q +P E ,F q +P E )+F e ”1) (3) F e ”2=LN(FF(F e '2)+F e '2) (4) S2.3.4, the F obtained in S2.3.3 e ”2After multi-layer perceptron and element-by-element addition, the final feature tensor F is obtained e ; F e =MLP(F e ”2)+F e ”2 (5) Where MLP is a multi-layer perceptron; The above process can be iterated N times, and the final output is 6. The method for counting targets with a small number of samples based on context-aware feature enhancement according to claim 1, characterized in that: In step 3, the fixed inner product similarity metric is improved by introducing learnable dilated convolution and linear layer operations to obtain the similarity between the query image and the visual sample more flexibly and accurately, including the following: S3.1, process the query features, query the multi-scale features F of the image q First, convolution operations with different expansion rates r=1, r=2, and r=3 are performed to expand the receptive field and capture information of different scales. Then, the spatial distribution of features is adjusted through 1×1 convolution and linear layer processing, making the features more suitable for the next similarity calculation. S3.2, the obtained visual sample features F e Through the linear layer and the bilinear module composed of two linear layers and the ReLU activation function, the spatial distribution of the features is changed to make them more suitable for subsequent similarity matching; the addition of the ReLU activation function introduces nonlinear factors, enhances the expressiveness of the features, and thus improves the matching quality between features; S3.3, processed query features F q ' and visual sample features F e 'The similarity graph is obtained by matrix multiplication; specifically, F q ' and F e The matrix product operation of ' can calculate the correlation between the query features and the example features, and the generated similarity graph reflects the degree of similarity between the two.

7. The method for counting targets with a small number of samples based on context-aware feature enhancement according to claim 1, characterized in that: In step 4, the feature fusion module effectively fuses feature information from different sources, that is, it performs a series of element-by-element addition, 3×3 convolution, and layer normalization operations on the multi-scale features of the query image obtained by the feature extractor and the example features weighted by similarity weights, thereby improving the robustness and accuracy of the model in processing few-sample target counting tasks.

Images