A unified counting method for ear crops based on few-sample learning

Abstract

The present invention proposes a unified counting method for ear crops based on few-sample learning. The present invention relates to six different crop data sets, which are captured by ground cameras and drones, visual encoders for enhancing feature embedding, basic models for alleviating the impact of complex backgrounds, and multi-scale feature interaction modules for integrating similarity metrics, promoting automatic learning of crop features at different scales, and enhancing the ability to describe ear crops of various sizes and shapes. The model adopts a two-stage training procedure, the initial stage focuses on potential feature mining to capture the common features of cereals, and subsequent stages can perform reasoning without additional training, thereby extracting the domain features of the target crop from the selected samples to achieve counting operations. The present invention counts multiple ears of cereals through few-sample learning, which can effectively reduce the labeling cost, improve the accuracy of cereal yield estimation, and ensure food security.

Description

Technical Field

[0001] The present invention relates to the technical field of few-sample counting, and in particular to a unified counting method for ear crops based on few-sample learning. Background Art

[0002] Accurate counting of cereal crops is essential for estimating food production and ensuring food security. By accurately quantifying the number of crops, farmers and researchers can gain valuable insights into crop growth and yield estimation. In addition, the number of crops is also an essential parameter in breeding programs to develop high-yield and stress-resistant crop varieties.

[0003] Traditional methods for counting cereal crops fall into two categories: manual counting and image processing-based techniques. Manual counting relies on visual inspection, so it is usually intensive, time-consuming, and prone to errors due to subjective observation. Therefore, it is not suitable for large-scale agricultural environments. Image processing techniques involve extracting hand-crafted features such as color, texture, morphology, etc., and converting the counting into a classification problem through machine learning. However, due to their dependence on predefined features, these methods are often inefficient when faced with non-uniform lighting conditions and backgrounds, so these methods may not be applicable in actual field environments.

[0004] In recent years, the rapid progress of deep learning has led to its widespread application in agriculture. Given its excellent ability to capture two-dimensional (2D) spatial information, counting methods based on convolutional neural networks (CNNs) are superior to traditional image processing techniques. These counting methods mainly include three approaches: segmentation, object detection, and regression based on density maps. Segmentation-based counting can achieve accurate separation between crops and backgrounds through pixel-level network training, and promote accurate demarcation of object boundaries. However, this method requires pixel-level annotations, and detailed annotation requirements require a lot of time and effort. Object detection-based counting methods can identify and classify objects by generating candidate regions and calculating scores, but their performance may decrease in dense areas. Density map-based regression methods establish a direct relationship between local image features and the number of objects, and map the original image with the corresponding density map. Density map-based methods perform well in accurately locating individual crop ears in the image, especially in densely occluded scenes. Due to their excellent counting performance, density map-based methods have received significant attention in counting ears of various cereal crops. However, in the prior art, density map methods for counting cereal crops mainly focus on establishing counting models for specific crop ears. However, due to the significant morphological differences between different cereal crops, a model trained on one crop may not be generalizable to other types of crops. Developing ear counting models for new crops requires re-annotating samples and training deep learning models, which is costly. In addition, even for the same crop ears, the counting performance of the established model may not be transferable due to changes in the data acquisition environment (such as shooting angle, distance, time, etc.). The fine-tuning strategy in transfer learning can alleviate this problem to a certain extent. However, the fine-tuning process may require training on annotated samples in the target dataset (domain).

[0005] In view of the characteristics of complex background, different scales and shapes, and large data set requirements in the current ear crop counting scenarios, the unified ear crop counting method based on few-sample learning proposed in the present invention can handle various types of crop ears and multiple data acquisition environments at a low annotation cost, so as to reduce the current method's demand for data sets and reduce the complicated training and fine-tuning workload, thereby achieving accurate and efficient crop head counting. Summary of the invention

[0006] In order to solve the above problems, the present invention proposes a unified counting method for ear crops based on few-sample learning, which can handle various types of crop ears and multiple data acquisition environments at low annotation cost, so as to reduce the current method's demand for data sets and reduce the complicated training and fine-tuning workload, thereby achieving accurate and efficient crop head counting.

[0007] In order to achieve the above object, the present invention is implemented by the following technical solutions:

[0008] A unified ear crop counting method based on few-sample learning, characterized by comprising the following operations:

[0009] An open source data set of ear crops is obtained from the Internet, and data is cleaned on the open source data set to obtain a processed data set, and data preprocessing is performed on the processed data set to construct a crop data set;

[0010] A visual basic model is selected as the backbone network, and the background of the supporting image in the crop dataset is filtered through a visual feature purification module;

[0011] By using a multi-scale similarity comparison module, point-to-point feature comparisons are performed on the query image and the support image in the crop dataset at different scales to obtain a multi-scale similarity graph;

[0012] The multiscale similarity map is fused with the supporting image features through a multiscale feature enhancement module to obtain a fused similarity feature map, the fused similarity feature map is added to the initial query image features to obtain an enhanced feature map, and the enhanced feature map is re-input into the multiscale similarity comparison module for feature enhancement iteration to generate an iteratively enhanced feature map;

[0013] The iteratively enhanced feature map is regressed through a regression head to generate a density map, and the density map is regressed to obtain a final count.

[0014] The further improvement of the present invention is that the visual feature extraction module uses a mask instead of a bounding box to depict the boundary and shape information of the object, uses the bounding box of the support image as a hint, generates a mask of the crop head, and extracts features aligned with the mask position from the embedded features, uses the visual transformer as an image encoder for the support image and the query image, and uses the pre-trained DINOv2 weights to embed the visual transformer. The selected features need to be average pooled to generate the final prototype vector. Set the support image to I S , the generated improved mask image M R The expression is:

[0015] M R =SAM(I S )

[0016] Where M R To improve the mask image; SAM is to use the SAM visual segmentation model for segmentation; I S To support the image;

[0017] Query image I Q The visual transformer embedding feature F P The expression is:

[0018] F P =ViT(I Q )

[0019] In the formula, F P is the feature embedding of the visual transformer; ViT is the feature extraction using the visual transformer; I Q for the query image;

[0020] The expression of the extracted mask feature is:

[0021] F S =F P ·M R

[0022] In the formula, F S is the extracted mask feature; F P Embedding features for visual transformers; M R To improve the mask image;

[0023] The extracted mask features are the supporting image features, which are aggregated into prototypes using three different scales k represents different scales, that is, using average pooling operations of different sizes.

[0024] A further improvement of the present invention is that the multi-scale similarity comparison module performs similarity comparison on the prototype in the visual feature purification module and the query image features of different scales, and uses three kernels of different sizes for pooling operations to generate prototypes of three different scales. represents the query image features, represents the prototype feature, N represents the number of supported images, and the multi-scale similarity comparison module includes the following steps:

[0025] Feature mapping, three prototype features of different sizes f S and query image features f Q They are mapped to the same embedding space through a combination of 1×1 convolution kernels and layer normalization;

[0026] Convolution comparison, three prototype features of different sizes f S as the convolution kernel and combined with the query image feature f Q Perform convolution to generate the initial similarity graph through point-surface feature interaction Initial Similarity Graph The expression is:

[0027]

[0028] In the formula, is the initial similarity graph; f Q To query image features; is the prototype feature; g is the combination of 1×1 convolution layer and layer normalization in the feature map; conv is the convolution operation on the feature map with kernel as the convolution kernel; kernel is the use of the feature map as the convolution kernel; k is the scale;

[0029] Normalization, use sample normalization and space normalization to normalize the initial similarity graph:

[0030]

[0031] In the formula, It is the feature map after the initial similarity map is normalized by samples; is the feature map after spatial normalization of the initial similarity map; dim is the feature map dimension, dim=0 represents the sample dimension, dim=1 represents the channel dimension, dim=2 represents the height dimension, and dim=3 represents the width dimension; H S is the height of the initial similarity graph; W S is the width of the initial similarity graph; C is the channel of the initial similarity graph; softmax dim is a softmax layer running along the dimension; max dim To determine the maximum value in a dimension; is the initial similarity graph; exp is the exponential function with e as the base; k is the scale;

[0032] Get the multi-scale similarity graph R k The expression is:

[0033]

[0034] In the formula, R k is a multi-scale similarity graph; is element-wise multiplication; k is the scale; It is the feature map after the initial similarity map is normalized by samples; It is the feature map after spatial normalization of the initial similarity map.

[0035] A further improvement of the present invention is that: according to the multi-scale similarity graphs of three different scales generated by the multi-scale similarity comparison module, three similarity feature graphs of similarity and spatial information of different scales are obtained, that is, a multi-scale similarity feature enhancement module, and the multi-scale similarity feature enhancement module includes the following steps:

[0036] The multi-scale similarity map is converted into a similarity feature map, and the flip prototype is used as the convolution kernel to convolve with the multi-scale similarity maps of three different scales. k and supporting image features Generate three similarity feature maps at different scales Similarity feature graph The expression is:

[0037]

[0038] In the formula, is the similarity feature map; sum is the accumulation of the input tensor along the dimension; R k is a multi-scale similarity graph; To support image features; k is the scale; kernel is to use the feature map as the convolution kernel; Flip is to flip the feature map; dim is the dimension; Conv is the convolution operation using the feature map as the convolution kernel;

[0039] Multi-scale spatial attention: the similarity feature maps of three different scales are compared in scale globally, without paying attention to the spatial dimension of the similarity feature map. According to the attention mechanism, MaxPool and AvgPool are performed along the channel dimension, and the MaxPool and AvgPool features are concatenated. A 7×7 convolution is applied to extract the spatial attention map, and the obtained spatial attention map is element-wise multiplied with the similarity feature map to obtain a similarity feature map focusing on the spatial scale.

[0040]

[0041] In the formula, It is a similarity feature map focusing on spatial scale; is the similarity feature map; k is the scale; σ is the s-type nonlinear function; Conv is the convolution operation, and its kernel size is 7×7; MaxPool is the maximum pooling operation on the feature map;

[0042] Multi-scale similarity feature map fusion, space-centered similarity feature map The fusion is obtained by simple connection C , local information is aggregated through 5×5 deep convolution, multi-branch deep convolution is used to capture multi-scale background, and 1×1 deep convolution is used to simulate the relationship between channels. After the dimensionality reduction processing of the 1×1 convolution layer, a fusion similarity feature map f' across three scales is obtained. C ;

[0043] Feature enhancement, adding the fused similarity feature map to the initial query image feature f Q Perform feature enhancement in the image to obtain the enhanced feature map f P , the enhanced feature map f P As input query image features f Q , re-input into the multi-scale similarity comparison module, and perform N feature enhancement iterations to obtain the iteratively enhanced feature map:

[0044] f P =LayerNorm(f Q +h(f′ C ))

[0045] In the formula, f P is the enhanced feature map; LayerNorm is layer normalization; f Q is the query image feature; f' C is the fusion similarity feature map; h is the convolution operation consisting of two convolution layers on the feature map.

[0046] A further improvement of the present invention is that the regression head includes a convolutional layer, an activation function and bilinear upsampling.

[0047] Beneficial effects of the present invention: The present invention proposes a unified counting method for ear crops based on few-sample learning, which obtains an open source data set of ear crops from the Internet, performs data clarification on the open source data set to obtain a processed data set, performs data preprocessing on the processed data set and constructs a crop data set; selects a visual basic model as a backbone network, and filters the background of the supporting image in the crop data set through a visual feature purification module; performs point-to-point feature comparison on the query image and the supporting image in the crop data set at different scales through a multi-scale similarity comparison module to obtain a multi-scale similarity graph; and performs feature enhancement on the multi-scale feature enhancement module. block, the multi-scale similarity map is fused with the supporting image features to obtain a fused similarity feature map, the fused similarity feature map is added to the initial query image features to obtain an enhanced feature map, and the enhanced feature map is re-input into the multi-scale similarity comparison module for feature enhancement iteration to generate an iteratively enhanced feature map; the iteratively enhanced feature map is regressed through a regression head to generate a density map, and the density map is regressed to obtain a final count, so that the present invention can count multiple grain ears through few-sample learning, effectively reduce the labeling cost, and improve the accuracy of grain yield estimation to ensure food security. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 A schematic diagram of a flow chart of a unified ear crop counting method based on few-sample learning provided by an embodiment of the present invention;

[0049] Figure 2 The overall framework diagram of the two-stage model provided by the embodiment of the present invention;

[0050] Figure 3 A structural diagram of a visual feature purification module provided in an embodiment of the present invention;

[0051] Figure 4A structural diagram of a multi-scale similarity comparison module provided in an embodiment of the present invention;

[0052] Figure 5 A structural diagram of a multi-scale feature enhancement module provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0053] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. In addition, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0054] See also Figure 1 , a unified ear crop counting method based on few-sample learning in this embodiment includes the following operations:

[0055] Acquire an open source dataset of ear crops from the Internet, perform data preprocessing on the open source dataset and construct a crop dataset;

[0056] A visual basic model is selected as the backbone network, and the background of the supporting image in the crop dataset is filtered through a visual feature purification module;

[0057] By using a multi-scale similarity comparison module, point-to-point feature comparisons are performed on the query image and the support image in the crop dataset at different scales to obtain a multi-scale similarity graph;

[0058] The multiscale similarity map is fused with the supporting image features through a multiscale feature enhancement module to obtain a fused similarity feature map, the fused similarity feature map is added to the initial query image features to obtain an enhanced feature map, and the enhanced feature map is re-input into the multiscale similarity comparison module for feature enhancement iteration to generate an iteratively enhanced feature map;

[0059] The iteratively enhanced feature map is regressed through a regression head to generate a density map, and the density map is regressed to obtain a final count.

[0060] See also Figure 2First, the main structure of the method of the present invention is introduced. Specifically, in order to achieve unified counting of cereal crops, the present invention proposes a new two-stage method to gradually establish a unified cereal crop counting method. In the first stage, the method mines potential features through offline training to identify the common features of cereal crops. In the subsequent stage, the method uses a few-sample learning paradigm to extract domain-specific features of the target scene, and then generates a similarity feature map through a similarity comparison network. The feature map can reflect the corresponding domain-specific features and can be decoded with the common features of cereal crops, so that accurate counting can be achieved.

[0061] Among them, the sizes of the query image and its corresponding feature map are set to 512×512 and 128×128, respectively. For the support images, the prototype sizes are defined as 1×1, 3×3, and 5×5 at three different scales, and are represented by k=1,3,5 in the visual feature purification module. The dimension of the projected feature is reshaped to 256. The Adam optimizer is used for 200 cycles of system training during training, and the batch size is set to 1. The learning rate is initially set to 2e-5 and decreases by a factor of 0.25 every 80 cycles to optimize the learning curve. In the CHCNet proposed in the present invention, each image is annotated with three bounding boxes and used as support images. It should be noted that the support images used in each learning iteration must remain consistent.

[0062] See also Figure 3 , the visual feature purification module uses masks instead of bounding boxes, thereby enhancing the distinction between the target crop head and the background. Compared with bounding boxes, masks provide better details and clearly depict the boundaries and shape information of objects. The recently introduced basic segmentation model SAM can segment objects using only boxes or points as hints, and it can efficiently segment all objects in the image in one or few modes. It should be noted that ViT stands for visual transformer. The present invention uses the bounding box of the support image as a hint to generate a mask of the crop head, and extracts features aligned with the mask position from the embedded features, and uses ViT as the image encoder for the support image and the query image. In order to improve the generalization ability, ViT embedding is performed using pre-trained DINOv2 weights. Subsequently, the selected features are average pooled to generate the final prototype vector. Set the support image to I S , the generated improved mask image M R The expression is:

[0063] M R =SAM(I S )

[0064] Where M R To improve the mask image; SAM is to use the SAM visual segmentation model for segmentation; I STo support the image;

[0065] Query image I Q The ViT embedding feature F P The expression is:

[0066] F P =ViT(I Q )

[0067] In the formula, F P is the ViT embedding feature; ViT is the feature extracted using the ViT encoder; I Q is the query image; the expression of the extracted mask feature is:

[0068] F S =F P ·M R

[0069] In the formula, F S is the extracted mask feature; F P is the ViT embedding feature; M R To improve the mask image;

[0070] To facilitate subsequent multi-scale analysis, the extracted mask features are the supporting image features, which are aggregated into prototypes using three different scales (i.e., small, medium, and large). k represents different scales, that is, using average pooling operations of different sizes.

[0071] See also Figure 4 The multi-scale similarity comparison module compares the similarity between the prototypes in the visual feature extraction module and the query image features of different scales, thereby enhancing the module's ability to recognize objects of different sizes. It uses three different sized kernels for pooling operations to generate prototypes of three different scales. Among them, the small-scale prototype The original feature scale is retained, and the nonlinearity and depth of the feature channel can be increased; the mesoscale prototype Integrates information from neighboring pixels to capture local spatial patterns; large-scale prototypes We extend this framework to include larger local regions and incorporate more comprehensive spatial context to facilitate comparisons from a wider perspective. These multi-scale prototypes enable the model to exploit multiple levels of spatial information, thus enhancing its adaptability in different visual tasks. represents the query image features, represents the prototype feature, N represents the number of supported images, and the multi-scale similarity comparison module includes the following steps:

[0072] Feature mapping, at the beginning, three prototype features f of different sizes S and query image features f QThey are mapped to the same embedding space through a combination of 1×1 convolution kernels and layer normalization, ensuring that they have the same distribution;

[0073] Convolution comparison, three prototype features of different sizes f S as the convolution kernel and combined with the query image feature f Q Perform convolution to generate the initial similarity graph through point-surface feature interaction Initial Similarity Graph The expression is:

[0074]

[0075] In the formula, is the initial similarity graph; f Q To query image features; is the prototype feature; g is the combination of 1×1 convolution layer and layer normalization in the feature map; conv is the convolution operation on the feature with kernel as the convolution kernel; kernel is the use of feature map as the convolution kernel; k is the scale;

[0076] Normalization, using sample normalization and spatial normalization to normalize the initial similarity map, so as to ensure the consistency of scale and distribution of the similarity map:

[0077]

[0078] In the formula, It is the feature map after the initial similarity map is normalized by samples; is the feature map after spatial normalization of the initial similarity map; dim is the feature map dimension, dim=0 represents the sample dimension, dim=1 represents the channel dimension, dim=2 represents the height dimension, and dim=3 represents the width dimension; H S is the height of the initial similarity graph; W S is the width of the initial similarity graph; C is the channel of the initial similarity graph; softmax dim is a softmax layer running along the dimension; max dim To determine the maximum value in a dimension; is the initial similarity graph; exp is the exponential function with e as the base; k is the scale;

[0079] Get the multi-scale similarity graph R k The expression is:

[0080]

[0081] In the formula, R k is a multi-scale similarity graph; is element-wise multiplication; k is the scale; It is the feature map after the initial similarity map is normalized by samples; It is the feature map after spatial normalization of the initial similarity map.

[0082] See also Figure 5 According to the multi-scale similarity graphs of three different scales generated by the multi-scale similarity comparison module, similarity feature graphs of three different scale similarities and spatial information are obtained, namely, the multi-scale similarity feature enhancement module (MSFEM). The multi-scale similarity feature enhancement module includes the following steps:

[0083] The multi-scale similarity map is converted into a similarity feature map, and the flip prototype is used as the convolution kernel to convolve with the multi-scale similarity maps of three different scales. k and supporting image features Generate three similarity feature maps at different scales And this flipping process can ensure the similarity feature map Keep The spatial structure of the similarity feature map The expression is:

[0084]

[0085] In the formula, is the similarity feature map; sum is the accumulation of the input tensor along the dimension; R k is a multi-scale similarity graph; To support image features; k is the scale; kernel is to use the feature map as the convolution kernel; Flip is to flip the feature map; dim is the dimension; Conv is the convolution operation using the feature map as the convolution kernel;

[0086] Multi-scale spatial attention: the similarity feature maps of three different scales are compared in scale globally, without paying attention to the spatial dimension of the similarity feature map. According to the attention mechanism, MaxPool and AvgPool are performed along the channel dimension, and the MaxPool and AvgPool features are concatenated. A 7×7 convolution is applied to extract the spatial attention map, and the obtained spatial attention map is element-wise multiplied with the similarity feature map to obtain a similarity feature map focusing on the spatial scale.

[0087]

[0088] In the formula, It is a similarity feature map focusing on spatial scale; is the similarity feature map; k is the scale; σ is the s-type nonlinear function; Conv is the convolution operation, and its kernel size is 7×7; MaxPool is the maximum pooling operation on the feature map;

[0089] Multi-scale similarity feature map fusion, space-centered similarity feature map The fusion is obtained by simple connection C , aggregates local information through 5×5 deep convolution, captures multi-scale background through multi-branch deep convolution, and simulates the relationship between channels through 1×1 deep convolution. By adopting multi-scale convolution attention, the model can adaptively select the weights of similarity feature maps at three different scales. After dimensionality reduction processing by 1×1 convolution layer, the fused similarity feature map f' across three scales is obtained C ;

[0090] Feature enhancement, adding the fused similarity feature map to the initial query image feature f Q Perform feature enhancement in the image to obtain the enhanced feature map f P , the enhanced feature map f P As input query image feature f Q , re-input into the multi-scale similarity comparison module, and perform N feature enhancement iterations to obtain the iteratively enhanced feature map:

[0091] f P =LayerNorm(f Q +h(f′ C ))

[0092] In the formula, f P is the enhanced feature map; LayerNorm is layer normalization; f Q is the query image feature; f' C is the fusion similarity feature map; h is the convolution operation consisting of two convolution layers on the feature map.

[0093] The regression head consists of a series of convolutional layers, Leaky ReLU activation functions, and bilinear upsampling.

[0094] The present invention will be further described below in conjunction with specific embodiments:

[0095] The image datasets used in the present invention include the online public datasets GWHD, WEDD, MTC, MTC-UAV, Sorghum, and Rice, which include four ear crops: wheat ears, corn ears, sorghum ears, and rice ears.

[0096] (1) Using the GWHD dataset as a training dataset, the image data of the supervision branch is enhanced before being input into the model method of the present invention.

[0097] (2) Input the extracted features into the network, train the model and find the convergence point.

[0098] (3) The trained model was directly tested on the remaining spike crop datasets to verify the generalization performance of the code model. The experimental results are shown in Table 1, where * indicates that the model directly adopts the checkpoint trained on the FSC147 dataset introduced in the original paper:

[0099] Table 1 Experimental quantitative results

[0100]

[0101] Through the introduction of the above embodiments, the present invention proposes a unified counting method for ear crops based on few-sample learning, which obtains an open source data set of ear crops from the Internet, performs data clarification on the open source data set to obtain a processed data set, performs data preprocessing on the processed data set and constructs a crop data set; selects a visual basic model as the backbone network, and filters the background of the supporting image in the crop data set through a visual feature purification module; performs point-to-point feature comparison on the query image and the supporting image in the crop data set at different scales through a multi-scale similarity comparison module to obtain a multi-scale similarity graph; and enhances the multi-scale feature by using a multi-scale feature extraction module. module, fusing the multi-scale similarity map with the supporting image features to obtain a fused similarity feature map, adding the fused similarity feature map to the initial query image features to obtain an enhanced feature map, and re-inputting the enhanced feature map into the multi-scale similarity comparison module to perform feature enhancement iteration to generate an iteratively enhanced feature map; regressing the iteratively enhanced feature map through a regression head to generate a density map, and regressing the density map to obtain a final count, so that the present invention can count multiple grain ears through few-sample learning, effectively reduce the labeling cost, and improve the accuracy of grain yield estimation to ensure food security.

[0102] It will be understood by those skilled in the art that, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as those generally understood by those skilled in the art in the art to which the present invention belongs. It should also be understood that terms such as those defined in common dictionaries should be understood to have the same meaning as in the context of the prior art, and will not be interpreted with an idealized or overly formal meaning unless defined similarly here.

[0103] The specific implementation methods described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific implementation method of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A unified ear crop counting method based on few-sample learning, characterized in that: The following operations are included: An open source data set of ear crops is obtained from the Internet, and data is cleaned on the open source data set to obtain a processed data set, and data preprocessing is performed on the processed data set to construct a crop data set; A visual basic model is selected as the backbone network, and the background of the supporting image in the crop dataset is filtered through a visual feature purification module; By using a multi-scale similarity comparison module, point-to-point feature comparisons are performed on the query image and the support image in the crop dataset at different scales to obtain a multi-scale similarity graph; The multiscale similarity map is fused with the supporting image features through a multiscale feature enhancement module to obtain a fused similarity feature map, the fused similarity feature map is added to the initial query image features to obtain an enhanced feature map, and the enhanced feature map is re-input into the multiscale similarity comparison module for feature enhancement iteration to generate an iteratively enhanced feature map; The iteratively enhanced feature map is regressed through a regression head to generate a density map, and the density map is regressed to obtain a final count.

2. The method for uniformly counting ear crops based on few-sample learning according to claim 1, characterized in that: The visual feature extraction module uses a mask instead of a bounding box to depict the boundary and shape information of the object, uses the bounding box of the support image as a hint, generates a mask of the crop head, and extracts features aligned with the mask position from the embedded features. The visual transformer is used as an image encoder for the support image and the query image, and the pre-trained DINOv2 weights are used to embed the visual transformer. The selected features need to be average pooled to generate the final prototype vector; the support image is set as I S , the generated improved mask image M R The expression is: M R =SAM(I S ) Where M R To improve the mask image; SAM is to use the SAM visual segmentation model for segmentation; I S To support the image; Query image I Q The visual transformer embedding feature F P The expression is: F P =ViT(I Q ) In the formula, F P embedding features for visual transformers; ViT is feature extraction using visual transformer; I Q for the query image; The expression of the extracted mask feature is: F S =F P ·M R In the formula, F S is the extracted mask feature; F P embedding features for visual transformers; M R To improve the mask image; The extracted mask features are the supporting image features, which are aggregated into prototypes using three different scales k represents different scales, that is, using average pooling operations of different sizes.

3. The method for unified ear crop counting based on few-sample learning according to claim 1, characterized in that: The multi-scale similarity comparison module performs similarity comparison between the prototype in the visual feature purification module and the query image features of different scales. It uses three kernels of different sizes for pooling operations to generate prototypes of three different scales. represents the query image features, represents the prototype feature, N represents the number of supported images, and the multi-scale similarity comparison module includes the following steps: Feature mapping, three prototype features of different sizes f S and query image features f Q They are mapped to the same embedding space through a combination of 1×1 convolution kernels and layer normalization; Convolution comparison, three prototype features of different sizes f S as the convolution kernel and combined with the query image feature f Q Perform convolution to generate the initial similarity graph through point-surface feature interaction Initial Similarity Graph The expression is: In the formula, is the initial similarity graph; f Q To query image features; is the prototype feature; g is the combination of 1×1 convolution layer and layer normalization in the feature map; conv is the convolution operation on the feature map with kernel as the convolution kernel; kernel is the use of the feature map as the convolution kernel; k is the scale; Normalization, use sample normalization and space normalization to normalize the initial similarity graph: In the formula, It is the feature map after the initial similarity map is normalized by samples; is the feature map after spatial normalization of the initial similarity map; dim is the feature map dimension, dim=0 represents the sample dimension, dim=1 represents the channel dimension, dim=2 represents the height dimension, and dim=3 represents the width dimension; H S is the height of the initial similarity graph; W S is the width of the initial similarity graph; C is the channel of the initial similarity graph; softmax dim is a softmax layer running along the dimension; max dim To determine the maximum value in a dimension; is the initial similarity graph; exp is the exponential function with e as the base; k is the scale; Get the multi-scale similarity graph R k The expression is: In the formula, R k is a multi-scale similarity graph; is element-wise multiplication; k is the scale; It is the feature map after the initial similarity map is normalized by samples; It is the feature map after spatial normalization of the initial similarity map.

4. The method for uniformly counting ear crops based on few-sample learning according to claim 1, characterized in that: According to the multiscale similarity graphs of three different scales generated by the multiscale similarity comparison module, similarity feature graphs of three different scale similarities and spatial information are obtained, i.e., a multiscale similarity feature enhancement module, which includes the following steps: The multi-scale similarity map is converted into a similarity feature map. The flip prototype is used as the convolution kernel to convolve with the multi-scale similarity maps of three different scales. k and supporting image features Generate three similarity feature maps at different scales Similarity feature graph The expression is: In the formula, is the similarity feature map; sum is the accumulation of the input tensor along the dimension; R k is a multi-scale similarity graph; To support image features; k is the scale; kernel is to use the feature map as the convolution kernel; Flip is to flip the feature map; dim is the dimension; Conv is the convolution operation using the feature map as the convolution kernel; Multi-scale spatial attention: the similarity feature maps of three different scales are compared in scale globally, without paying attention to the spatial dimension of the similarity feature map. According to the attention mechanism, MaxPool and AvgPool are performed along the channel dimension, and the MaxPool and AvgPool features are concatenated. A 7×7 convolution is applied to extract the spatial attention map, and the obtained spatial attention map is element-wise multiplied with the similarity feature map to obtain a similarity feature map focusing on the spatial scale. In the formula, It is a similarity feature map focusing on spatial scale; is the similarity feature map; k is the scale; σ is the s-type nonlinear function; Conv is the convolution operation, and its kernel size is 7×7; MaxPool is the maximum pooling operation on the feature map; Multi-scale similarity feature map fusion, space-centered similarity feature map The fusion is obtained by simple connection C , local information is aggregated through 5×5 deep convolution, multi-branch deep convolution is used to capture multi-scale background, and 1×1 deep convolution is used to simulate the relationship between channels. After the dimensionality reduction processing of the 1×1 convolution layer, a fusion similarity feature map f' across three scales is obtained. C ; Feature enhancement, adding the fused similarity feature map to the initial query image feature f Q Perform feature enhancement in the image to obtain the enhanced feature map f P , the enhanced feature map f P As input query image features f Q , re-input into the multi-scale similarity comparison module, and perform N feature enhancement iterations to obtain the iteratively enhanced feature map: f P =LayerNorm(f Q +h(f' C )) In the formula, f P is the enhanced feature map; LayerNorm is layer normalization; f Q is the query image feature; f' C is the fusion similarity feature map; h is the convolution operation consisting of two convolution layers on the feature map.

5. The method for uniformly counting ear crops based on few-sample learning according to claim 1, characterized in that: The regression head includes a convolutional layer, an activation function, and bilinear upsampling.

Images