A method for counting wheat ear density in the field based on local-global feature fusion

Abstract

The present invention discloses a method for counting wheat ears in the field based on local-global feature fusion. The method first collects wheat ear RGB image data, unifies the size, and uses point annotation to preprocess the wheat ear data set to establish labels and construct a training data set; then the training data set is input into the model for training, and the wheat ear features of the wheat image are learned and counted. The counting model uses CSPDarknet53 to extract local features of the image, and adds a Pyramid Pooling Transformer with a self-attention mechanism to extract global context information during the feature extraction process. By fusing local features and global context information, the accuracy of wheat ear counting is improved, and it can be effectively applied to actual field wheat ear counting.

Description

Technical Field

[0001] The invention belongs to the field of image density counting, and in particular relates to a method for counting density of wheat ears in a field based on local-global feature fusion. Background Art

[0002] The number of wheat ears is one of the important indicators reflecting the wheat phenotype. Accurately counting the number of wheat ears is crucial for agricultural producers to scientifically plan and effectively use resources such as water, fertilizers, and pesticides, improve production efficiency, and reduce the burden on the environment. However, the traditional manual counting method is subjective, easily leads to misidentification and omission of wheat ears, is inefficient, and is time-consuming and labor-intensive. Therefore, the use of modern and automated counting methods is of strategic importance for agricultural production and management.

[0003] With the rapid development of deep learning, deep learning technology has been widely used in wheat counting tasks with its powerful computing power and feature extraction capabilities. Past counting methods were mainly based on convolutional neural networks, such as MCNN, CSRNet, and SCAR. These methods rely on deep learning models and weakly labeled training data sets, and use their excellent local feature extraction capabilities to generate density maps by learning the density information of wheat ears at each position in the image. Proper processing and thresholding of the density map can obtain the location and quantity information of wheat ears, thereby realizing the counting of wheat ears. Compared with traditional methods based on manual feature extraction, these deep learning methods perform better. However, since the convolution operation is limited to the receptive field of the convolution kernel, it is impossible to fully understand the global information of the image, which may lead to a decrease in counting accuracy.

[0004] In recent years, transformer models based on self-attention mechanisms have been widely used in visual tasks. The self-attention mechanism has significant advantages in extracting global context information and can handle complex spatial transformations and long-term feature dependencies. Such global information and long-term feature dependencies are crucial components for the success of tasks. Therefore, in order to fully integrate and utilize local features and global information, we studied a field wheat ear density counting method based on local-global feature fusion. By extracting local features and global information separately and fusing them, the representation ability of the feature map is enhanced to improve the counting accuracy. Summary of the invention

[0005] This paper aims to propose a method for counting wheat ears in the field based on local-global feature fusion, combining the global context representation capability of the transformer model with the local properties of the convolutional network. This application aims to improve the accuracy of wheat ear counting by increasing the diversity of features.

[0006] Technical solution: The present invention provides a method for counting wheat ear density in a field based on local-global feature fusion, comprising the following steps:

[0007] Step 1: Collect wheat RGB image datasets, unify pixel sizes, annotate wheat ears using point annotation, generate true density maps using Gaussian functions, and divide images into training and test sets.

[0008] Step 2, constructing a field wheat ear density counting model based on local-global feature fusion and training it, the counting model includes a feature extraction network for extracting multi-scale features, a feature fusion network for multi-scale feature fusion and a density regression network for generating a density map, wherein the feature extraction network is a network structure combining a CSPDarknet53 network and a pyramid pooling Transformer network, the CSPDarknet53 network extracts multi-scale local features, and inputs the extracted feature map of the smallest scale into the pyramid pooling Transformer network, based on a multi-head pooling self-attention mechanism to capture global context information and extract global features, and then through a jump connection feature fusion network, use an attention mechanism to fuse feature maps of different scales to obtain an effective feature map, and finally input the density regression network, and construct a density regression network based on dilated convolutions with different dilated rates to generate a final predicted density map.

[0009] Step 3, input the test data set into the counting model trained in step 2 for testing, input the RGB image of wheat to be counted, and obtain the wheat ear distribution density map and wheat ear counting results;

[0010] Preferably, step 1 includes the following steps: step 1.1, collecting images: collecting different types of wheat RGB images from public data sets to construct a diversity data set; step 1.2, format unification: converting the collected wheat RGB image files into PNG format files, and cutting the processed image files into uniform size; step 1.3, data annotation: using point annotation to annotate the wheat ear area on the image data file, and one-to-one correspondence between the original image data file of the wheat ear and the annotated data file generated by the annotation; step 1.4, density generation: obtaining wheat ear position data from the annotated data file, and using a Gaussian kernel to convert the wheat ear position data into a ground wheat ear real density distribution map. Step 1.5, data augmentation: randomly flipping the processed image data file and the density distribution file and randomly changing the contrast to achieve image augmentation.

[0011] As a preferred method, the method for generating the real density distribution map is: using the Gaussian function Gσ(x) to smooth the discrete density map generated by H(x), converting it into a continuous function, and generating the ground wheat ear real density distribution map F(x):

[0012]

[0013]

[0014] In the formula, p i is the position of the ith wheat ear, H(x) is the generated discrete density map, σ represents the standard deviation, and the resulting Gaussian density map has the property that the sum is equal to the total number of wheat ears in the image.

[0015] Preferably, step 2 comprises the following steps:

[0016] Step 2.1, the input image is first preprocessed and normalized to scale the pixel value to between 0 and 1; the preprocessed image is input into the CSPDarknet53 network to extract feature maps of four different scales, represented as X1, X2, X3, and X4; the feature extraction network includes a downsampling process, and the downsampling is used to train the input image data to obtain multi-scale feature information;

[0017] Step 2.2, reshape the dimension of the minimum scale feature map X4 extracted in step 2.1, reduce the length of the image feature sequence with the help of the pyramid pooling multi-head self-attention, and then input it into the pyramid pooling Transformer network to further extract the context global feature map X5;

[0018] Step 2.3, input the local feature maps X1, X2, X3 extracted in step 2.1 and the global feature map X5 extracted in step 2.2 into the feature fusion network based on the attention mechanism to perform multi-scale feature fusion and obtain a valid feature map;

[0019] Step 2.4, input the effective feature map into the density regression network to generate a predicted density map and perform density summation to achieve wheat ear counting.

[0020] Preferably, the feature extraction network includes 4 CSPDarknet53s of different scales. In step 2.1, the input feature map is divided into two parts according to the number of channels. One part undergoes a product operation with a convolution kernel size of 3x3, BN normalization and SILU function, and the other part is directly connected to the output and connected to the first part; each CSPDarknet53 module first uses a product operation with a convolution kernel size of 3x3, BN normalization and SILU function to perform scale downsampling to achieve multi-scale feature extraction.

[0021] Preferably, in step 2.2, the dimension reshaping and global feature acquisition are specifically as follows: reshaping the feature map of (B, C, H, W) dimension into a feature map of (B, C, H*W) dimension, inputting the flattened feature map into the image block encoder PatchEmbedding, adding the position code and inputting it into the stacked pyramid pooling Transformer module, and obtaining the multi-scale feature map P by applying pooling operations of different scales. i and relative position encoding It is expressed as:

[0022] P i =AvgPool i (x)

[0023]

[0024] Where DWConv(·) is the depth convolution;

[0025] The input sequence is obtained by flattening and concatenating the feature maps of the different pools Calculate its attention to obtain the final feature X5, expressed as:

[0026]

[0027] Where Q, K, and V represent the query, key, and value obtained by linear projection of sequence P, respectively, and dk is the length of the key vector K.

[0028] Preferably, in step 2.3, the feature fusion network uses an attention mechanism to fuse multi-scale features, and the fusion process is specifically as follows:

[0029] Step 2.31, upsample the extracted multi-scale features using dilated convolution to obtain feature maps of corresponding scales;

[0030] Step 2.32, after adding the feature maps of the same scale, they are divided into two branches according to the channel dimension. One branch uses Global Avg Pooling to extract global attention from the feature map; the other branch uses point-wise to extract local channel attention from the feature map.

[0031] Step 2.33, add the global attention and channel attention after point multiplication to obtain the fused feature map, that is, combine the global channel context g(X) and the local channel context L(X), and obtain the refined feature X'∈R through MS-CAM C×H×W It is expressed as:

[0032] G(x)=BN(PWConv2(δ(BN(PWconv1(GlobalPooling(X))))))

[0033] L(x)=BN(PWConv2(δ(BN(PWconv1(X)))))

[0034] X′=X⊕σ(L(X)⊕G(X))

[0035] Where GlobalPooling is the global average pooling operation, PWconv1 is a point convolution with a kernel size of 1, which is used to reduce the number of channels of the input feature X, BN represents BatchNorm normalization, δ is the ReLU activation function, and the number of channels is restored to the same as the original input channel number through PWconv2 point convolution, σ is the Sigmode activation function, the purpose of which is to map the variable to between 0 and 1.

[0036] Preferably, in step 2.4, three convolution blocks with different void rates are used in the density regression network to process features separately, and the three obtained feature maps are concat spliced, and the concatenated feature maps are subjected to convolution Conv, normalization BatchNorm and activation function ReLU operations to generate a density map.

[0037] Preferably, step 3 comprises the following steps:

[0038] Step 3.1, input the test data set into the wheat ear counting model trained in step 2 to obtain the wheat ear distribution density map and the wheat ear counting results;

[0039] Step 3.2, compare the wheat ear counting result obtained in step 3.1 with the actual number to verify the accuracy of the wheat ear counting model.

[0040] The present invention also discloses a field wheat ear density counting model based on local-global feature fusion, including a feature extraction network, a feature fusion network and a density regression network, a feature fusion network for multi-scale feature fusion and a density regression network for generating a density map, wherein:

[0041] The feature extraction network includes several CSPDarknet53 networks of different scales for multi-scale local feature extraction and a pyramid pooling Transformer network for global feature extraction. The CSPDarknet53 network outputs feature maps of different scales, and the smallest feature map is used as the input of the pyramid pooling Transformer network to extract global features.

[0042] The feature fusion network includes an image block encoder and a pyramid pooling Transformer network. The image block encoder adds position coding to the input feature map and then inputs it into the stacked pyramid pooling Transformer network to obtain a multi-scale feature map and relative position coding, calculates the attention of the feature map and obtains the final feature;

[0043] The density regression network includes three columns and a skip connection. Each column includes a hole convolution layer with different hole rates and an extended convolution layer. The output end of each convolution layer is connected to a batch normalization BN layer and a ReLU activation function. The output feature map of each column is connected and added using a skip connection. Finally, a 1×1 convolution layer is used to regress the density map.

[0044] The beneficial effects of the present invention are as follows: the field wheat ear density counting method based on local-global feature fusion extracts multi-scale local features of the image through the CSPDarknet53 network, and further extracts powerful context features with the help of the Pyramid Pooling Transformer network. In addition, the multi-scale features are fused using the attention mechanism, which enhances the representation ability of the feature map and improves the counting accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] Figure 1 A flowchart of an embodiment of the present invention;

[0046] Figure 2 (a) and (b) are respectively data images of an embodiment of the present invention and the density map and ear number predicted by the counting model. DETAILED DESCRIPTION

[0047] The present invention will be further described in detail below in conjunction with the accompanying drawings.

[0048] The present invention proposes a field wheat ear counting method based on local-global feature fusion, wherein the collected wheat RGB image is annotated with a point annotation method to mark the wheat ear area; the preprocessed image is input into a wheat ear counting model for training, and after the training, the wheat ear image can be directly counted, and the output is the number of wheat ears predicted by the model. The counting model combines CSPDarknet53 and Pyramid Pooling Transformer, and constructs a staged feature pyramid structure to generate feature maps of different scales, uses the network constructed by CSPDarknet53 as a network for extracting multi-scale local features in the model, and uses a multi-head self-attention Pyramid Pooling Transformer containing pyramid pooling as a network for extracting global features in the model. The feature information of each level is fused through a jump connection and an attention module. Finally, an effective feature map is generated, and a predicted density map is generated under the condition of retaining global information by means of convolution blocks with different void rates, and finally a field wheat ear counting method based on local-global feature fusion is realized.

[0049] like Figure 1 As shown, the following steps are included:

[0050] Step 1: Collect wheat RGB image datasets and annotate wheat ears by point annotation. Preprocess the collected datasets, generate true density maps using Gaussian functions, and divide the images into training and test sets. Specifically, the following steps are performed:

[0051] Step 1.1, collect different types of wheat RGB images from public datasets and construct a diversity dataset, including regional diversity, density diversity, light condition diversity, and variety diversity.

[0052] Step 1.2, the collected wheat data are uniformly adjusted to PNG format, and the data sets after the uniform format are cropped into a uniform size. In this embodiment, the processed data sets are cropped into a size of 1024*1024.

[0053] Step 1.3, rename each wheat ear image file according to the Arabic numeral 0000, use point annotation to mark the wheat ear area on the image data file, generate annotation information, use MATLAB tools to generate a mat file from the annotation information, each mat file contains the number of wheat ears and wheat ear position information of the original image. And the original image data file of the wheat ear corresponds to the annotation data file generated by the annotation.

[0054] Step 1.4, with the help of the wheat ear position information in the mat file, use a Gaussian function Gσ(x) to smooth the discrete density map generated by H(x), convert it into a continuous function, and generate the ground wheat ear real density distribution map F(x):

[0055] First, create an all-zero matrix x of the same size as the image. Based on the function δ(xp i )P i The positions are set to 1, resulting in a discrete density map.

[0056]

[0057]

[0058] Where pi is the position of the i-th wheat ear, H(x) is the generated discrete density map, σ represents the standard deviation, and in this example, 5 is set in the generated density map, and the resulting Gaussian density map has the property of being equal to the total number of wheat ears in the image.

[0059] In step 1.5, the preprocessed image and the generated density map are randomly flipped to randomly change the contrast, thereby achieving image augmentation to improve the generalization of the model. During the random flipping process, the image and the density map remain consistent.

[0060] Step 2: Build a field wheat ear density counting model based on local-global feature fusion, and train the model using the image data preprocessed by the wheat ear dataset in step 1 and the generated real density map. The feature extraction network uses CSPDarknet53 to extract local features and uses Pyramid Pooling Transformer to capture global context information. The feature extraction network includes a downsampling process, which trains the input image data to obtain multi-scale feature information; the obtained multi-scale feature information is based on the feature fusion network for multi-scale feature fusion, and the density map is generated based on the density regression network:

[0061] The image is first extracted through a local feature extraction network (4 CSPDarknet53 modules) to obtain 4 sets of feature maps of different scales. The feature map of the smallest scale is input into the Pyramid Pooling Transformer network, and the global features are extracted with the help of the multi-head pooling self-attention mechanism. Then, the feature maps of different scales are fused using the attention mechanism through jump connections to obtain an effective feature map. Finally, a density regression network is constructed with the help of dilated convolutions with different dilation rates to generate the final predicted density map. The feature extraction network structure disclosed in the present invention can extract and fuse local features and global features, effectively improving the detection accuracy.

[0062] The specific process includes:

[0063] Step 2.1, the input image is first normalized and preprocessed to scale the pixel values ​​to between 0 and 1. The preprocessed image file is input into the counting model. The image is an H*W*3 image, that is, an H*W pixel, 3-channel RGB image. First, a primary feature of size H / 2*W / 2*64 is extracted through a convolution block. The convolution block contains a convolution layer, a normalization layer, and an activation layer. The convolution kernel of the convolution layer is 3*3, the stride is 2, and the number of channels is 64. The primary feature is input into 4 groups of CSPDarknet53 networks to extract local features. Before each input, the feature map is first downsampled by the convolution layer, the normalization layer, and the activation layer. That is, each output will downsample the feature map by 1 / 2 and use it as the input of the next stage. Repeat in sequence to output four groups of feature maps of different sizes X1, X2, X3, and X4.

[0064] It should be noted that CSPDarknet53 divides the input feature map into two parts according to the number of channels. The first part performs convolution operation: after the product operation with a convolution kernel size of 3x3, BN normalization and SILU function, the feature map is output, and the other part is fused with the feature map after the convolution operation of the first part as the output of CSPDarknet53. Before each CSPDarknet53 module, the product operation with a convolution kernel size of 3x3, BN normalization and SILU function are used for scale downsampling to achieve multi-scale feature extraction. Therefore, the convolution, normalization and SILU function downsampling operations need to be repeated twice, the first time before the CSPDarknet53 module for scale downsampling; the second time in the CSPDarkne53 module for feature extraction.

[0065] Step 2.2: First, reshape the minimum size feature map X4 output in step 2.1. With the help of pyramid pooling multi-head self-attention, the length of the image feature sequence is reduced, and the Pyramid Pooling Transformer is used to further obtain global features. Specifically,

[0066] The feature map of (B, C, H, W) dimensions is reshaped into a feature map of (B, C, H*W), the flattened feature map is input into the image block encoder Patch Embedding, the learnable position encoding is added, and then input into the stacked pyramid pooling Transformer module:

[0067] p i =AvgPool i (x)

[0068]

[0069] Where P i is the feature map obtained by different pooling. DWConv(·) represents the depth convolution with a convolution kernel size of 3. i Perform convolution to obtain relative position encoding Then, these pyramid feature maps are flattened and concatenated to obtain the input sequence Calculate its attention to obtain the final feature A.

[0070]

[0071] In the formula, Q, K, and V represent the query, key, and value obtained by linear projection of sequence P, dk is the length of key vector K, and A is the global feature X5. X5 is a powerful context feature (global feature) further extracted based on X4 based on the Pyramid Pooling Transformer network, and its scale is the same as X4.

[0072] Step 2.3, use the attention fusion module to fuse the multi-scale feature information (X1, X2, X3 and X5) extracted in step 2.1 and step 2.2 to obtain a valid feature map. Specifically,

[0073] First, the extracted multi-scale features are upsampled using dilated convolution to obtain feature maps of the corresponding scale. Then, after adding the feature maps of the same scale, they are divided into two branches of different scales according to the channel dimension. One branch uses Global Avg Pooling to extract the channel attention of global features, and the other branch uses point-wise convolution to extract the channel attention of local features. Finally, the global channel context g(X) and the local channel context L(X) are combined to obtain the refined feature X'∈RC×H×W through MS-CAM:

[0074] G(x)=BN(PWConv2(δ(BN(PWconv1(GlobalPooling(X))))))

[0075] L(x)=BN(PWConv2(δ(BN(PWconv1(X)))))

[0076] X′=X⊕σ(L(X)⊕G(X))

[0077] Where GlobalPooling is the global average pooling operation, PWconv1 is a point convolution with a kernel size of 1, which is used to reduce the number of channels of the input feature X, BN represents BatchNorm normalization, δ is the ReLU activation function, and the number of channels is restored to the same as the original input channel number through PWConv2 point convolution, σ is the Sigmode activation function, the purpose of which is to map the variable to between 0 and 1.

[0078] In step 2.4, the fused effective features are passed into the density return network, and the density is generated and summed using dilated convolutions with different dilation rates to achieve wheat ear counting. In this example, three convolution blocks with different dilation rates are used to process features separately, preserving the context information of the image to the greatest extent.

[0079] Specifically,

[0080] The density regression network contains three columns (C1, C2, C3) and a skip connection. Each column consists of a convolution layer and an expanded convolution layer. The corresponding kernel size and expansion rate are set as small as possible. In this embodiment, dilated convolutions with dilation rates of 1, 2, 3 and convolution kernel sizes of 1, 3, and 5 are used for processing, which can expand the receptive field while including spatial resolution to adapt to high-density, small-scale wheat ear counting scenarios. Each convolution layer is followed by a batch normalization (BN) layer and a ReLU activation function. The output feature maps of each column are connected and added using skip connections to utilize multi-scale features. Finally, a 1×1 convolution layer is used to regress the density map to generate a density map.

[0081] Step 3: Keep the training model trained in step 2, input the labeled test data set, and test the model. Compare the counting results to obtain the optimal model. Use the counting model, input the wheat image, and count the number of wheat ears. The specific process includes:

[0082] Step 3.1 Based on the test data set, use the wheat ear counting model trained in step 2, input the images in the test set into the trained counting model, and compare the wheat ear distribution density map and the wheat ear counting results with the actual number to verify the accuracy of the wheat ear counting model.

[0083] Step 3.2 uses the selected optimal model, inputs the wheat ear image file into the counting model, detects and counts the number of wheat ears in the image, and generates a density map.

[0084] like Figure 2 As shown in (a)(b), the left side is the original RGB image of wheat ears and the actual number, and the right side is the density map and the number of wheat ears predicted by this counting model. It can be seen that the number of wheat ears obtained by this counting model is accurate.

[0085] The above embodiments are preferred implementation modes of the present invention, but the implementation modes of the present invention are not limited to the above embodiments. Any other changes, modifications, substitutions, combinations, and simplifications that do not deviate from the spirit and principles of the present invention are equivalent replacement methods and are included in the protection scope of the present invention.

Claims

1. A method for counting wheat ear density in a field based on local-global feature fusion, characterized in that: The following steps are involved: Step 1: Collect wheat RGB image datasets, unify pixel sizes, annotate wheat ears by point annotation, generate true density distribution maps using Gaussian functions, and divide images into training and test sets; Step 2, constructing and training a field wheat ear density counting model based on local-global feature fusion, the counting model includes a feature extraction network for extracting multi-scale features, a feature fusion network for multi-scale feature fusion and a density regression network for generating a density map, wherein the feature extraction network is a network structure combining a CSPDarknet53 network and a pyramid pooling Transformer network, and the CSPDarknet53 network of four different scales extracts multi-scale local features, and the extracted feature map of the smallest scale is input into the pyramid pooling Transformer network, and the global context information is captured and the global features are extracted based on the multi-head pooling self-attention mechanism, and then the feature fusion network of the jump connection is used to fuse the feature maps of different scales using the attention mechanism to obtain an effective feature map, and finally the density regression network is input, and the density regression network is constructed based on the void convolution of different void rates to generate the final predicted density map, specifically including, Step 2.1, the input image is first preprocessed and normalized to scale the pixel values ​​to between 0 and 1; the preprocessed image is input into the CSPDarknet53 network to extract feature maps of four different scales, denoted as X1, X2, X3, and X4; The CSPDarknet53 network divides the input feature map into two parts according to the number of channels, one part undergoes a product operation with a convolution kernel size of 3x3, BN normalization and SILU function, and the other part is directly connected to the output and connected to the first part; each CSPDarknet53 network first uses a product operation with a convolution kernel size of 3x3, BN normalization and SILU function to perform scale downsampling, and downsampling is used to train the input image data to obtain multi-scale feature information; Step 2.2, reshape the dimension of the minimum scale feature map X4 extracted in step 2.1, reduce the length of the image feature sequence with the help of pyramid pooling multi-head self-attention, and then input it into the pyramid pooling Transformer network to further extract the context global feature map X5; The dimension reshaping and global feature acquisition are specifically as follows: reshaping the feature map of (B, C, H, W) dimension into a feature map of (B, C, H*W) dimension, inputting the flattened feature map into the image block encoder Patch Embedding, adding position encoding and inputting into the stacked pyramid pooling Transformer module, and obtaining a multi-scale feature map P by applying pooling operations of different scales. i and relative position encoding It is expressed as: P i =AvgPool i (x) Where P i Represents the multi-scale feature map of the pooling tower generated by different pooling. The subscript i represents the i-th pooling layer, x is the input feature map, B, C, H, W, B is the number of input images, that is, the batch_size of the input data during network training, C is the number of channels of the image itself, H and W refer to the length and width of the image, AvgPool i is average pooling, DWConv(·) is depthwise convolution; The input sequence is obtained by flattening and concatenating the feature maps of different pools Calculate its attention to obtain the final feature X5, expressed as: In the formula, The pooled feature map P i The relative position encoding is obtained by deep convolution DWConv operation, LayerNorm is layer normalization, Concat is concatenation, Softmax is the normalization function, Q, K, V represent the query Query, key Key and value Value obtained by linear projection of sequence P, respectively, and dk is the length of key vector K; Step 2.3: Input the local feature maps X1, X2, X3 extracted in step 2.1 and the global feature map X5 extracted in step 2.2 into the feature fusion network based on the attention mechanism for multi-scale feature fusion to obtain the effective feature map. The fusion process is as follows: the extracted multi-scale features are upsampled using dilated convolution to obtain feature maps of the corresponding scale; then the feature maps of the same scale are added together and divided into two branches of different scales according to the channel dimension, one of which uses Global Avg Pooling to extract the channel attention of the global features, and the other uses point-wise convolution to extract the channel attention of the local features; finally, the global attention and channel attention are added after point multiplication to obtain the fused feature map, that is, the global channel context G(X) and the local channel context L(X) are combined to obtain the refined feature X'∈ R C×H×W It is expressed as: G(x)=BN(PWConv2(δ(BN(PWConv1(GlobalPooling(X)))))) L(x)=BN(PWConv2(δ(BN(PWConv1(X))))) Where GlobalPooling is the global average pooling operation, PWConv1 is a point convolution with a kernel size of 1, which is used to reduce the number of channels of the input feature X, BN is BatchNorm normalization, δ is the ReLU activation function, and the number of channels is restored to the same as the original input channel number through PWConv2 point convolution, σ is the Sigmode activation function, the purpose is to map the variable to between 0 and 1, is an XOR operation; Step 2.4, input the effective feature map into the density regression network to generate a predicted density map and perform density summation to realize wheat ear counting. The density regression network uses three convolution blocks with different void rates to process features separately, and concat the three obtained feature maps, and perform convolution Conv, normalization BatchNorm and activation function ReLU operations on the spliced ​​feature map to generate a density map; Step 3: input the test set into the trained counting model for testing, input the RGB image of wheat to be counted, and obtain the wheat ear distribution density map and the wheat ear counting result.

2. The method for counting wheat ears density in the field based on local-global feature fusion according to claim 1, characterized in that: The step 1 comprises the following steps: Step 1.1, collect images: collect different types of wheat RGB images from public datasets to build a diversity dataset; Step 1.2, format unification: convert the collected wheat image files into a unified format and crop them into a unified size; Step 1.3, data annotation: use point annotation to mark the wheat ear area on the image data file, and make the original image data file of the wheat ear correspond to the annotated data file generated by the annotation one by one; Step 1.4, density generation: obtain wheat ear position data from the labeled data file, and use the Gaussian kernel to convert the wheat ear position data into the actual density distribution map of the ground wheat ears; Step 1.5, data augmentation: randomly flip the processed image data file and density distribution file and randomly change the contrast to achieve image augmentation.

3. The method for counting wheat ears density in the field based on local-global feature fusion according to claim 2, characterized in that: The method for generating the real density distribution map is as follows: using the Gaussian function Gσ(x) to smooth the discrete density map generated by H(x), converting it into a continuous function, and generating the ground wheat ear real density distribution map F(x): In the formula, p i is the position of the ith wheat ear, H(x) is the generated discrete density map, σ represents the standard deviation, and the resulting Gaussian density map has the property that the sum is equal to the total number of wheat ears in the image, N is the total number of wheat ears in the image, and x represents the position variable in the image.

4. The method for counting wheat ears density in the field based on local-global feature fusion according to claim 1 or 3, characterized in that: The step 3 comprises the following steps: Step 3.1, input the test data set into the wheat ear counting model trained in step 2 to obtain the wheat ear distribution density map and the wheat ear counting results; Step 3.2, compare the wheat ear counting result obtained in step 3.1 with the actual number to verify the accuracy of the wheat ear counting model.

5. A counting model for the field wheat ear density counting method based on local-global feature fusion according to claim 1, characterized in that: It includes a feature extraction network, a feature fusion network and a density regression network, a feature fusion network for multi-scale feature fusion and a density regression network for generating a density map, wherein: The feature extraction network includes several CSPDarknet53 networks of different scales for multi-scale local feature extraction and a pyramid pooling Transformer network for global feature extraction. The CSPDarknet53 network outputs feature maps of different scales, and the smallest feature map is used as the input of the pyramid pooling Transformer network to extract global features. The feature fusion network includes an image block encoder and a pyramid pooling Transformer network. The image block encoder adds position coding to the input feature map and then inputs it into the stacked pyramid pooling Transformer network to obtain a multi-scale feature map and relative position coding, calculates the attention of the multi-scale feature map and obtains the final feature; The density regression network includes three columns and a skip connection. Each column includes a hole convolution layer with different hole rates and an extended convolution layer. The output end of each convolution layer is connected to a batch normalization BN layer and a ReLU activation function. The output feature map of each column is connected and added using a skip connection. Finally, a 1×1 convolution layer is used to regress the density map.

Images