Method and system for counting variety-independent plants

Abstract

The invention provides a variety-independent plant counting method and system. The method comprises the following steps: S1, segmenting a to-be-detected image and a plant prompt sample image into image blocks with fixed sizes; extracting features from the image blocks of the to-be-detected image and the plant prompt sample image by using a visual converter encoder, and generating a main feature map; s2, based on the enhanced features, performing local counting at the token level by adopting a multi-scale dynamic counter; s3, normalizing the redundancy counting graph by adopting an overlapping region weighted average strategy, and eliminating context redundancy; and S4, combining the local counting graph with the attention matching graph to generate an attention guiding prompt graph, and performing fusion up-sampling on the normalized local counting graph and the attention guiding prompt graph to generate a final high-resolution visual counting result graph. The plant image input method and device can adapt to plant image input of different types and different growth stages, support a unified counting framework and output format, and have high variety independence, accuracy and interpretability.

Description

Technical Field

[0001] The present application relates to the field of plant phenotyping technology, and in particular to a method and system for counting variety-independent plants. Background Art

[0002] Plant counting is a key technology in agricultural automation and plant phenotyping, widely used in tasks such as crop yield prediction, planting density assessment, and early disease identification. With the scale and sophistication of global agricultural production, the demand for plant counting is growing. However, traditional plant counting methods often rely on manual labor, which is inefficient and prone to errors. In recent years, with the rapid development of deep learning technology, computer vision-based plant counting methods have gradually become a research hotspot. These methods automatically analyze plant images to efficiently and accurately complete counting tasks, providing strong technical support for agricultural production.

[0003] With the continuous evolution of deep learning technology, the Vision Transformer (ViT), as a new neural network architecture, has been gradually applied to various computer vision tasks. Compared with traditional Convolutional Neural Networks (CNNs), the Vision Transformer has stronger global modeling capabilities and can capture long-range dependencies in images through a self-attention mechanism, making it suitable for handling challenging problems such as complex backgrounds and densely packed objects. In target counting tasks, the global perception characteristics of the Vision Transformer have also been gradually introduced into the research of Class-Agnostic Counting (CAC), providing a theoretical basis and technical support for estimating the number of targets across species and scenarios. Related work has shown that the integration of example-guided mechanisms, self-attention structures, and scale modeling strategies is expected to improve counting accuracy in non-rigid structures, high intra-class variation, and complex field environments.

[0004] In recent years, plant counting has become an important branch of deep counting research, and the academic community has proposed a variety of counting models for specific plant targets. In 2017, Lu et al. proposed TasselNet in their paper (Lu., et al. "TasselNet: Counting maize tassels in the wild via local counts regression network" Plant methods. 2017; 13: 1-17.), which was the first time that deep learning was applied to the task of counting maize tassels. A local regression strategy was used to achieve density estimation. In 2019, Xiong et al. proposed TasselNetV2 in their paper (Xiong., et al. "TasselNetV2: Counting maize tassels via local counts regression network " Plant methods. 2019; 15: 150), which introduced contextual information and a local region partitioning mechanism, significantly improving the counting performance in densely overlapping areas. In 2021, Lu et al. proposed TasselNetV3 in their paper (Lu., et al. "Counting dense objects in remote sensing images via hierarchical local regression network" Remote Sensing. 2021; 60:1–15 ), which combined hierarchical convolutional structure with feature pyramid technology to enhance the model's ability to recognize and interpret multi-scale targets. These models have promoted the development of plant image counting research, but they generally have the problem of insufficient variety independence, making it difficult to directly reuse across plant species or different growth stages. To address this problem, Lu et al. proposed the GMNet model in 2019 (Lu., et al. "Class-agnostic counting." Asian Conference on Computer Vision. 2019; 669-684), using CNN to extract features from images and examples and achieve category-independent counting through inner product matching.Subsequently, Shi et al. proposed BMNet+ in 2022 (Shi., et al. "Towards Class-Agnostic Counting via Similarity Learning." Conference on Computer Vision and Pattern Recognition. 2022; 9529-9538), which introduced a similarity-aware mechanism and a metric learning framework, further improving generalization capabilities. However, these methods still struggle with the non-rigid deformations and complex field disturbances associated with plant counting tasks, and further optimization is urgently needed to incorporate the characteristics of plant targets.

[0005] Therefore, although existing technologies have achieved certain success in specific plant counting tasks, they still have many shortcomings. First, existing methods are usually designed for specific plant species. When the target plant species changes, data needs to be collected again and the model needs to be trained. Secondly, plants themselves are dynamic systems. Their shape and appearance will change with the growth stage and environmental conditions. Existing methods perform poorly when dealing with non-rigid plant targets. In addition, existing technologies lack effective adaptability when dealing with plant instances of different scales and densities. In particular, in field environments, when the size and morphology of plants vary greatly with the growth stage, the counting accuracy drops significantly. Summary of the Invention

[0006] In view of this, the present application provides a species-independent plant counting method and system to solve the existing plant counting model. The problems of insufficient generalization ability and low counting accuracy caused by the dependence of plant types on specific species and non-rigid plant morphology are discussed.

[0007] To achieve the above objectives, the technical solutions adopted in this application are as follows: A species-independent plant counting method comprising: S1: Image block feature extraction and fusion: First, the image to be detected and the plant cue sample image are segmented into image blocks of fixed size. Then, an encoder is used to extract features from the image blocks of the image to be detected and the plant cue sample image to generate a main feature map. The similarity between the image blocks of the image to be detected and the image blocks of the plant cue sample image is dynamically calculated through the decoupled attention mechanism to generate a fused high-discriminative feature representation and an attention match map. The main feature map and the attention match map are then concatenated in the channel dimension. The multi-target response distortion caused by the softmax function is corrected through the width / height compensation map to form an enhanced feature map. The plant cue sample condition is then injected into the enhanced feature. S2: Dynamic Multi-Scale Block-Level Count Regression: Based on enhanced features, a multi-scale dynamic counter is used to perform local counting at the image block level. The multi-scale dynamic counter includes multiple counting heads, each corresponding to a plant instance of different scales or an image block of different sizes. The receptive field (block_size) and stride (output_stride) of each counting head can be dynamically adjusted according to the scale information of the target plant. Each counting head processes the enhanced features through average pooling layers and convolution layers; the redundant count map is regressed at the image block level; S3: Normalization: Use the overlapping area weighted average strategy to normalize the redundant count map and eliminate context redundancy to obtain a more accurate local count map; S4: Attention-guided interpretability generation: The local count map and the attention match map are combined to generate an attention-guided prompt map. The normalized local count map and the attention-guided prompt map are fused and upsampled to generate a final high-resolution visualization of the counting result map. The attention-guided prompt map is used to indicate the possible location and number of plants, further improving the interpretability of the counting results.

[0008] Optionally, the method further includes before step S1: Preprocessing: The collected images to be detected and plant tip sample images are uniformly standardized, including image size normalization and image enhancement methods to improve image quality to reduce the impact of environmental factors.

[0009] Optionally, in step S1, dynamically calculating the similarity between the image blocks of the image to be detected and the image blocks of the plant hint sample image through a decoupled attention mechanism to generate a fused high-discrimination feature representation and an attention match map specifically includes: The decoupled attention mechanism models the global contextual information between the image tokens to be detected through self-attention; and injects the plant hint sample image tokens as prior information into the image to be detected through cross-attention, generating a fused high-discrimination feature representation and attention matching map.

[0010] Optionally, the attention matching graph is decoupled into four parts:

[0011] in, represents the self-attention inside the image to be detected, Represents the cross-modal similarity between the image to be detected and the plant hint sample image, Indicates the reverse matching of the plant hint sample image to the image to be detected, Represents self-attention inside the sample image of plant cues.

[0012] Optionally, the step S2 of using a multi-scale dynamic counter to perform local counting at the token level specifically includes: According to the size information of the plant hint sample, an appropriate local counting branch is selected for density prediction; each local counting head corresponds to a set of different block sizes and output step sizes.

[0013] Optionally, the number of the multiple counting heads in step S2 is specifically three, and the corresponding image block resolution sizes are , and , where are the height and width of the input image block, is the resolution scaling factor of the counting head, the three counting heads correspond to three counting branches respectively, and the regression of the redundant counting map at the image block level specifically includes: S2.1: Counting branch 1: The corresponding image block resolution size is The counting head performs an average pooling operation on the enhanced feature map to reduce the spatial resolution, and extracts local features through a set of convolutional layers and normalized activation units to generate a local count map of the corresponding scale; S2.2: Counting branch 2: The corresponding image block resolution size is The counting head performs an average pooling operation on the enhanced feature map to reduce the spatial resolution, and extracts local features through a set of convolutional layers and normalized activation units to obtain feature representations within the mesoscale range and output a local count map; S2.3: Counting branch 3: The corresponding image block resolution size is The counting head performs an average pooling operation on the enhanced feature map to reduce the spatial resolution, and extracts local features through a set of convolutional layers and normalized activation units to adapt to the feature extraction requirements of larger-scale target instances, and outputs the corresponding count map.

[0014] S2.4: Dynamic routing: Based on the original width and height of the target instance in the plant hint sample image, calculate the interval to which its scale belongs, and use the routing mechanism to select the output result of the branch corresponding to the interval to which the scale belongs from the three counting branches.

[0015] Optionally, the dynamic routing process can be formally expressed as:

[0016] in, represents the encoder backbone network, Indicates the parameters of different counting heads, represents the size of the target, Indicates the preset size range, is a hard switching function, Indicates different counting heads.

[0017] Optionally, before step S4, the following steps are further included: Based on the bounding box of the target instance in the plant cue sample image, its width and height information are extracted, and a scale embedding vector is constructed. The scale embedding vector is dimensionally expanded and expanded to the spatial dimension of the feature map of the image to be detected through a broadcast mechanism. It is then spliced ​​with the image block features of the image to be detected in the channel dimension to provide scale prior information. The spliced ​​features are then fed into the multi-head decoupled attention module in the visual transformer structure to guide the attention distribution while maintaining the spatial structure.

[0018] Optionally, in step S4, fusing and upsampling the normalized local count map with the attention guidance prompt map to generate a final high-resolution visualized count result map specifically includes: S4.1: Bilinearly interpolate the local count map to match the resolution of the attention Figure 1 To; S4.2: Counting based on predictions Compensation factor for sample size with plant tips , where The length and width of the sample image token for the plant hint, The original length and width of the plant hint sample image are calculated respectively, and the number of key areas is calculated ; S4.3: Select the top most similarity in the attention matching map Image blocks, generate binary masks; S4.4: Performing a Hadamard product operation on the binary mask and the local count map to generate a density map or a detection mask, thereby providing an understandable count result; The final high-resolution visualization of the counting result map is usually displayed in two forms: one is a density map, which intuitively shows the density of plant distribution through color gradients; the other is a detection mask, which intuitively shows the position of each counted target by marking the center position or boundary of the target instance.

[0019] A variety-independent plant counting system comprising an encoder, a multi-scale dynamic counter, a normalizer, and a visualizer; Encoder: Used for image block feature extraction and fusion: First, the image to be detected and the plant cue sample image are segmented into image blocks of fixed size. Then, the encoder is used to extract features from the image blocks of the image to be detected and the plant cue sample image to generate a main feature map. The similarity between the image blocks of the image to be detected and the image blocks of the plant cue sample image is dynamically calculated through the decoupled attention mechanism to generate a fused high-discriminative feature representation and an attention match map. The main feature map and the attention match map are then spliced ​​in the channel dimension. The multi-target response distortion caused by the softmax function is corrected through the amplitude compensation map (Width / Height Compensation Map) to form an enhanced feature map, and the plant cue sample condition is injected into the enhanced feature. Multi-scale dynamic counter: Based on enhanced features, it uses a multi-scale dynamic counter to perform local counting at the image block level. The multi-scale dynamic counter includes multiple counting heads, each corresponding to plant instances of different scales or image blocks of different sizes. The receptive field (block_size) and stride (output_stride) of the counting head can be dynamically adjusted according to the scale information of the target plant. Each counting head processes enhanced features through average pooling layers and convolution layers; the multi-target response distortion caused by the softmax function is corrected through amplitude compensation mapping (width / height compensation map); and redundant count maps are regressed at the image block level. Normalizer: Used to normalize redundant count maps using the overlapping area weighted average strategy to eliminate context redundancy and obtain a more accurate local count map; Visualizer: For attention-guided interpretability generation: The local count map and the attention match map are combined to generate an attention-guided prompt map. The normalized local count map and the attention-guided prompt map are fused and upsampled to generate a final high-resolution visualization of the count results. The attention-guided prompt map is used to indicate the possible location and number of plants, further improving the interpretability of the count results.

[0020] Compared with the prior art, the present invention has the following advantages: 1. The system in this application can adapt to the input of plant images of different species and different growth stages, support a unified counting framework and output format, and have strong variety independence, accuracy and interpretability.

[0021] 2. During inference, this application automatically selects the corresponding branch for output based on the size of the plant prompt sample, which is suitable for counting tasks of multi-scale plant targets.

[0022] 3. This application achieves unified counting of different plant species without the need to retrain the model for specific plant species; and can adapt to plant targets of different scales and growth stages, solving the challenges of non-rigid plant shapes and high intra-class variability. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following is a brief introduction to the drawings required for use in the embodiments. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can be obtained based on these drawings without creative work.

[0024] Figure 1 This is a flow chart of a variety-independent plant counting method of the present application.

[0025] Figure 2 This is a framework diagram of the variety-independent plant counting system provided by this application.

[0026] Figure 3 This is a flow chart of the encoder and single decoder provided by this application.

[0027] Figure 4 This is the architecture diagram of the multi-scale perceptual plant counting system provided by this application.

[0028] Figure 5 This is the attention mechanism process diagram provided by this application.

[0029] Figure 6 This is an original picture of wheat in the field provided by this application.

[0030] Figure 7 This is an original picture of nuts provided by this application.

[0031] Figure 8 This is an original picture of grapes between branches provided by this application.

[0032] Figure 9 This application provides Figure 6 Annotated ground truth map of wheat.

[0033] Figure 10 This application provides Figure 7 Annotated ground truth map of nuts.

[0034] Figure 11 This application provides Figure 8 Annotated ground truth map of grapes.

[0035] Figure 12 Yes Figure 6Wheat counting results obtained using a common local counting algorithm.

[0036] Figure 13 Yes Figure 6 Wheat counting results obtained using the method of this application.

[0037] Figure 14 Yes Figure 7 A diagram showing the nut counting results obtained using the method of this application.

[0038] Figure 15 Yes Figure 8 A graph showing the grape counting results obtained using the method of this application.

[0039] Figure 16 This application provides a structural block diagram of a variety-independent plant counting system. DETAILED DESCRIPTION

[0040] In order to make the purpose, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below in combination with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments.

[0041] like Figure 1 and Figure 2 A variety-independent plant counting method is shown, comprising: S1: Image block feature extraction and fusion: First, the image to be detected and the plant cue sample image are segmented into image blocks of fixed size. Then, an encoder is used to extract features from the image blocks of the image to be detected and the plant cue sample image to generate a main feature map. The similarity between the image blocks of the image to be detected and the image blocks of the plant cue sample image is dynamically calculated through the decoupled attention mechanism to generate a fused high-discriminative feature representation and an attention match map. The main feature map and the attention match map are then concatenated in the channel dimension. The multi-target response distortion caused by the softmax function is corrected through the width / height compensation map to form an enhanced feature map. The plant cue sample condition is then injected into the enhanced feature. The image to be detected is an image for plant counting, and the plant prompt sample image is a single plant image. The image to be detected and the plant prompt sample image are divided into fixed block sizes, for example, each image is evenly divided into Each image patch is converted into an embedding vector of fixed dimension through linear mapping. The conversion process can be described as ,in, is the preset projection matrix, For bias.

[0042] The encoder first converts the image to be detected ( ) and plant tip sample images ( ) are generated as and The token sequence of the image to be detected and the token of the plant hint sample image are concatenated into a unified sequence, and after linear transformation, the query (Q), key (K) and value (V) are generated respectively. Under the multi-head self-attention mechanism, the formula is used:

[0043]

[0044] The attention weights are calculated, where Q, K, and V represent query, key, and value vectors respectively, and D is the feature dimension.

[0045] Then, the Vision Transformer (ViT) module in the encoder dynamically calculates the similarity between the image patch to be detected and the plant clue sample image patch through the self-attention mechanism, and generates an attention matching map, which specifically includes: The global contextual information between the image tokens to be detected is modeled through the self-attention mechanism; and the plant hint sample image tokens are injected into the image to be detected as prior information through the cross-attention mechanism to generate a fused high-discrimination feature representation and attention matching map.

[0046] like Figure 4 As shown ( Figure 2-Figure 5 The input image in is the image to be detected), and the attention matching map is then passed to the multi-scale dynamic counter, Figure 4 The architectural connection between the feature fusion module and the multi-scale dynamic counter is shown in detail.

[0047] like Figure 5 As shown in the figure, the global context information between the tokens of the image to be detected is first modeled through the self-attention mechanism. Then, the decoupled attention mechanism is used to treat the attention map calculated by the self-attention as a self-attention map and a cross-attention map calculated simultaneously: the self-attention part focuses on the internal dependencies between the tokens of the image to be detected, while the cross-attention part injects the prior information in the plant prompt sample image token into the image to be detected. By dynamically matching the feature similarity between the tokens of the image to be detected and the tokens of the plant prompt sample image, a fusion feature is generated. Specifically, the attention map is decoupled into four parts:

[0048]

[0049] in, represents the self-attention inside the image to be detected, Represents the cross-modal similarity between the image to be detected and the plant hint sample image, Indicates the reverse matching of the plant hint sample image to the image to be detected, Represents the self-attention within the plant cue sample image. In one specific embodiment, after N layers of stacking, the output fused feature sequence contains both the global semantics of the image to be detected and the feature information of the plant cue sample image. These fused features are passed to the multi-scale dynamic counter.

[0050] Optionally, the method further includes before step S1: Preprocessing: The collected images to be detected and plant tip sample images are uniformly standardized, including image size normalization and image enhancement methods to improve image quality to reduce the impact of environmental factors.

[0051] In the preprocessing stage, the collected images to be detected and the plant hint sample images are first standardized, such as Figure 6-8 As shown in the figure, the images to be tested include wheat in the field, nuts, and grapes between branches. The system normalizes the image size and improves the image quality through image enhancement methods to reduce the impact of environmental factors on the model. Specifically, the images to be tested are uniformly adjusted to pixels, while the plant tip sample image is resized to Pixels are normalized to ensure that all input data are consistent in size, brightness, and color. Next, image normalization algorithms are used (for example, normalizing pixel values ​​to the range [0, 1]) to eliminate differences under different lighting conditions.

[0052] S2: Dynamic Multi-Scale Token-Level Count Regression: Based on the enhanced features, a multi-scale dynamic counter is used to perform local counting at the token level. The multi-scale dynamic counter includes multiple counting heads, each corresponding to a plant instance of different scales or an image block of different sizes. The receptive field (block_size) and stride (output_stride) of the counting head can be dynamically adjusted according to the scale information of the target plant. Each counting head processes the enhanced features through average pooling layers and convolution layers; the redundant count map is regressed at the image block level; The multi-scale dynamic counter is used to make local density predictions on the fused feature maps output by the encoder, such as Figure 4 In a specific embodiment, the fusion feature output by the encoder is of size The token-level design uses image blocks as the minimum computational unit and optimizes context information fusion by constraining the block size to be an integer multiple of the step size.

[0053] Optionally, the step S2 of using a multi-scale dynamic counter to perform local counting at the token level specifically includes: According to the size information of the plant hint sample, an appropriate local counting branch is selected for density prediction; each local counting head corresponds to a set of different block sizes and output step sizes.

[0054] Optionally, the number of the multiple counting heads in step S2 is specifically three, and the corresponding image block resolution sizes are , and , the three counting heads correspond to three counting branches respectively, and the regression of the redundant counting map at the image block level specifically includes: S2.1: Counting branch 1: The corresponding image block resolution size is The counting head performs an average pooling operation on the enhanced feature map to reduce the spatial resolution, and extracts local features through a set of convolutional layers and normalized activation units to generate a local count map of the corresponding scale; S2.2: Counting branch 2: The corresponding image block resolution size is The counting head performs an average pooling operation on the enhanced feature map to reduce the spatial resolution, and extracts local features through a set of convolutional layers and normalized activation units to obtain feature representations within the mesoscale range and output a local count map; S2.3: Counting branch 3: The corresponding image block resolution size is The counting head performs an average pooling operation on the enhanced feature map to reduce the spatial resolution, and extracts local features through a set of convolutional layers and normalized activation units to adapt to the feature extraction requirements of larger-scale target instances, and outputs the corresponding count map.

[0055] S2.4: Dynamic routing: Based on the original width and height of the target instance in the plant hint sample image, calculate the interval to which its scale belongs, and use a gating mechanism to select the output result of the branch corresponding to the scale interval from the three counting branches.

[0056] Specifically, the image block resolution size of counting branch 1 is , which is suitable for crops in the seedling stage or with small instances, such as wheat and cotton; the image block resolution size of the counting branch 2 is , which is suitable for medium-sized plants, such as nuts; the image block size of counting branch 3 is , which is suitable for plants in the ripening period or with dense fruit ears, such as grapes or corn ears. In each branch, the input fusion feature map is first subjected to a 3×3 average pooling operation with a stride of 16 to convert The feature map is compressed into a local block feature with lower spatial resolution, which also plays the role of local information aggregation. Subsequently, the pooled features are sent to a 3×3 standard convolution layer to perform feature extraction and density regression in the local area, and output the local count map under the current branch. Finally, each branch will output a The three local counting branches mentioned above will perform forward reasoning simultaneously, but the system only uses the output of one of the branches as the final result. The system will calculate the original width and height information of the target in the sample image of the plant (through (Calculated) to determine the true perceived size of the target instance.

[0057] Optionally, the dynamic routing process can be formally expressed as:

[0058] in, represents the encoder backbone network, Indicates the parameters of different counting heads, represents the size of the target, Indicates the preset size range, is a hard switching function, Indicates different counting heads.

[0059] During the training phase, the system calculates the target scale and each preset block size ( , , ) and selects the branch input with the smallest distance as the training sample for forward propagation and backward updates. During the testing phase, the system directly selects the branch with the closest target size from the three branches based on the rules, a strategy known as "hard routing." This formula implements a dynamic counter selection mechanism based on target size priority. The final output, a local count map, comes from the selected branch and represents the local density estimate of the plant area at the current scale. This map is passed to the normalization module and further processed in conjunction with the attention matching map to generate a normalized density map at a uniform resolution.

[0060] S3: Normalization: Use the overlapping area weighted average strategy to normalize the redundant count map and eliminate context redundancy to obtain a more accurate local count map; The normalization module processes the redundant count maps output by the dynamic multi-scale local count module. Adaptive Gaussian smoothing is first used to smooth the redundant count maps. Then, a weighted average of the local count values ​​within each overlapping region is taken. After smoothing, the weighted average of the local count values ​​within each overlapping region is taken. This normalization process results in a standardized local count map that accurately reflects the actual plant population in each region.

[0061] S4: Attention-guided interpretability generation: The local count map and the attention match map are combined to generate an attention-guided prompt map. The normalized local count map and the attention-guided prompt map are fused and upsampled to generate a final high-resolution visualization of the counting result map. The attention-guided prompt map is used to indicate the possible location and number of plants, further improving the interpretability of the counting results.

[0062] Optionally, the final high-resolution visualization counting result map is usually displayed in two forms: one is a density map, which intuitively displays the density of plant distribution through color gradient; the other is a detection mask, which intuitively displays the position of each counting target by marking the center position or boundary of the target instance.

[0063] Optionally, in step S4, fusing and upsampling the normalized local count map with the attention guidance prompt map to generate a final high-resolution visualized count result map specifically includes: S4.1: Bilinearly interpolate the local count map to match the resolution of the attention Figure 1 To; The normalized count map is upsampled to the same resolution as the criss-cross attention map output by the encoder using bilinear interpolation, thus providing data of uniform size for the next module.

[0064] S4.2: Counting based on predictions Compensation factor for sample size with plant tips ( ), where The length and width of the sample image token for the plant hint, The original length and width of the plant hint sample image are calculated respectively, and the number of key areas is calculated ( ); S4.3: Select the top most similarity in the attention matching map Image blocks, generate binary masks; S4.4: Performing a Hadamard product operation on the binary mask and the local count map to generate a modified count map or detection mask, thereby providing an understandable counting result.

[0065] Among them, the calculation of the mask is based on the predicted count and the plant hint sample scale compensation factor to determine the key area, realizing the dual interpretation of instance positioning and density distribution.

[0066] The attention-guided cue map generation step uses the cross-attention information output by the encoder to further enhance the normalized count map.

[0067] like Figure 5 As shown, the cross-attention matching map is extracted from the decoupled attention mechanism, identifying the high similarity areas between the image block to be detected and the plant hint sample image block.

[0068] First, the regions with higher responses are extracted from the cross-attention matching map, and the similarity of each region is counted. The model naturally selects the top region with the highest response based on the similarity. Image blocks are used to generate binary dynamic masks. The value can be obtained by the current prediction count and scale compensation factor The product of is determined, that is The generated binary mask directly identifies the possible locations of plant targets. Subsequently, the mask is multiplied element-by-element with the upsampled normalized count map to obtain the output hint map H, which is expressed as , where U(R) represents the upsampled count map. This hint map not only visually displays the dense vegetation area with color gradient, but also marks the center or boundary of the target instance in the detection mode. The output of the hint map serves as the input of the scale-aware optimization of the next module. Figure 13 As shown, Figure 6 The original wheat field map is generated by the counting result map of the present application method, and the wheat instance locations are clearly marked; Figure 14 As shown, Figure 7 The counting result graph of the original nuts graph shows the distribution of nuts; Figure 15 As shown, Figure 8 The resulting count plot of the original grape image highlights dense areas of grape instances.

[0069] The visualization processing stage in this step fuses the normalized local count map with the attention guidance prompt map and upsamples it to generate the final high-resolution count result map. The training labels are derived from the ground truth of the PAC-105 dataset, such as Figure 9 As shown, Figure 6 The wheat true value map annotated with the original map of wheat in the field; Figure 10 As shown, Figure 7 The true value map of nuts annotated with the original map of nuts; Figure 11 As shown, Figure 8 The ground truth map of grapes with the original image of grapes annotated. The upsampling operator is used to restore the low-resolution count map to the same size as the original image. The output results are usually displayed in two forms: one is a density map, which visually shows the density of plant distribution through color gradients; the other is a detection mask, which visually shows the location of each counted target by marking the center position or boundary of the target instance. Figure 12 As shown, Figure 6 The counting result obtained by using the common local counting algorithm for the original wheat field image is relatively rough, making it difficult to distinguish the instance locations; Figure 13 As shown in the figure, the counting result map generated by the method of this application clearly marks the location of the wheat instance. Figure 14 and Figure 15 The counting results for nuts and grapes are shown separately, highlighting the fineness of the target distribution. The visualization module ultimately outputs the fused high-resolution image for direct observation and further analysis.

[0070] Optionally, before step S4, the following steps are further included: Based on the bounding box of the target instance in the plant cue sample image, its width and height information are extracted, and a scale embedding vector is constructed. The scale embedding vector is dimensionally expanded and expanded to the spatial dimension of the feature map of the image to be detected through a broadcast mechanism. It is then spliced ​​with the image block features of the image to be detected in the channel dimension to provide scale prior information. The spliced ​​features are then fed into the multi-head decoupled attention module in the visual transformer structure to guide the attention distribution while maintaining the spatial structure.

[0071] This step completes the scale-aware optimization function to compensate for the scale distortion caused by the fixed token division. The original size information of the target plant is extracted from the plant hint sample image, and a width compensation map and a height compensation map are generated respectively. Then, a scale embedding vector is formed by broadcast addition. This vector is concatenated with the original features of the image token to be detected and input into the decoupled attention layer to correct the scale distortion. Specifically, first, the original size information of the target plant (e.g. ), generate width compensation map and height compensation map respectively, and then form scale embedding vector S by broadcast addition. Concatenate S with the original feature F of the image token to be detected to obtain enhanced feature This concatenation operation allows the subsequent decoupled attention layer to utilize the additional scale information to fine-tune the features, automatically favoring the larger count branch when the object size is large, and the smaller count branch when the object is rich in detail. The enhanced features after scale-aware optimization are passed as input to the final visualization module.

[0072] In particular, the parameters in this application are set to image resolution , Plant Tips Sample Image Resolution , image block size , scaling factor , the number of visual converter modules . The scope of protection of this application is not limited to the specific parameter values ​​listed in the embodiments. Although specific parameter settings were used during the experiment to verify the feasibility and effectiveness of the present invention, the scope of protection of this application should cover all other parameter settings that can achieve the same or similar functions. Specifically, the parameter range involved in this application should not be understood as only the specific numerical values ​​used in the experiment, but should include all parameter combinations that are technically feasible and can achieve the purpose of the present invention.

[0073] When encoding a sample image of a plant cue, this application extracts the original width and height of the target instance, generates a scale embedding vector, expands it through a broadcast mechanism, and concatenates it with the image block features of the main image. Scale prior information is introduced during the converter encoding process to guide attention to areas that match the target size.

[0074] like Figure 16 As shown, a variety-independent plant counting system includes an encoder 110, a multi-scale dynamic counter 120, a normalizer 130 and a visualizer 140; Encoder 110: for image block feature extraction and fusion: First, the image to be detected and the plant cue sample image are segmented into image blocks of fixed size. Then, the encoder is used to extract features from the image blocks of the image to be detected and the plant cue sample image to generate a main feature map. The similarity between the image blocks of the image to be detected and the image blocks of the plant cue sample image is dynamically calculated through a decoupled attention mechanism to generate a fused high-discrimination feature representation and an attention match map. The main feature map and the attention match map are then concatenated in the channel dimension to form an enhanced feature map, and the plant cue sample condition is injected into the enhanced feature map. The image to be detected is an image for plant counting, and the plant hint sample image is a single plant image. The image to be detected and the plant hint sample image are divided into fixed block sizes, for example, each image is evenly divided into small blocks of 16×16 pixels. Each image block (patch) is converted into an embedding vector of fixed dimension through linear mapping. The conversion process can be described as ,in, is the preset projection matrix, For bias.

[0075] The encoder 110 first converts the image to be detected ( ) and plant tip sample images ( ) are generated as and The token sequence of the image to be detected and the token of the plant hint sample image are concatenated into a unified sequence, and after linear transformation, the query (Q), key (K) and value (V) are generated respectively. Under the multi-head self-attention mechanism, the formula is used:

[0076]

[0077] The attention weights are calculated, where Q, K, and V represent query, key, and value vectors respectively, and D is the feature dimension.

[0078] Then, the visual converter module in the encoder dynamically calculates the similarity between the image block to be detected and the plant clue sample image block through the self-attention mechanism, and generates an attention matching map, which specifically includes: The self-attention mechanism models the global contextual information between the image tokens to be detected; and through the cross-attention mechanism, the plant hint sample image tokens are injected into the image to be detected as prior information to generate a fused high-discrimination feature representation and attention matching map.

[0079] like Figure 4 As shown, the attention matching map is then passed to a multi-scale dynamic counter, Figure 4 The architectural connection between the feature fusion module and the multi-scale dynamic counter is shown in detail.

[0080] like Figure 5 As shown in the figure, the global context information between the tokens of the image to be detected is first modeled through the self-attention mechanism. Then, the decoupled attention mechanism is used to treat the attention map calculated by the self-attention as a self-attention map and a cross-attention map calculated simultaneously: the self-attention part focuses on the internal dependencies between the tokens of the image to be detected, while the cross-attention part injects the prior information in the plant prompt sample image token into the image to be detected. By dynamically matching the feature similarity between the tokens of the image to be detected and the tokens of the plant prompt sample image, a fusion feature is generated. Specifically, the attention map is decoupled into four parts:

[0081]

[0082] in, represents the self-attention inside the image to be detected, Represents the cross-modal similarity between the image to be detected and the plant hint sample image, Indicates the reverse matching of the plant hint sample image to the image to be detected, Represents the self-attention within the plant cue sample image. In one specific embodiment, after N layers of stacking, the output fused feature sequence contains both the global semantics of the image to be detected and the feature information of the plant target image. These fused features are passed to the multi-scale dynamic counter.

[0083] Multi-scale dynamic counter 120: Based on the enhanced features, the multi-scale dynamic counter 120 is used to perform local counting at the image block level. The multi-scale dynamic counter includes multiple counting heads, each corresponding to a plant instance of different scales or an image block of different sizes. The receptive field (block_size) and stride (output_stride) of the counting head can be dynamically adjusted according to the scale information of the target plant. Each counting head processes the enhanced features through average pooling layers and convolution layers; the multi-target response distortion caused by the softmax function is corrected through the amplitude compensation map (width / height compensation map); and the redundant count map is regressed at the image block level. The multi-scale dynamic counter 120 is used to perform local density prediction on the fusion feature map output by the encoder, such as Figure 4 In a specific embodiment, the fusion feature output by the encoder is of size 2D feature map of .

[0084] Optionally, the adopting of a multi-scale dynamic counter to perform local counting at the token level specifically includes: According to the size information of the plant hint sample, an appropriate local counting branch is selected for density prediction; each local counting head corresponds to a set of different block sizes and output step sizes.

[0085] Optionally, the number of the plurality of counting heads is specifically three, and the corresponding image block resolution sizes are , and , the three counting heads correspond to three counting branches respectively, and the regression of the redundant counting map at the image block level specifically includes: S2.1: Counting branch 1: The corresponding image block resolution size is The counting head performs an average pooling operation on the enhanced feature map to reduce the spatial resolution, and extracts local features through a set of convolutional layers and normalized activation units to generate a local count map of the corresponding scale; S2.2: Counting branch 2: The corresponding image block resolution size is The counting head performs an average pooling operation on the enhanced feature map to reduce the spatial resolution, and extracts local features through a set of convolutional layers and normalized activation units to obtain feature representations within the mesoscale range and output a local count map; S2.3: Counting branch 3: The corresponding image block resolution size is The counting head performs an average pooling operation on the enhanced feature map to reduce the spatial resolution, and extracts local features through a set of convolutional layers and normalized activation units to adapt to the feature extraction requirements of larger-scale target instances, and outputs the corresponding count map.

[0086] S2.4: Dynamic routing: Based on the original width and height of the target instance in the plant hint sample image, calculate the interval to which its scale belongs, and use a gating mechanism to select the output result of the branch corresponding to the scale interval from the three counting branches.

[0087] Specifically, the image block resolution size of counting branch 1 is , which is suitable for crops in the seedling stage or with small instances, such as wheat and cotton; the image block resolution size of the counting branch 2 is , which is suitable for medium-sized plants, such as nuts; the image block size of counting branch 3 is , which is suitable for plants in the ripening period or with dense fruit ears, such as grapes or corn ears. In each branch, the input fusion feature map is first subjected to a 3×3 average pooling operation with a stride of 16 to convert The feature map is compressed into a local block feature with lower spatial resolution, which also plays the role of local information aggregation. Subsequently, the pooled features are sent to a 3×3 standard convolution layer to perform feature extraction and density regression in the local area, and output the local count map under the current branch. Finally, each branch will output a The three local counting branches mentioned above will perform forward reasoning simultaneously, but the system only uses the output of one of the branches as the final result. The system will calculate the original width and height information of the target in the sample image of the plant (through (Calculated) to determine the true perceived size of the target instance.

[0088] Optionally, the dynamic routing process can be formally expressed as:

[0089] in, represents the encoder backbone network, Indicates the parameters of different counting heads, represents the size of the target, Indicates the preset size range, is a hard switching function, Indicates different counting heads.

[0090] During the training phase, the system calculates the target scale and each preset block size ( , , ) and selects the branch input with the smallest distance as the training sample for forward propagation and backward updates. During the testing phase, the system directly selects the branch with the closest target size from the three branches based on the rules, a strategy known as "hard routing." This formula implements a dynamic counter selection mechanism based on target size priority. The final output, a local count map, comes from the selected branch and represents the local density estimate of the plant area at the current scale. This map is passed to the normalization module and further processed in conjunction with the attention matching map to generate a normalized density map at a uniform resolution.

[0091] Normalizer 130: used to normalize the redundant count map using an overlapping area weighted average strategy to eliminate context redundancy and obtain a more accurate local count map; Normalizer 130 processes the redundant count map output by the dynamic multi-scale local count module. It first smoothes the redundant count map using adaptive Gaussian smoothing, then performs a weighted average of the local count values ​​within each overlapping region. After smoothing, the local count values ​​within each overlapping region are weighted averaged. This normalization process results in a standardized local count map that accurately reflects the actual plant population in each region.

[0092] Visualizer 140: Interpretable generation for attention guidance: The local count map and the attention match map are combined to generate an attention guidance prompt map, and the normalized local count map and the attention guidance prompt map are fused and upsampled to generate a final high-resolution visualization of the counting result map. The attention guidance prompt map is used to indicate the possible location and number of plants, further improving the interpretability of the counting results.

[0093] Optionally, the final high-resolution visualization counting result map is usually displayed in two forms: one is a density map, which intuitively displays the density of plant distribution through color gradient; the other is a detection mask, which intuitively displays the position of each counting target by marking the center position or boundary of the target instance.

[0094] Optionally, fusing and upsampling the normalized local count map with the attention guidance prompt map to generate a final high-resolution visualized count result map specifically includes: S4.1: Bilinearly interpolate the local count map to match the resolution of the attention Figure 1 To; The normalized count map is upsampled to the same resolution as the criss-cross attention map output by the encoder using bilinear interpolation, thus providing data of uniform size for the next module.

[0095] S4.2: Counting based on predictions Compensation factor for sample size with plant tips ( ), where The length and width of the sample image token for the plant hint, The original length and width of the plant hint sample image are calculated respectively, and the number of key areas is calculated ( ); S4.3: Select the top most similarity in the attention matching map Image blocks, generate binary masks; S4.4: Performing a Hadamard product operation on the binary mask and the local count map to generate a density map or a detection mask, thereby providing an understandable counting result.

[0096] Among them, the calculation of the mask is based on the predicted count and the plant hint sample scale compensation factor to determine the key area, realizing the dual interpretation of instance positioning and density distribution.

[0097] The attention-guided cue map generation step uses the cross-attention information output by the encoder to further enhance the normalized count map.

[0098] like Figure 5 As shown, the cross-attention matching map is extracted from the decoupled attention mechanism, identifying the high similarity areas between the image block to be detected and the plant hint sample image block.

[0099] First, the regions with higher responses are extracted from the cross-attention matching map, and the similarity of each region is counted. The model naturally selects the top region with the highest response based on the similarity. Image blocks are used to generate binary dynamic masks. The value can be obtained by the current prediction count and scale compensation factor The product of is determined, that is The generated binary mask directly identifies the possible locations of plant targets. Subsequently, the mask is multiplied element-by-element with the upsampled normalized count map to obtain the output hint map H, which is expressed as , where U(R) represents the upsampled count map. This hint map not only visually displays the dense vegetation area with color gradient, but also marks the center or boundary of the target instance in the detection mode. The output of the hint map serves as the input of the scale-aware optimization of the next module. Figure 13 As shown, Figure 6 The original wheat field map is generated by the counting result map of the present application method, and the wheat instance locations are clearly marked; Figure 14 As shown, Figure 7 The counting result graph of the original nuts graph shows the distribution of nuts; Figure 15 As shown, Figure 8The resulting count plot of the original grape image highlights dense areas of grape instances.

[0100] The visualizer 140 fuses and upsamples the normalized local count map with the attention guidance prompt map to generate the final high-resolution count result map. The training labels are derived from the ground truth of the PAC-105 dataset, such as Figure 9 As shown, Figure 6 The wheat true value map annotated with the original map of wheat in the field; Figure 10 As shown, Figure 7 The true value map of nuts annotated with the original map of nuts; Figure 11 As shown, Figure 8 The ground truth map of grapes with the original image of grapes annotated. The upsampling operator is used to restore the low-resolution count map to the same size as the original image. The output results are usually displayed in two forms: one is a density map, which visually shows the density of plant distribution through color gradients; the other is a detection mask, which visually shows the location of each counted target by marking the center position or boundary of the target instance. Figure 12 As shown, Figure 6 The counting result obtained by using the common local counting algorithm for the original wheat field image is relatively rough, making it difficult to distinguish the instance locations; Figure 13 As shown in the figure, the counting result map generated by the method of this application clearly marks the location of the wheat instance. Figure 14 and Figure 15 The counting results for nuts and grapes are shown separately, highlighting the fineness of the target distribution. The visualization module ultimately outputs the fused high-resolution image for direct observation and further analysis.

[0101] The system of this application can adapt to the input of plant images of different types and different growth stages, supports a unified counting framework and output format, and has strong variety independence, accuracy and interpretability.

[0102] The above are only specific embodiments of the present application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this application should be included in the scope of protection of this application. Therefore, the scope of protection of this application should be based on the scope of protection of the claims.

Claims

1. A method for counting plants regardless of species, characterized in that: include: S1: Image block feature extraction and fusion: First, the image to be detected and the plant cue sample image are segmented into image blocks of fixed size. Then, an encoder is used to extract features from the image blocks of the image to be detected and the plant cue sample image to generate a main feature map. The similarity between the image blocks of the image to be detected and the image blocks of the plant cue sample image is dynamically calculated through a decoupled attention mechanism to generate a fused high-discrimination feature representation and an attention matching map. The main feature map and the attention matching map are then concatenated in the channel dimension. The multi-target response distortion caused by the softmax function is corrected through amplitude compensation mapping to form an enhanced feature map. The plant cue sample condition is then injected into the enhanced feature. S2: Dynamic Multi-Scale Block-Level Count Regression: Based on the enhanced features, a multi-scale dynamic counter is used to perform local counting at the image block level. The multi-scale dynamic counter includes multiple counting heads, each corresponding to a plant instance of different scales or an image block of different sizes. The receptive field and step size of each counting head can be dynamically adjusted according to the scale information of the target plant. Each counting head processes the enhanced features through average pooling layers and convolution layers; the redundant count map is regressed at the image block level; S3： Normalization: The redundant count map is normalized using the weighted average strategy of overlapping areas to eliminate contextual redundancy and obtain a more accurate local count map. S4: Attention-guided interpretability generation: The local count map and the attention matching map are combined to generate an attention-guided prompt map. The normalized local count map and the attention-guided prompt map are fused and upsampled to generate a final high-resolution visual count result map. The attention-guided prompt map is used to indicate the possible location and number of plants, further improving the interpretability of the counting results.

2. A method for counting plants regardless of species according to claim 1, characterized in that: The method further comprises, before step S1: Preprocessing: The collected images to be detected and plant tip sample images are uniformly standardized, including image size normalization and image enhancement methods to improve image quality to reduce the impact of environmental factors.

3. A method for counting plants regardless of species according to claim 1 or 2, characterized in that: In step S1, the similarity between the image blocks of the image to be detected and the image blocks of the plant hint sample image is dynamically calculated by the decoupled attention mechanism to generate the fused high-discrimination feature representation and attention matching map, which specifically includes: The decoupled attention mechanism models the global contextual information between the image tokens to be detected through self-attention; and injects the plant hint sample image tokens as prior information into the image to be detected through cross-attention, generating a fused high-discrimination feature representation and attention matching map.

4. A method for counting plants regardless of species according to claim 3, characterized in that: The attention matching graph is decomposed into four parts: in, represents the self-attention inside the image to be detected, Represents the cross-modal similarity between the image to be detected and the plant hint sample image, Indicates the reverse matching of the plant hint sample image to the image to be detected, Represents self-attention inside the sample image of plant cues.

5. A method for counting plants regardless of species according to claim 4, characterized in that: The step S2 in which a multi-scale dynamic counter is used to perform local counting at the token level specifically includes: According to the size information of the plant hint sample, an appropriate local counting branch is selected for density prediction; each local counting head corresponds to a set of different block sizes and output step sizes.

6. A method for counting plants regardless of species according to claim 5, characterized in that: In step S2, there are three counting heads, each corresponding to an image block resolution of , and , where are the height and width of the input image block, is the resolution scaling factor of the counting head, the three counting heads correspond to three counting branches respectively, and the regression of the redundant counting map at the image block level specifically includes: S2.1: Counting branch 1: The corresponding image block resolution size is The counting head performs an average pooling operation on the enhanced feature map to reduce the spatial resolution, and extracts local features through a set of convolutional layers and normalized activation units to generate a local count map of the corresponding scale; S2.2: Counting branch 2: The corresponding image block resolution size is The counting head performs an average pooling operation on the enhanced feature map to reduce the spatial resolution, and extracts local features through a set of convolutional layers and normalized activation units to obtain feature representations within the mesoscale range and output a local count map; S2.3: Counting branch 3: The corresponding image block resolution size is The counting head performs an average pooling operation on the enhanced feature map to reduce the spatial resolution, and extracts local features through a set of convolutional layers and normalized activation units to adapt to the feature extraction requirements of larger-scale target instances, and outputs the corresponding count map; S2.4: Dynamic routing: Based on the original width and height of the target instance in the plant hint sample image, calculate the interval to which its scale belongs, and use the routing mechanism to select the output result of the branch corresponding to the interval to which the scale belongs from the three counting branches.

7. A method for counting plants regardless of species according to claim 6, characterized in that: The dynamic routing process can be formally expressed as follows: in, represents the encoder backbone network, Indicates the parameters of different counting heads, represents the size of the target, Indicates the preset size range, is a hard switching function, Indicates different counting heads.

8. A method for counting plants regardless of species according to claim 7, characterized in that: Before step S4, the method further includes: Based on the bounding box of the target instance in the plant cue sample image, its width and height information are extracted, and a scale embedding vector is constructed. The scale embedding vector is dimensionally expanded and expanded to the spatial dimension of the feature map of the image to be detected through a broadcast mechanism. It is then spliced ​​with the image block features of the image to be detected in the channel dimension to provide scale prior information. The spliced ​​features are then fed into the multi-head decoupled attention module in the visual transformer structure to guide the attention distribution while maintaining the spatial structure.

9. A method for counting plants regardless of species according to claim 8, characterized in that: In step S4, the normalized local count map and the attention guidance prompt map are fused and upsampled to generate the final high-resolution visual count result map, which specifically includes: S4.1: Perform bilinear interpolation on the local count map to make its resolution consistent with the attention matching map; S4.2: Counting based on predictions Compensation factor for sample size with plant tips , where The length and width of the sample image token for the plant hint, The original length and width of the plant hint sample image are calculated respectively, and the number of key areas is calculated ; S4.3: Select the top most similarity in the attention matching map Image blocks, generate binary masks; S4.4: Performing a Hadamard product operation on the binary mask and the local count map to generate a density map or a detection mask, thereby providing an understandable count result; The final high-resolution visualization of the counting result map is usually displayed in two forms: one is a density map, which intuitively shows the density of plant distribution through color gradients; the other is a detection mask, which intuitively shows the position of each counted target by marking the center position or boundary of the target instance.

10. A species-independent plant counting system, characterized in that: Includes encoder, multi-scale dynamic counter, normalizer and visualizer; Encoder: for image block feature extraction and fusion: First, the image to be detected and the plant cue sample image are segmented into image blocks of fixed size; then, the encoder is used to extract features from the image blocks of the image to be detected and the plant cue sample image to generate a main feature map. The similarity between the image blocks of the image to be detected and the image blocks of the plant cue sample image is dynamically calculated through a decoupled attention mechanism to generate a fused high-discrimination feature representation and an attention matching map. The main feature map and the attention matching map are then spliced ​​in the channel dimension. The multi-target response distortion caused by the softmax function is corrected through amplitude compensation mapping to form an enhanced feature map, and the plant cue sample condition is injected into the enhanced feature. Multi-scale dynamic counter: Based on the enhanced features, a multi-scale dynamic counter is used to perform local counting at the image block level. The multi-scale dynamic counter includes multiple counting heads, each corresponding to a plant instance of different scales or an image block of different sizes. The receptive field and step size of the counting head can be dynamically adjusted according to the scale information of the target plant. Each counting head processes the enhanced features through average pooling layers and convolution layers; Correct the multi-target response distortion caused by the softmax function through amplitude compensation mapping; regress the redundant count map at the image block level; Normalizer: Used to normalize redundant count maps using the overlapping area weighted average strategy to eliminate context redundancy and obtain a more accurate local count map; Visualizer: For attention-guided interpretability generation: The local count map and the attention matching map are combined to generate an attention-guided prompt map. The normalized local count map and the attention-guided prompt map are fused and upsampled to generate a final high-resolution visualization of the counting result map. The attention-guided prompt map is used to indicate the possible location and number of plants, further improving the interpretability of the counting results.

Images