Intelligent wheat ear counting method based on computer vision

Abstract

The invention relates to the field of computer vision, and particularly discloses an intelligent wheat ear counting method based on computer vision, which does not adopt traditional bounding box detection, but extracts multi-scale features, and synchronously predicts a key point thermodynamic diagram, an offset prediction diagram and an instance embedding diagram through a multi-task prediction head. Thirdly, introducing a self-adaptive key point decoding mechanism, fusing thermodynamic diagram response and neighborhood semantic consistency, dynamically adjusting the confidence coefficient, inhibiting a false peak and recalling a real key point of a shielding region; then, performing adaptive clustering by using instance embedding vectors of the key points, and aggregating the key points belonging to the same wheat ear instance into one cluster; and finally counting the total number of effective clusters to realize accurate counting. According to the scheme, the instance is reconstructed through the detection component, and the problem of inaccurate counting of a bounding box method in a dense adhesion scene is solved.

Description

Technical Field

[0001] This application relates to the field of computer vision, and more specifically, to a computer vision-based intelligent counting method for wheat ears. Background Technology

[0002] In agricultural production and crop breeding, accurate wheat yield forecasting is of great significance. The number of wheat ears is one of the core elements constituting yield; therefore, rapid, automatic, and accurate counting of wheat ears in the field is a key link in achieving precision agricultural management and efficient breeding selection. Traditional manual field counting methods are not only time-consuming, labor-intensive, and highly subjective, but also often destructive, making them unsuitable for large-scale, high-throughput applications.

[0003] In real-world field environments, especially in the later stages of wheat growth or in high-yield, densely planted experimental fields, severe shading and dense clustering of wheat ears are common. In such complex scenarios, existing bounding box-based target detection methods reveal inherent technical flaws. First, these methods typically rely on non-maximum suppression (NMS) as a post-processing step to remove overlapping redundant detection boxes. The NMS mechanism makes decisions by calculating the intersection-union ratio (IoU) between boxes. When two or more truly independent wheat ears are physically close together or severely overlapping, the IoU values ​​of their respective true bounding boxes can be very high. This leads to a high degree of overlap in the effective detection boxes predicted by the model, making it easy for the NMS mechanism to misclassify one or more lower-scoring true detection boxes as redundant and suppress them, resulting in significant missed detections. Second, a bounding box is a coarse-grained rectangular region representation that only defines the outer contour of the target but loses information about the instances within the region. When multiple ears of wheat are stuck together, a single bounding box can enclose them all. The model has difficulty distinguishing from the mixed features whether the bounding box contains an irregularly shaped ear of wheat or a collection of multiple ears of wheat, resulting in insufficient representation ability.

[0004] Therefore, an optimized intelligent counting scheme for wheat ears is desired. Summary of the Invention

[0005] To address the aforementioned technical problems, this application is proposed. Embodiments of this application provide a computer vision-based intelligent counting method for wheat ears, comprising: Acquire raw images of wheat ears captured by camera equipment; Image feature extraction is performed on the original image of wheat ears to obtain a visual multi-scale feature map of wheat ears; The visual multi-scale feature map of wheat ears is input into the multi-task joint prediction head to obtain key point heatmap, instance embedding map and offset prediction map; Keypoints are decoded and instance vectors are extracted from the keypoint heatmap, instance embedding map and offset prediction map to obtain a list of keypoints. Adaptive clustering based on instance embedding is performed on the list of key points to obtain wheat ear instance clusters; The total number of wheat ears is obtained by counting based on clusters of wheat ear instances.

[0006] Compared with existing technologies, this application provides a computer vision-based intelligent wheat ear counting method that does not employ the traditional bounding box detection paradigm. Instead, it extracts multi-scale features through a deep network and simultaneously predicts the location heatmap of key points on the wheat ear, the offset prediction map for precise localization, and the instance embedding map for distinguishing instances via a multi-task prediction head. To address the problem of inaccurate key point detection in complex environments, this scheme introduces an adaptive key point decoding mechanism. This mechanism integrates the response intensity of the key point heatmap with the local neighborhood semantic consistency information of the instance embedding map, dynamically adjusting the confidence of each candidate key point. This effectively recalls true key points in occluded or poorly lit areas while suppressing false peak noise. After obtaining a list of high-confidence key points, adaptive clustering is performed in the feature space using the instance embedding vector carried by each key point. In this way, multiple key points belonging to the same wheat ear instance are aggregated into the same cluster, while key points of different instances are separated. Ultimately, the accurate counting of wheat ears was achieved by counting the total number of effective clusters, thus solving the problem of missed detections and inaccurate counting caused by non-maximum suppression and fuzzy representation in densely clustered scenarios using traditional bounding box methods. Attached Figure Description

[0007] The above and other objects, features, and advantages of this application will become more apparent from the more detailed description of the embodiments of this application in conjunction with the accompanying drawings. The drawings are provided to further illustrate the embodiments of this application and form part of the specification. They are used together with the embodiments of this application to explain this application and do not constitute a limitation thereof. In the drawings, the same reference numerals generally represent the same components or steps.

[0008] Figure 1 This is a flowchart of a computer vision-based intelligent wheat ear counting method according to an embodiment of this application; Figure 2 This is a schematic diagram of the data flow of a computer vision-based intelligent wheat ear counting method according to an embodiment of this application; Figure 3 This is a flowchart illustrating the process of decoding key points and extracting instance vectors from a key point heatmap, instance embedding map, and offset prediction map to obtain a key point list, according to an embodiment of the computer vision-based intelligent wheat ear counting method of this application. Figure 4This is a flowchart illustrating the process of performing neighborhood embedding consistency calculation and confidence dynamic adjustment on a candidate keypoint list to obtain an adjusted keypoint list based on an instance embedding graph of the intelligent wheat ear counting method based on computer vision according to an embodiment of this application. Figure 5 This is a flowchart illustrating the process of using a computer vision-based intelligent wheat ear counting method according to an embodiment of this application to perform adaptive clustering of a list of key points based on instance embedding to obtain wheat ear instance clusters. Detailed Implementation

[0009] Hereinafter, exemplary embodiments according to this application will be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of this application, and not all embodiments of this application. It should be understood that this application is not limited to the exemplary embodiments described herein.

[0010] As indicated in this application and claims, unless the context clearly indicates otherwise, the words "a," "an," "an," and / or "the" are not specifically singular and may include plural forms. Generally speaking, the terms "comprising" and "including" only indicate the inclusion of explicitly identified steps and elements, which do not constitute an exclusive list, and the method or apparatus may also include other steps or elements.

[0011] While this application makes various references to certain modules of the systems according to embodiments of this application, any number of different modules can be used and run on user terminals and / or servers. The modules described are merely illustrative, and different aspects of the systems and methods may use different modules.

[0012] Flowcharts are used in this application to illustrate the operations performed by the system according to embodiments of this application. It should be understood that the preceding or following operations are not necessarily performed in exact order. Instead, various steps can be processed in reverse order or simultaneously as needed. Furthermore, other operations can be added to these processes, or one or more steps can be removed from them.

[0013] Existing computer vision-based wheat ear counting methods suffer from insufficient counting accuracy, especially in high-density planting and severely occluded scenarios. This is mainly because traditional bounding box-based detection methods, with their non-maximum suppression post-processing mechanism, are prone to misclassifying closely packed independent wheat ears as redundant detections. Furthermore, the bounding box itself is insufficient to represent multiple instances clustered together, leading to missed detections. Therefore, this application proposes a computer vision-based intelligent wheat ear counting method. This method circumvents the shortcomings of traditional bounding box detection by detecting key components of the wheat ear and reconstructing instances. The method first acquires raw images of wheat ears captured by a camera and then extracts features from the raw images using a convolutional neural network backbone with a feature pyramid network to obtain a multi-scale visual feature map of the wheat ears. Next, this multi-scale feature map is input in parallel into a multi-task joint prediction head, simultaneously outputting a keypoint heatmap, an instance embedding map, and an offset prediction map. Subsequently, a preliminary list of candidate keypoints is determined through maximum value filtering and a lower confidence threshold. Neighborhood embedding consistency calculation and dynamic confidence adjustment are then performed on this candidate keypoint list. Using this score and the original confidence, a dynamically adjusted final confidence is calculated. Next, based on the adjusted final confidence and combined with the offset prediction map, final screening and information integration are performed to obtain a high-confidence keypoint list. After obtaining this list, the scheme performs adaptive clustering based on instance embedding. This clustering process constructs a cross-reachability distance matrix based on the instance embedding vectors of each keypoint in the keypoint list, and performs hierarchical structure agglomeration and stable cluster extraction on this matrix to obtain a cluster label list. Finally, the keypoints are mapped and aggregated based on the cluster label list, grouping keypoints belonging to the same wheat ear instance into the same wheat ear instance cluster. Finally, the total number of wheat ears is obtained by enumerating instances and counting the total number of wheat ear instance clusters.

[0014] Figure 1 This is a flowchart of a computer vision-based intelligent counting method for wheat ears according to an embodiment of this application. Figure 2 This is a schematic diagram of the data flow of a computer vision-based intelligent wheat ear counting method according to an embodiment of this application. Figure 1 and Figure 2As shown, the intelligent wheat ear counting method based on computer vision according to an embodiment of this application includes: S100, acquiring an original image of wheat ears captured by a camera device; S200, extracting image features from the original wheat ear image to obtain a multi-scale visual feature map of wheat ears; S300, inputting the multi-scale visual feature map of wheat ears into a multi-task joint prediction head to obtain a keypoint heatmap, an instance embedding map, and an offset prediction map; S400, performing keypoint decoding and instance vector extraction on the keypoint heatmap, instance embedding map, and offset prediction map to obtain a keypoint list; S500, performing adaptive clustering based on instance embedding on the keypoint list to obtain wheat ear instance clusters; S600, counting based on the wheat ear instance clusters to obtain the total number of wheat ears.

[0015] Specifically, in step S100, raw images of wheat ears are acquired using a camera device. It is understood that traditional manual field counting methods are inefficient, subjective, and unable to meet the needs of large-scale, high-throughput wheat ear surveys, necessitating the use of objective visual data as the analysis medium. Therefore, in the technical solution of this application, raw images of wheat ears are acquired using a camera device to capture the actual spatial distribution, morphological appearance, and dense clustering and occlusion relationships of the wheat ears in the field environment. This provides the necessary data input and analytical foundation for subsequent intelligent analysis steps such as feature extraction based on computer vision, key point detection, and instance clustering.

[0016] More specifically, in a particular example of this application, the acquisition process is achieved through drone aerial photography. First, a drone equipped with a high-resolution visible light camera is deployed over the target wheat field, and an autonomous flight path covering the area to be measured is planned. Then, camera parameters are set to continuously capture images at a fixed flight altitude of approximately 10 meters above the canopy and a fixed overlap rate of 80% in the forward direction and 70% in the lateral direction, ensuring effective coverage between images. After acquisition, the raw image data from the camera's storage medium, such as in JPEG or TIFF format, is exported and transmitted to a data processing terminal to form a raw image dataset of wheat ears.

[0017] Specifically, in step S200, image features are extracted from the original wheat ear image to obtain a multi-scale visual feature map of the wheat ear. It is understood that the original wheat ear image only contains low-level pixel information, lacking the high-level semantic information needed to distinguish key points of the ear from complex backgrounds (such as leaves and soil). Furthermore, the wheat ear target exhibits significant multi-scale characteristics in different scenarios such as aerial or close-up field photography. Therefore, in the technical solution of this application, image features are further extracted from the original wheat ear image to obtain a multi-scale visual feature map of the wheat ear. This allows a deep convolutional neural network to automatically learn and construct a hierarchical feature representation that includes everything from low-level texture details to high-level semantic concepts, ensuring that this feature representation can uniformly capture target information at different scales. This provides a high-resolution location information and rich semantic information input for the subsequent multi-task joint prediction head, enabling it to effectively detect and distinguish key points of wheat ears of different sizes, laying the foundation for addressing the challenges of multi-scale target recognition.

[0018] More specifically, in a specific example of this application, image feature extraction is performed on the original image of wheat ears to obtain a visual multi-scale feature map of wheat ears, including: normalizing the size and pixel value of the original image of wheat ears to obtain a preprocessed image of wheat ears; and inputting the preprocessed image of wheat ears into the backbone of a convolutional neural network with a feature pyramid network to obtain a visual multi-scale feature map of wheat ears.

[0019] Accordingly, the original wheat ear images are normalized in size and pixel value to obtain preprocessed wheat ear images. It is understood that original wheat ear images acquired by camera devices often have different image resolutions and pixel value ranges, such as 0 to 255, while the subsequent convolutional neural network backbone has fixed requirements for the dimension and numerical distribution of the input data. Therefore, in the technical solution of this application, the original wheat ear images are further normalized in size and pixel value to obtain preprocessed wheat ear images, thereby converting images from different sources into a standardized data format that meets the model input requirements. This ensures that the subsequent feature extraction model receives input with consistent dimension and stable numerical range, guaranteeing the effectiveness of model processing and the stability of convergence.

[0020] More specifically, in a concrete example of this application, the preprocessing procedure comprises two sub-steps. First, size normalization is performed. For an original image, for example, 4000x3000 pixels, a bilinear interpolation algorithm is used to resample it to an image of 1024x768 pixels. This operation maintains the original aspect ratio of the image while unifying the input image to the fixed input size required by the subsequent backbone network. Second, pixel value normalization is performed. Each pixel value of the size-normalized image is divided by 255.0, linearly scaling its data range from 0 to 255 to between 0 and 1. Alternatively, the mean of the RGB channels of a specific dataset can be subtracted and divided by the corresponding standard deviation to complete data standardization. The final generated image is the preprocessed wheat ear image.

[0021] Accordingly, the preprocessed wheat ear image is input into the backbone of a convolutional neural network with a feature pyramid network to obtain a multi-scale visual feature map of the wheat ear. It is understandable that the preprocessed wheat ear image only contains raw pixel information and lacks the high-level semantic features needed to distinguish key points of the ear. Furthermore, wheat ears in field-collected images often exhibit multi-scale characteristics; for example, ears in close-up are large while ears in distant or aerial photographs are small. Therefore, in the technical solution of this application, the preprocessed wheat ear image is further input into the backbone of a convolutional neural network with a feature pyramid network to automatically learn and extract hierarchical features from low-level texture to high-level semantics in the image, and the feature pyramid network structure is used to fuse feature information from different levels. This generates a series of multi-scale visual feature maps that combine high-resolution detail information and rich semantic information, providing a high-quality feature foundation for subsequent multi-task prediction heads to accurately identify and locate key points of wheat ears at different scales.

[0022] More specifically, in a concrete example of this application, firstly, a ResNet-50 network pre-trained on the large ImageNet dataset is selected as the backbone of the convolutional neural network. The preprocessed wheat ear image generated in the previous step is input into this backbone network for forward propagation. During propagation, feature maps output from different stages of the backbone network (e.g., layers C2, C3, C4, and C5) are extracted, and these feature maps have different spatial resolutions and semantic strengths. Subsequently, the feature pyramid network fusion process is initiated, which includes a top-down path and a lateral connection. In the top-down path, the high-level semantic features of layer C5 are first subjected to channel dimensionality reduction through 1x1 convolution, and then their spatial resolution is increased to be consistent with that of layer C4 through upsampling. At the same time, the feature map of layer C4 is also subjected to channel dimensionality reduction through lateral connections. Next, the upsampled C5 feature map and the dimensionality-reduced C4 feature map are fused element-wise, and the fused feature map is then smoothed through 3x3 convolution to obtain the P4 feature map. This fusion process is repeated sequentially between P4 and C3, and P3 and C2, ultimately outputting a set of multi-scale feature maps (e.g., P2, P3, P4, P5) with the same number of channels but different spatial resolutions. This set of feature maps is the multi-scale visual feature map of wheat ears.

[0023] Specifically, in step S300, the visual multi-scale feature map of wheat ears is input into a multi-task joint prediction head to obtain a keypoint heatmap, an instance embedding map, and an offset prediction map. It is understood that the keypoint information of the wheat ears decoded from the multi-scale feature map needs to include not only its approximate position confidence, but also identity information to distinguish different wheat ear instances in densely clustered scenes, and precise coordinate information to correct positional deviations caused by downsampling. These three tasks are interrelated but functionally independent. Therefore, in the technical solution of this application, the visual multi-scale feature map of wheat ears is further input into a multi-task joint prediction head to obtain a keypoint heatmap, an instance embedding map, and an offset prediction map, thereby utilizing shared deep features to predict these three key component information for subsequent instance reconstruction in parallel and specifically. In this way, within a unified framework, we can simultaneously obtain a probability map representing the probability of the existence of key points, a feature vector map representing the instance identity of each key point, and a position correction map representing sub-pixel precision, providing complete and decoupled data support for subsequent high-precision key point decoding and instance vector extraction, as well as adaptive clustering based on instance embedding.

[0024] More specifically, in a specific example of this application, inputting the visual multi-scale feature map of wheat ears into a multi-task joint prediction head to obtain a keypoint heatmap, an instance embedding map, and an offset prediction map includes: inputting the visual multi-scale feature map of wheat ears into a keypoint prediction head to obtain a keypoint heatmap; inputting the visual multi-scale feature map of wheat ears into an instance embedding prediction head to obtain an instance embedding map; and inputting the visual multi-scale feature map of wheat ears into an offset prediction head to obtain an offset prediction map.

[0025] Accordingly, the visual multi-scale feature map of wheat ears is input into the keypoint prediction head to obtain a keypoint heatmap. It is understandable that while the visual multi-scale feature map of wheat ears contains rich semantic information, a dedicated decoder is needed to convert it into spatial location predictions of wheat ear keypoints. Therefore, in the technical solution of this application, the visual multi-scale feature map of wheat ears is further input into the keypoint prediction head to obtain a keypoint heatmap. This allows for the specialized learning of a spatial mapping from features to the probability of keypoint presence, based on shared features. This generates a probability map spatially aligned with the input image (or its downsampled version), where the response value of each pixel represents the confidence level of the presence of a wheat ear keypoint at that location, providing a direct basis for subsequent keypoint peak location.

[0026] More specifically, in a concrete example of this application, firstly, all feature maps in the visual multi-scale feature map set generated in the previous step are upsampled to the same spatial resolution using bilinear interpolation, for example, unified to the size of the highest resolution feature map P2. Then, they are concatenated along the channel dimension to form an aggregated multi-scale feature map. Subsequently, this aggregated feature map is input into a keypoint prediction head. This prediction head consists of four stacked 3x3 convolutional layers, each followed by a ReLU activation function, used for deep fusion of multi-scale information and extraction of keypoint features. After the last convolutional layer, a 1x1 convolutional layer is connected, which reduces the number of channels in the feature map to 1. Finally, a Sigmoid activation function is applied to this 1-channel feature map, normalizing its output value to the range of 0 to 1, thereby generating the final keypoint heatmap.

[0027] Accordingly, the visual multi-scale feature map of wheat ears is input into the instance embedding prediction head to obtain an instance embedding map. It is understandable that since the keypoint heatmap only provides the positional probability information of keypoints on the wheat ear but does not provide identification identifiers to distinguish different wheat ear instances, this can lead to the inability to correctly aggregate keypoints belonging to the same instance in densely clustered scenes. Therefore, in the technical solution of this application, the visual multi-scale feature map of wheat ears is further input into the instance embedding prediction head to obtain an instance embedding map, thereby specifically learning a high-dimensional feature space. In this space, the embedding vectors of pixels and their keypoints belonging to the same wheat ear instance are close to each other in feature distance, while the embedding vectors of different wheat ear instances are far apart in feature distance. This generates an instance embedding map aligned with the keypoint heatmap space, where the vector stored at each location provides the instance identity information to which that location belongs, providing a discriminative basis for subsequent adaptive clustering based on instance embedding.

[0028] More specifically, in a concrete example of this application, this processing flow is executed in parallel with the keypoint prediction head. First, an aggregated multi-scale feature map, identical to the input of the keypoint prediction head, is obtained. Then, this aggregated feature map is input to the instance embedding prediction head. This prediction head also consists of four stacked 3x3 convolutional layers with ReLU activation functions, used for deep extraction of instance discriminative features. After this convolutional stack, a 1x1 convolutional layer is connected, which converts the number of channels in the feature map to a preset embedding dimension, such as a 16-dimensional vector. The output of this 1x1 convolutional layer is not activated by Sigmoid or ReLU to allow the embedding vector to be freely distributed in the feature space. The resulting 16-channel feature map is the instance embedding map, whose spatial dimension is consistent with the keypoint heatmap. The 16-dimensional vector stored at any pixel location in the map is the instance embedding vector at that location.

[0029] Accordingly, the visual multi-scale feature map of wheat ears is input into the offset prediction head to obtain the offset prediction map. It is understandable that the downsampling operation performed by the convolutional neural network during feature extraction results in a spatial resolution of the final keypoint heatmap that is much lower than the original image. This leads to the peak coordinates located on the heatmap being merely discrete grid indices, which have quantization errors compared to the precise sub-pixel positions of the true keypoints in the original image. Therefore, in the technical solution of this application, the visual multi-scale feature map of wheat ears is further input into the offset prediction head to obtain the offset prediction map, thereby specifically learning a regression mapping from the features to the sub-pixel offset between the discrete grid coordinates and their true coordinates. This generates an offset prediction map spatially aligned with the keypoint heatmap, where each location stores the X and Y offsets used to correct the keypoint coordinates at that location, providing necessary data support for recovering high-precision sub-pixel coordinates in subsequent keypoint decoding steps.

[0030] More specifically, in a concrete example of this application, this processing flow is executed in parallel with the keypoint prediction head and the instance embedding prediction head. First, an aggregated multi-scale feature map, identical to the inputs of the other two prediction heads, is acquired. Then, this aggregated feature map is input to the offset prediction head. This prediction head consists of two stacked 3x3 convolutional layers with a ReLU activation function, used to regress fine-grained coordinate information. After this convolutional stack, a 1x1 convolutional layer is connected, which sets the number of channels of the feature map to 2. These two channels correspond to the offsets of the keypoints in the X and Y directions, respectively. The output of this 1x1 convolutional layer is not passed through a specific activation function to allow it to predict continuous offset values. The final generated feature map with two channels is the offset prediction map, whose spatial dimensions are consistent with the keypoint heatmap.

[0031] Specifically, in step S400, keypoint decoding and instance vector extraction are performed on the keypoint heatmap, instance embedding map, and offset prediction map to obtain a keypoint list. It is understandable that the keypoint heatmap, instance embedding map, and offset prediction map output by the multi-task joint prediction head are only intermediate, multimodal feature representations and cannot be directly used for counting. Furthermore, the traditional global static thresholding decoding mechanism for keypoint heatmaps has inherent flaws. For example, wheat ear keypoints in well-lit, clearly imaged areas typically have higher response values ​​on the heatmap, while real wheat ear keypoints in shadowed, motion-blurred, or distant areas may have lower response values. In complex field scenes with uneven lighting and crop occlusion, a fixed threshold set too high will inevitably filter out real wheat ear keypoints in shadowed or blurred areas, leading to missed detections; if set too low, it will introduce a large amount of pseudo-peak noise generated by background textures in clearly imaged areas, leading to false detections. Meanwhile, this method judges peak intensity in isolation, completely ignoring the deep and special relationship between candidate peak points and their neighboring pixels in the instance embedding feature space. A genuine wheat ear keypoint not only has a high response value, but its surrounding pixels should also have similar wheat ear semantic features; conversely, an isolated peak generated by noise often has chaotic embedding vectors for its neighboring pixels. The original mechanism fails to effectively utilize the rich spatial-semantic context information in the instance embedding map to assist in judging the authenticity of peaks, thus relying on a single decision basis and lacking dynamic adaptability to local context. Therefore, in the technical solution of this application, keypoint decoding and instance vector extraction are further performed on the keypoint heatmap, instance embedding map, and offset prediction map to obtain a keypoint list, thereby abolishing the fixed global threshold and instead executing an adaptive decoding process that integrates multi-dimensional information. This process dynamically calculates a final confidence score for each candidate peak by fusing the original response intensity of the keypoint heatmap with the neighborhood semantic consistency of the instance embedding map, and uses this as the basis for screening, while combining the offset prediction map to correct coordinates and integrate instance vectors. In this way, we can overcome the problems of poor scene adaptability and single decision basis caused by static thresholds. While suppressing spurious peak noise, we can effectively enhance and recall real key points in complex environments, and finally generate a list of key points with complete information and high purity, providing reliable data input for subsequent adaptive clustering.

[0032] Figure 3 This is a flowchart illustrating the process of decoding keypoints and extracting instance vectors from a keypoint heatmap, instance embedding map, and offset prediction map to obtain a keypoint list, according to an embodiment of the computer vision-based intelligent wheat ear counting method of this application. Figure 3As shown, step S400 includes: S410, determining a candidate key point list based on the key point heatmap; S420, performing neighborhood embedding consistency calculation and dynamic confidence adjustment on the candidate key point list based on the instance embedding graph to obtain an adjusted key point list; S430, performing final screening and information integration on the adjusted key point list based on the instance embedding graph and the offset prediction graph to obtain the key point list.

[0033] In step S410, a candidate keypoint list is determined based on the keypoint heatmap. It is understood that since the keypoint heatmap is a continuous probability response map, discrete peak points need to be located as the initial positions of the keypoints. Furthermore, the subsequent adaptive decoding mechanism requires a comprehensive, unfiltered set of peaks as the starting point for processing to avoid prematurely discarding true keypoints in poorly imaged areas such as shadows or blur due to excessively high or inappropriate static thresholds used in the early stages. Therefore, in the technical solution of this application, a candidate keypoint list is further determined based on the keypoint heatmap. This allows for the application of maximum value filtering on the heatmap to locate all local peak points, and a low confidence threshold, used only to filter out most background noise, is used for preliminary screening of these peak points. This generates a list containing a large number of candidate keypoints while preserving all potential possibilities. This list provides a comprehensive and unbiased input data source for subsequent neighborhood embedding consistency calculations and dynamic confidence adjustment steps, ensuring the coverage of subsequent processing.

[0034] More specifically, in a specific example of this application, determining a candidate key point list based on a key point heatmap includes: performing 3x3 maximum value filtering on the input key point heatmap to locate all local peak points; and performing preliminary screening of all local peak points based on a confidence threshold to obtain a candidate key point list.

[0035] Accordingly, a 3x3 maximum value filter is applied to the input keypoint heatmap to locate all local peak points. It is understood that, since the keypoint heatmap is a continuous response surface, the wheat ear keypoints on the map ideally represent local peaks with the highest response values, and these peak points need to be explicitly located by comparing them with their neighboring pixels. Therefore, in the technical solution of this application, a 3x3 maximum value filter is further applied to the input keypoint heatmap to locate all local peak points, thereby efficiently traversing each pixel on the heatmap and comparing its response value with the values ​​of its eight neighboring pixels in its 3x3 neighborhood, thus identifying all pixels with response values ​​higher than or equal to their direct neighbors. In this way, all local maximum points on the heatmap can be comprehensively extracted without suppression, providing a complete set of all potential keypoints for the subsequent preliminary screening step using a lower confidence threshold.

[0036] More specifically, in a concrete example of this application, the 3x3 maximum value filtering process is used as the first step in peak detection. This process is performed on a keypoint heatmap and is functionally equivalent to a max pooling operation with a kernel size of 3x3, a stride of 1, and padding of 1. This operation generates a maximum value map of the same size as the original heatmap, where the value of each pixel is the maximum response value within its corresponding 3x3 neighborhood in the original heatmap. Subsequently, by comparing the original heatmap and this maximum value map, all pixels whose response value in the original heatmap is equal to their corresponding response value in the maximum value map are identified as local peak points. The coordinates of all these identified local peak points are aggregated to form a complete list of local peak points for subsequent confidence filtering.

[0037] Accordingly, a preliminary screening of all local peak points is performed based on a confidence threshold to obtain a candidate keypoint list. It is understandable that the local peak points located in the previous step contain a large amount of spurious peak noise with extremely low response values ​​generated by background textures such as soil and leaves. Furthermore, if no screening is performed at this stage, the subsequent dynamic confidence adjustment step will bear an unnecessary computational burden. However, this step should not use a high threshold to avoid incorrectly filtering out true keypoints located in shadow or blurred areas. Therefore, in the technical solution of this application, a preliminary screening of all local peak points is further performed based on a confidence threshold to obtain a candidate keypoint list. This uses a lower confidence threshold, only used to filter out most background noise, to perform a non-high-precision preliminary screening. In this way, based on the principle of retaining all possible candidate keypoints as comprehensively as possible, while eliminating obvious noise, a comprehensive and unbiased input data source is ensured for the subsequent dynamic confidence adjustment step, guaranteeing the coverage and potential of subsequent processing.

[0038] More specifically, in a concrete example of this application, the filtering process begins by receiving the complete list of local peak points output from the previous step. A low confidence threshold is set, for example, 0.1. Each peak point in the list of local peak points is traversed, and its original confidence score in the keypoint heatmap is obtained. This original confidence score is compared with the set low confidence threshold of 0.1. If the original confidence score of the peak point is greater than 0.1, it is retained, and its coordinates and original confidence score are stored in a new list; if its original confidence score is less than or equal to 0.1, it is considered background noise and discarded. After traversing all local peak points, the newly generated list is the final candidate keypoint list.

[0039] In step S420, based on the instance embedding map, the candidate keypoint list is subjected to neighborhood embedding consistency calculation and dynamic confidence adjustment to obtain an adjusted keypoint list. It is understandable that the candidate keypoint list obtained in the previous step contains a large number of points with low original confidence. These points may be genuine keypoints in shadow or occluded areas, or they may be pseudo-peak noise generated by background textures. Relying solely on the original confidence of the keypoint heatmap is insufficient to effectively distinguish between these two situations. Furthermore, this isolated judgment method ignores the rich spatial semantic context information contained in the instance embedding map; that is, for a genuine keypoint, its neighboring pixels should also possess similar wheat-ear semantic features. Therefore, in the technical solution of this application, based on the instance embedding graph, the candidate keypoint list is further processed by calculating neighborhood embedding consistency and dynamically adjusting confidence to obtain an adjusted keypoint list. This quantifies the spatial semantic context information of the keypoint neighborhood and integrates it into the confidence evaluation system. By calculating the consistency score between each candidate point and its neighboring pixel embedding vectors, and constructing a dynamic adjustment function based on this score, the original confidence is reweighted in a context-aware manner. In this way, the final confidence of real keypoints located in complex environments with low original confidence but strong neighborhood semantic consistency can be adaptively enhanced, while suppressing noise spurious peaks caused by neighborhood semantic confusion, thereby generating a more discriminative confidence set, i.e., the adjusted keypoint list.

[0040] Figure 4 This is a flowchart illustrating the process of calculating neighborhood embedding consistency and dynamically adjusting confidence levels in a candidate keypoint list based on an instance embedding graph, according to an embodiment of the computer vision-based intelligent wheat ear counting method of this application, to obtain an adjusted keypoint list. (See flowchart for example.) Figure 4As shown, step S420 includes: S421, calculating the neighborhood embedding consistency score of each candidate key point in the candidate key point list based on the instance embedding graph; S422, calculating the final confidence of the candidate key point based on the neighborhood embedding consistency score and the original confidence of the candidate key point.

[0041] In step S421, based on the instance embedding graph, the neighborhood embedding consistency score of each candidate keypoint in the candidate keypoint list is calculated. It is understood that since the original confidence scores of each candidate keypoint in the candidate keypoint list are insufficient to determine its authenticity, the instance embedding graph provides crucial contextual discriminative information. Therefore, in the technical solution of this application, the neighborhood embedding consistency score of each candidate keypoint in the candidate keypoint list is further calculated based on the instance embedding graph. This quantifies the semantic purity of the region where the candidate keypoint is located by calculating the average cosine similarity between the central embedding vector and all embedding vectors in its neighborhood. This provides a crucial, quantified contextual discriminative factor for the subsequent dynamic confidence adjustment step, enabling the numerical distinction between true keypoints with high neighborhood consistency (scores approaching 1) and noise pseudo-peaks with low neighborhood consistency (scores very low).

[0042] More specifically, in a particular example of this application, the calculation process is performed as follows: traversing each candidate point in the candidate keypoint list. First, the candidate point is extracted from the instance embedding graph. The center embedding vector And determine a candidate point A neighborhood of a pre-defined size centered on the target For example, a 7x7 pixel window and its neighborhood. All pixels Embedded vector Then, the candidate point is calculated according to the following formula. Neighborhood embedding consistency score :

[0043] in, Representative candidate points Neighborhood embedding consistency score Represents the candidate key points currently being processed. represent neighborhood Any pixel in the image, It is the total number of pixels in the neighborhood; and They are and The embedding vector corresponding to the instance embedding graph. Represents the vector dot product. This represents the L2 norm of the vector. This formula quantifies the semantic purity of the region where a candidate point is located by calculating the average cosine similarity between the central embedding vector and all its neighboring embedding vectors. In the scenario of counting wheat ears, if the candidate point is a real wheat ear keypoint, its neighboring pixels are also likely to belong to the same wheat ear, and the embedding vectors tend to have consistent directions in the feature space, thus the consistency score will approach 1. Conversely, if it is a noisy spurious peak, its neighborhood semantics are mixed, and the score will be very low. For example, an isolated spurious peak caused by leaf edge reflection or soil exposed texture on a heatmap will have its neighboring pixel embedding vectors pointing to the semantic features of different instances such as leaves or soil, resulting in messy vector directions and a very low average cosine similarity score.

[0044] In step S422, the final confidence level of the candidate keypoint is calculated based on its neighborhood embedding consistency score and its original confidence level. It is understood that since the neighborhood embedding consistency score calculated in the previous step has not yet been integrated with the original confidence level in the keypoint heatmap, the decision-making basis remains separate. A mechanism is needed to effectively combine these two information sources to achieve context-aware reweighting of the original confidence level. Therefore, in the technical solution of this application, the final confidence level of the candidate keypoint is further calculated based on its neighborhood embedding consistency score and its original confidence level to construct a dynamic confidence adjustment function. This function uses the neighborhood embedding consistency score as a gain factor to adjust the original confidence level.

[0045] More specifically, in a particular example of this application, the calculation process is performed as follows: traversing each candidate point in the candidate keypoint list. Obtain the candidate point. Original confidence level in the key point heatmap And the neighborhood embedding consistency score calculated in the previous step. Set a hyperparameter to control the consistency score enhancement weights. For example, Set it to 1.5. Next, use the following formula to evaluate the candidate points. The original confidence level is dynamically adjusted to generate the final confidence level. :

[0046] in, Candidate point The final confidence level, It is its original confidence level in the key point heatmap. It is its neighborhood embedding consistency score. This is a hyperparameter used to control the weighting of consistency score enhancement. This formula constructs a dynamic confidence adjustment function. For points with high neighborhood semantic consistency, it acts as a gain factor, significantly increasing their confidence, thus rescuing true keypoints located in shaded areas that initially have low confidence but possess strong contextual information. However, for noisy points with confused neighborhood semantics, this gain factor has a weak effect, and their low confidence is not incorrectly amplified. For example, a true wheat ear keypoint located in a shaded area... It may only be 0.3, but its It is very high, reaching 0.9. At this point, its calculated gain factor is... , It was boosted to 0.705. Conversely, a noise pseudo-peak generated by blade texture, its It is also 0.3, but its It's very low, only 0.1. At this point, its calculated gain factor is only 1.15. It was only slightly amplified to 0.345. After performing this calculation on all points in the candidate keypoint list, an adjusted keypoint list containing the final confidence score was obtained.

[0047] In step S430, the adjusted keypoint list is finally filtered and information integrated based on the instance embedding map and the offset prediction map to obtain the keypoint list. It is understood that the adjusted keypoint list generated in the previous step only contains candidate points with context-aware final confidence, but has not yet utilized this reliable metric that integrates multi-dimensional information for final filtering. Furthermore, these candidate points only have discrete grid coordinates, lacking the precise sub-pixel coordinates and instance identity information necessary for subsequent clustering and counting. Therefore, in the technical solution of this application, the adjusted keypoint list is further filtered and information integrated based on the instance embedding map and the offset prediction map to obtain the keypoint list. This utilizes the robust final confidence generated in the previous step to perform a reliable keypoint filtering. For all qualified keypoints that pass the filtering, sub-pixel coordinate corrections are extracted from the offset prediction map, and the corresponding instance embedding vectors are extracted from the instance embedding map. In this way, precise filtering from a large number of candidate points to a small group of high-confidence keypoints can be completed, and all necessary information can be integrated to finally generate a clean and information-complete keypoint list for subsequent clustering and counting. More specifically, in a concrete example of this application, the process first performs a final screening. A fixed base threshold is applied to all points in the adjusted keypoint list. For example, set the confidence level to 0.5 for final filtering. Iterate through the list, retaining all final confidence levels. Greater than this basic threshold For points that do not meet the conditions, discard those points. Continuing from the previous specific example, when... When set to 0.5, the true key points in the shadow area with a final confidence level of 0.705 calculated earlier will be retained, while the false peaks of leaf texture noise with a final confidence level of 0.345 will be discarded, thus achieving precise screening.

[0048] in, The final confidence level of the point to be judged. This is a fixed baseline screening threshold. The significance of this step is that, because... It is already a reliable metric that integrates multidimensional information, so using a fixed threshold for filtering becomes efficient and robust. Subsequently, for all qualified key points that pass the filtering, information integration is performed. For a retained key point whose discrete grid coordinates on the heatmap are... The key point is, firstly, in the offset prediction map The position reads its corresponding 2-channel offset vector. And correct its sub-pixel coordinates to Next, in the instance embedding graph The location is read from its corresponding 16-dimensional instance embedding vector. Finally, the sub-pixel precise coordinates of the keypoint and its instance embedding vector are combined into a data entry and stored in a new list. After traversing all qualified keypoints, a clean and information-complete list of keypoints is finally generated for subsequent clustering and counting.

[0049] Through the aforementioned technical means, this optimization mechanism successfully overcomes the technical shortcomings of the original mechanism, which relies on a global static threshold, resulting in poor scene adaptability and a single decision-making basis. It transforms the keypoint decoding process from a simple judgment based on the response intensity of isolated points into a dynamic, comprehensive decision-making process that integrates local spatial and semantic context. This mechanism can adaptively enhance the confidence of true keypoints in complex environments while suppressing spurious peak noise caused by background textures. Ultimately, it improves the accuracy and robustness of wheat ear keypoint detection in complex field scenarios such as dense clustering and variable lighting, reducing missed and false detections. This provides a more reliable and pure data input for subsequent instance clustering steps, laying a solid foundation for achieving high-precision intelligent wheat ear counting.

[0050] Specifically, in step S500, the keypoint list is subjected to adaptive clustering based on instance embedding to obtain wheat ear instance clusters. It is understood that the keypoint list generated in the previous step is merely a collection of all detected, discrete wheat ear keypoints; these keypoints have not yet been organized into instances representing independent wheat ears. In densely clustered scenes, the spatial coordinate information of keypoints alone is insufficient to accurately distinguish which keypoints belong to the same wheat ear; the crucial identity information—the attached instance embedding vector—must be used for attribution determination. Therefore, in the technical solution of this application, the keypoint list is further subjected to adaptive clustering based on instance embedding to obtain wheat ear instance clusters. This allows clustering to be performed in the high-dimensional feature space formed by the instance embedding vectors, rather than in the image space susceptible to occlusion interference. This operation automatically aggregates keypoints whose embedding vectors are close in distance in the feature space (representing the same instance) into the same cluster, while assigning keypoints whose embedding vectors are far apart (representing different instances) to different clusters. In this way, all the previously detected scattered key points can be reconstructed and combined into a series of independent clusters of wheat ear instances that correspond one-to-one with the physical wheat ears. Even those wheat ears that are severely overlapping and tightly connected in the image can have their key points accurately separated in the feature space due to the difference in their embedding vectors, thus providing accurate grouping results for the final step of instance enumeration and total count.

[0051] Figure 5 This is a flowchart illustrating the process of using a computer vision-based intelligent wheat ear counting method according to an embodiment of this application to perform adaptive clustering of a key point list based on instance embedding to obtain wheat ear instance clusters. Figure 5 As shown, step S500 includes: S510, constructing a mutual reachability distance matrix based on the key point list; S520, performing hierarchical structure aggregation and stable cluster extraction on the mutual reachability distance matrix to obtain a cluster label list; S530, performing label mapping and wheat ear instance aggregation on the key point list based on the cluster label list to obtain wheat ear instance clusters.

[0052] In step S510, a cross-reachability distance matrix is ​​constructed based on the keypoint list. It is understood that since the keypoint list contains all high-confidence keypoints and their corresponding instance embedding vectors, the subsequent adaptive clustering operation requires a metric that can accurately measure the similarity or distance between any two keypoints in terms of instance identity (i.e., in the embedding feature space). If only standard Euclidean distance is used, the clustering results will be highly sensitive to noise points and clusters of varying densities. Therefore, in the technical solution of this application, a cross-reachability distance matrix is ​​further constructed based on the keypoint list to transform the Euclidean distance between keypoints into a more robust distance metric that considers local density information. This metric integrates the core distance between two points and the Euclidean distance between them. This effectively smooths out the influence of sparse noise points, making the distances between points within dense clusters more compact, thereby constructing a metric space that more accurately reflects instance affiliation, laying a solid foundation for subsequent hierarchical structure agglomeration and stable cluster extraction.

[0053] More specifically, in a particular example of this application, the construction process is performed as follows: First, extract all key points from the list of key points. The instance embedding vectors of each key point constitute a The embedding vector matrix. Next, a neighborhood number of points is defined. ,For example This value represents the minimum number of keypoints required to form a valid cluster of wheat ear instances. Next, each keypoint is calculated. core distance The distance is defined as The embedding vector to its first The Euclidean distance between the nearest neighbor embedding vectors is then calculated for any two keypoints. and Euclidean distance between Finally, based on the core distance and Euclidean distance, the following calculations are performed. and Interreachability distance between The calculation formula is as follows:

[0054] in, Key point and The mutual reachability distance between them Key point Core distance, Key point Core distance, Key point and The Euclidean distance between them The number of neighboring points calculated from the core distance. The minimum cluster size chosen for a stable cluster. This calculation will include all... A pair of points Store the value in a In the symmetric matrix, this matrix is ​​the final reachability distance matrix.

[0055] In step S520, the reachability distance matrix is ​​subjected to hierarchical structure agglomeration and stable cluster extraction to obtain a cluster label list. It is understood that the reachability distance matrix generated in the previous step only provides a robust distance metric based on local density between all key points, but it has not yet organized these points into instance clusters with affiliation relationships. Traditional methods require pre-setting the number of clusters or distance thresholds, which is ineffective in complex field scenarios where the number of wheat ears is unknown and their density is uneven. Therefore, in the technical solution of this application, the reachability distance matrix is ​​further subjected to hierarchical structure agglomeration and stable cluster extraction to obtain a cluster label list. This allows the construction of a complete hierarchical tree structure based on the distance matrix, and the application of stability metrics to automatically extract the most persistent stable clusters representing real wheat ear instances from this hierarchy, while identifying sparse points that cannot be assigned to any stable cluster as noise. In this way, all key points can be adaptively assigned to corresponding instance clusters or labeled as noise without pre-setting parameters, ultimately generating a cluster label list containing the affiliation information of all key points.

[0056] More specifically, in a particular example of this application, the process first performs hierarchical cohesion. Each key point is considered as a graph. There are vertices, and the mutual reachability distance matrix is ​​used. The values ​​are used as weights for edges connecting any two vertices. On this weighted complete graph, Prim's or Kruskal's algorithm is applied to construct a minimum spanning tree connecting all vertices. Then, stable cluster extraction is performed. All edges of the minimum spanning tree are sorted in descending order of weight. Starting with the edge with the largest weight, these edges are broken sequentially, implicitly constructing a hierarchical tree structure for clustering. Next, this hierarchical structure is traversed, and the stability of each cluster appearing in the hierarchy is calculated. The stability of a cluster is a comprehensive measure based on the distance range of its members surviving in the hierarchy—the distance difference from cluster formation to cluster split—and the size of the cluster. Finally, the algorithm automatically selects the clusters with the highest stability scores as the final wheat ear instance clusters, assigning a unique cluster ID to all keypoints falling into these stable clusters, while all sparse keypoints that fail to be assigned to any stable cluster are assigned a special noise label, such as -1. The cluster IDs or noise labels obtained from the key points together constitute the final list of cluster labels.

[0057] In step S530, the keypoint list is labeled and aggregated with wheat ear instances based on the cluster label list to obtain wheat ear instance clusters. It is understood that the cluster label list generated in the previous step is merely a set of index labels corresponding one-to-one with the keypoint list, where each label represents the instance affiliation of its corresponding keypoint. However, these keypoint data are still stored as discrete lists and have not yet been physically grouped and aggregated according to their label affiliation. Therefore, in the technical solution of this application, the keypoint list is further labeled and aggregated with wheat ear instances based on the cluster label list to obtain wheat ear instance clusters. This allows the cluster label list to be traversed, and keypoints with the same valid cluster label (i.e., non-noise labels) are extracted from the keypoint list and aggregated into the same dataset. In this way, all scattered, high-confidence keypoints can be reconstructed into a series of independent wheat ear instance clusters, each cluster representing a detected independent wheat ear instance, thus providing structured, grouped input data for subsequent instance enumeration and total count steps.

[0058] More specifically, in a concrete example of this application, the process first initializes an empty hash table for aggregating key points by cluster ID. Then, the list of cluster labels is traversed, with its index... From 0 to , This represents the total length of the keypoint list. In each iteration, the cluster label list is obtained. Cluster label at location and list of key points Key data at the location Next, determine... ,like If the label is noise (e.g., -1), then discard it. ;like If it is a valid cluster ID, then look up the cluster ID in the hash table. The entry is the key. If the entry does not exist, a new empty list is created as the value for that key; then, Add it to the list corresponding to that key. After iterating through all... After identifying the key points, all values ​​stored in the hash table, i.e., all non-empty lists, are extracted and combined into a top-level list, which is the final cluster of wheat ear instances.

[0059] Specifically, in step S600, the total number of wheat ears is obtained by counting based on the clusters of wheat ear instances. It is understood that since the wheat ear instance clusters generated in the previous step are a structured data set, where each element (i.e., cluster) represents an independently identified and separated wheat ear instance, and the ultimate goal of this invention is to provide a quantitative counting result. Therefore, in the technical solution of this application, instance enumeration and total count are further performed on the wheat ear instance clusters to obtain the total number of wheat ears. This is done by traversing each instance cluster in the set of wheat ear instance clusters, and the enumerated total number of instances is used as the final counting result. In this way, the previously complex result based on feature space clustering can be converted into an intuitive and accurate numerical value, namely the total number of wheat ears, achieving a high-precision counting target in densely clustered scenarios.

[0060] More specifically, in a specific example of this application, counting based on clusters of wheat ear instances to obtain the total number of wheat ears includes: performing instance enumeration and total count on clusters of wheat ear instances to obtain the total number of wheat ears.

[0061] More specifically, this process receives the cluster of wheat ear instances output from the previous step. This cluster is a top-level list, where each element (i.e., a sublist) represents an independent wheat ear instance. Then, a list length calculation function is called to obtain the number of elements in the top-level list of the wheat ear instance cluster. For example, if the top-level list contains 289 elements (sublists) after enumeration, the function returns the integer 289. This returned integer is the total number of instances after enumeration. Finally, this integer value of 289 is assigned to the total number of wheat ears, serving as the final quantitative output of this method.

[0062] In summary, the computer vision-based intelligent wheat ear counting method according to the embodiments of this application is explained. Instead of employing the traditional bounding box detection paradigm, it extracts multi-scale features through a deep network and simultaneously predicts the location heatmap of key points on the wheat ear, the offset prediction map for precise localization, and the instance embedding map for distinguishing instances via a multi-task prediction head. To address the problem of inaccurate key point detection in complex environments, this solution introduces an adaptive key point decoding mechanism. This mechanism integrates the response intensity of the key point heatmap with the local neighborhood semantic consistency information of the instance embedding map, dynamically adjusting the confidence of each candidate key point. This effectively recalls true key points in occluded or poorly lit areas while suppressing false peak noise. After obtaining a list of high-confidence key points, adaptive clustering is performed in the feature space using the instance embedding vector carried by each key point. In this way, multiple key points belonging to the same wheat ear instance are aggregated into the same cluster, while key points from different instances are separated. Ultimately, the accurate counting of wheat ears was achieved by counting the total number of effective clusters, thus solving the problem of missed detections and inaccurate counting caused by non-maximum suppression and fuzzy representation in densely clustered scenarios using traditional bounding box methods.

[0063] As described above, the computer vision-based intelligent wheat ear counting method according to the embodiments of this application can be implemented in various smart agriculture and crop phenotyping systems, such as agricultural IoT cloud platforms, UAV ground station analysis software, high-throughput crop phenotyping platforms, intelligent agricultural machinery control terminals, or agricultural remote sensing data processing platforms. In one possible implementation, the computer vision-based intelligent wheat ear counting method according to the embodiments of this application can be integrated as a software module or hardware module into the overall management system of smart agriculture or precision agriculture service platform. For example, the computer vision-based intelligent wheat ear counting method can be an independent yield estimation application running on an agricultural IoT cloud platform or remote sensing data processing server, or it can be an intelligent counting function module of existing UAV ground station analysis software, breeding management information system, or field management system, or it can be a service deployed on edge computing devices and providing real-time counting results to the cloud platform through an application programming interface. Of course, the core feature extraction, multi-task joint prediction, and instance clustering functions of this method can also be embedded in dedicated AI acceleration chips, field-programmable gate arrays, or embedded vision processing units, as one of the edge computing acceleration modules of the real-time crop phenotyping perception system.

[0064] The various embodiments of this disclosure have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the described embodiments. The terminology used herein is chosen to best explain the principles, practical application, or improvement of the technology in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A method for intelligent counting of wheat ears based on computer vision, characterized in that, include: Acquire raw images of wheat ears captured by camera equipment; Image feature extraction is performed on the original image of wheat ears to obtain a visual multi-scale feature map of wheat ears; The visual multi-scale feature map of wheat ears is input into the multi-task joint prediction head to obtain key point heatmap, instance embedding map and offset prediction map; Keypoints are decoded and instance vectors are extracted from the keypoint heatmap, instance embedding map and offset prediction map to obtain a list of keypoints. Adaptive clustering based on instance embedding is performed on the list of key points to obtain wheat ear instance clusters; The total number of wheat ears is obtained by counting based on clusters of wheat ear instances.

2. The intelligent wheat ear counting method based on computer vision according to claim 1, characterized in that, Image feature extraction is performed on the original image of wheat ears to obtain a visual multi-scale feature map of wheat ears, including: The original wheat ear image is normalized in size and pixel value to obtain a preprocessed wheat ear image; The preprocessed wheat ear image is input into the backbone of a convolutional neural network with a feature pyramid network to obtain a visual multi-scale feature map of the wheat ear.

3. The intelligent wheat ear counting method based on computer vision according to claim 1, characterized in that, The visual multi-scale feature map of wheat ears is input into a multi-task joint prediction head to obtain a keypoint heatmap, instance embedding map, and offset prediction map, including: Input the visual multi-scale feature map of wheat ears into the keypoint prediction head to obtain a keypoint heatmap. The visual multi-scale feature map of wheat ears is input into the instance embedding prediction head to obtain the instance embedding map; Input the visual multi-scale feature map of wheat ears into the offset prediction head to obtain the offset prediction map.

4. The intelligent wheat ear counting method based on computer vision according to claim 1, characterized in that, Keypoint heatmaps, instance embedding maps, and offset prediction maps are decoded for keypoints and instance vectors are extracted to obtain a list of keypoints, including: Based on the key point heatmap, a list of candidate key points is determined; Based on the instance embedding graph, the candidate key point list is calculated for neighborhood embedding consistency and dynamically adjusted for confidence to obtain the adjusted key point list. Based on the instance embedding graph and the offset prediction graph, the adjusted key point list is finally filtered and information is integrated to obtain the key point list.

5. The intelligent wheat ear counting method based on computer vision according to claim 4, characterized in that, Based on the key point heatmap, a list of candidate key points is determined, including: Perform 3x3 maximum value filtering on the input key point heatmap to locate all local peak points; A preliminary screening of all local peak points is performed based on the confidence threshold to obtain a list of candidate key points.

6. The intelligent wheat ear counting method based on computer vision according to claim 4, characterized in that, Based on the instance embedding graph, the candidate keypoint list is subjected to neighborhood embedding consistency calculation and dynamic confidence adjustment to obtain an adjusted keypoint list, including: Based on the instance embedding graph, the neighborhood embedding consistency score of each candidate key point in the candidate key point list is calculated. The final confidence of the candidate keypoint is calculated based on the neighborhood embedding consistency score and the original confidence of the candidate keypoint.

7. The intelligent wheat ear counting method based on computer vision according to claim 1, characterized in that, Adaptive clustering based on instance embedding is performed on the list of key points to obtain wheat ear instance clusters, including: Construct a mutual reachability distance matrix based on the list of key points; Hierarchical structure aggregation and stable cluster extraction are performed on the interreachability distance matrix to obtain a cluster label list; Based on the cluster label list, the key point list is labeled and aggregated with wheat ear instances to obtain wheat ear instance clusters.

8. The intelligent wheat ear counting method based on computer vision according to claim 1, characterized in that, The total number of wheat ears is obtained by counting based on clusters of wheat ear instances, including: enumerating instances of wheat ear clusters and counting the total number of wheat ears.

Images