A method for detecting the number of grains per ear, the number of rows per ear and the number of grains per row

Abstract

The application discloses a corn ear grain number, ear row number and ear row grain number detection method, and belongs to the image detection field.The corn ear grain number counting method encodes the multi-scale features of the corn ear image through a Transformer encoder, enhances the long-distance dependency relationship and global context information between the features, provides more abundant semantic information for a point query and a Transformer decoding module, and outputs the feature representation of each query point by the decoder, so that abundant semantic information is provided for the classification and positioning tasks of a prediction head, background interference can be effectively excluded, and the counting precision is improved.The ear row number and ear row grain number detection method solves common problems in corn ear row detection, such as half rows and non-straight lines, by using clustering and a vector-based point searching and updating iterative method, and can efficiently and accurately realize row detection and ear row number detection.

Description

Technical Field

[0001] This invention belongs to the field of image detection, and more specifically, relates to a method for detecting the number of kernels per ear, the number of rows per ear, and the number of kernels per row in corn. Background Technology

[0002] Maize is one of the world's three major food crops, playing a vital role in animal husbandry and industry. Analyzing and studying various traits of maize can help establish the relationship between these traits and yield, thereby maximizing profits. Maize yield is mainly determined by the number of ears, the number of kernels per ear, and the kernel weight. Within a certain range, increasing the number of ears and kernels per ear often directly increases yield potential. Maize phenotyping is usually done manually through random sampling and visual observation. This method is tedious, time-consuming, labor-intensive, and prone to errors. Furthermore, manual observation may damage the maize's growing environment. Manual observation cannot meet the needs of modern high-throughput phenotyping analysis. Computer vision image analysis technology can be used to extract maize phenotypic characteristics.

[0003] With the popularization of plant phenotyping platforms, research on automated plant phenotyping based on digital images has increased year by year. Many plant phenotyping extraction methods rely heavily on deep convolutional neural networks (CNNs). One study proposed using cross-sections in maize images to estimate the number of rows per ear, but this study used a 600 dpi resolution flatbed scanner and was only suitable for small-scale detection against simple backgrounds, exhibiting poor robustness in large-scale field environments. Another study proposed a neural network that used mean absolute error and mean squared error to evaluate counting performance; its local counting regression method outperformed other baseline methods, which improves detection efficiency, but its robustness in natural field environments still needs improvement. In recent years, research has proposed a method using distance metric learning to measure data similarity, while simultaneously learning background network parameters, embedding space, and the feature distribution of each training data point within that space. This method uses the PyTorch deep learning framework for model training and employs data augmentation techniques such as random scaling, cropping, rotation, color jittering, and horizontal flipping to enhance the robustness of maize row detection in open environments; however, some angular bias still exists, leading to certain errors in the detection and counting of maize rows and ears. Summary of the Invention

[0004] In view of the above-mentioned defects or improvement needs of the existing technology, the present invention provides a method for detecting the number of kernels, the number of rows, and the number of kernels per row of corn. This method can accurately identify the number of kernels in corn ears from corn ear images taken in an open environment by mobile terminals such as mobile phones, and can also locate and detect the number of rows and the number of kernels per row of corn based on the results of the corn kernel count.

[0005] To achieve the above objectives, according to a first aspect of the present invention, a method for counting the number of kernels in a corn ear is provided, characterized by comprising:

[0006] Training phase:

[0007] A corn ear kernel counting model was constructed and trained using a dataset to obtain a trained corn ear kernel counting model. The dataset includes corn ear images and their annotation data collected in residential, field, and solid color scenarios, respectively. The annotation data includes the number of corn kernels per ear and the location coordinates of each kernel. The corn ear kernel counting model includes:

[0008] The feature extraction module is used to extract multi-scale feature maps from corn ear images;

[0009] A Transformer encoder is used to encode the multi-scale feature map;

[0010] The query point initialization module is used to process the corn ear image to evenly distribute the original query points.

[0011] A quadtree splitter is used to split the original query points of the dense corn kernel region in the corn ear image into four query points according to the encoded feature map, while retaining the original query points of the sparse corn kernel region in the corn ear image.

[0012] The Transformer decoder is used to calculate the attention of each target query point based on the encoded multi-scale feature map to obtain the feature representation of each target query point; wherein, the target query point includes the split query point and the original query point that is retained;

[0013] The prediction head includes a classification branch and a regression branch; the classification branch is used to determine whether the target query point corresponds to a corn ear based on its feature representation; the regression branch is used to predict the position offset of the target query point if it corresponds to a corn ear, so as to obtain the position coordinates of the corn ear.

[0014] Application phase:

[0015] The image of the corn ear to be detected is input into the trained corn ear kernel counting model to obtain the counting result and location coordinates of the corn ear kernels.

[0016] According to a second aspect of the present invention, a method for detecting the number of rows and kernels per ear of maize is provided, comprising:

[0017] S1. Based on the coordinates of all corn kernels in the corn ear image to be detected, perform principal component analysis on all points to calculate the direction of the first principal axis, rotate all points to make the first principal axis parallel to the x-axis, and sort them according to the x-coordinate from smallest to largest to form a point set;

[0018] S2, the point set is clipped in the x direction according to a preset clipping ratio, and the point set is divided into a first point set and a second point set; wherein, the points in the first point set are the points located in the middle part of the point set, and the points in the second point set are the points located in other positions in the point set.

[0019] S3, perform K-Means clustering on the first point set, take the number of clusters with the highest silhouette coefficient as the optimal number of clusters, perform cluster analysis on the first point set according to the optimal number of clusters, generate cluster labels, and assign the label of its row to each point in the first point set.

[0020] S4. Using a vector-based point-finding method, find points in the second point set that are in the same row as each row of points in the first point set and assign a label to the row they belong to, until all points in the second point set have been traversed.

[0021] The vector-based point-finding method is as follows: the point with the smallest and largest x-coordinate in each row of the first point set is taken as the left and right boundary points of that row, respectively.

[0022] When searching for points to the left, an initial vector is set horizontally to the left, starting from the left boundary point of each row. Points in the first point set that are to the left of the second point set are selected as points to be processed in descending order of their x-coordinates. The following operations are performed: a new vector is obtained by connecting the left boundary point of each row to the point to be processed. It is then determined whether the angle between the initial vector and the new vector of each row is outside the preset angle range. If so, the point to be processed is considered a noise point. Otherwise, the point to be processed is assigned the label of the row containing the point with the smallest angle, and the corresponding new vector is used as the initial vector of that row for the next operation. The same principle applies when searching for points to the right.

[0023] S5 outputs the detection results of the number of rows and kernels per ear of corn.

[0024] According to a third aspect of the present invention, an electronic device is provided, comprising: a computer-readable storage medium and a processor;

[0025] The computer-readable storage medium is used to store executable instructions;

[0026] The processor is configured to read executable instructions stored in the computer-readable storage medium and execute the method as described in the first aspect or the second invention.

[0027] According to a fourth aspect of the invention, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions for causing a processor to perform the method as described in the first or second aspect.

[0028] According to a fifth aspect of the invention, a computer program product is provided, comprising a computer program or instructions that, when executed by a processor, implement the method as described in the first or second aspect.

[0029] In summary, compared with the prior art, the above-described technical solutions conceived by this invention can achieve the following beneficial effects:

[0030] 1. The corn ear kernel counting method provided by this invention uses a point query quadtree method to count corn ear kernels in an open environment. In an open environment, corn kernel count data and visualization results can be accurately obtained by taking pictures of corn with a handheld device. Background interference is eliminated, and the method has accuracy and robustness.

[0031] 2. The method for detecting the number of rows and kernels per row of corn ears provided by this invention solves common problems in corn ear row detection, such as half-row and non-linear detection, by using clustering and vector-based point-finding and update iteration methods. It can efficiently and accurately realize row detection and kernel count detection. Attached Figure Description

[0032] Figure 1 This is an overall flowchart of the method for detecting the number of kernels per ear, the number of rows per ear, and the number of kernels per row in corn provided in this embodiment of the invention;

[0033] Figure 2 This is a schematic flowchart of the corn ear kernel counting method provided in an embodiment of the present invention;

[0034] Figure 3 This is an original field image of corn ears provided in an embodiment of the present invention;

[0035] Figure 4 This is an original image of a corn cob in a real-life scenario provided in an embodiment of the present invention;

[0036] Figure 5 This is the original image of a corn ear obtained by the trait scanner provided in this embodiment of the invention;

[0037] Figure 6 This is a field corn ear labeling diagram provided in an embodiment of the present invention;

[0038] Figure 7 This is a labeled image of corn ears in a real-life scenario provided in an embodiment of the present invention;

[0039] Figure 8 This is a labeled image of a corn ear taken by the machine provided in this embodiment of the invention;

[0040] Figure 9 This is a graph showing the field corn ear count results provided in an embodiment of the present invention;

[0041] Figure 10 This is a diagram showing the results of counting corn ears in a real-life scenario, as provided in an embodiment of the present invention.

[0042] Figure 11 This is a graph showing the number of corn ears obtained by the trait scanner provided in this embodiment of the invention;

[0043] Figure 12 This is a flowchart illustrating the specific process for detecting the number of rows of ears and the number of grains per row provided in this embodiment of the invention.

[0044] Figure 13 This is a graph showing the results of field maize ear row number detection provided in an embodiment of the present invention;

[0045] Figure 14 This is a diagram showing the detection results of the number of rows of corn ears in a real-life scenario, provided by an embodiment of the present invention.

[0046] Figure 15 This is a diagram showing the number of rows in a corn ear obtained by the trait scanner provided in this embodiment of the invention.

[0047] Figure 16 This is a schematic diagram of the trait scoring instrument and data acquisition process provided in an embodiment of the present invention; Figure 16 Image (a) is a schematic diagram of a corn ear trait scoring instrument used to acquire images with a solid color background. Figure 16 (b) and (c) in the figure are schematic diagrams of corn ears being placed into the container of the trait scoring instrument from different perspectives. Figure 16 (d) in the diagram is a schematic diagram for inspecting corn ears. Detailed Implementation

[0048] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0049] Although there are many detection technologies related to the number of ears of maize, they are difficult to apply in actual field environments due to the limitations of various methods or strategies. There are still significant gaps in the counting of phenotypic traits such as the total number of kernels per ear and the number of ears per row, and the robustness in open environments has not achieved the desired effect.

[0050] Based on this, embodiments of the present invention provide a method for counting the number of kernels in a corn ear, comprising:

[0051] Training phase:

[0052] A corn ear kernel counting model was constructed and trained using a dataset to obtain a trained corn ear kernel counting model. The dataset includes corn ear images and their labeled data collected in residential, field, and solid color scenarios. The labeled data includes the number of kernels per ear and the location coordinates of each kernel. Figure 2 As shown, the corn ear kernel counting model includes:

[0053] The feature extraction module is used to extract multi-scale feature maps from corn ear images;

[0054] A Transformer encoder is used to encode the multi-scale feature map;

[0055] The query point initialization module is used to process the corn ear image to evenly distribute the original query points.

[0056] A quadtree splitter is used to split the original query points of the dense corn kernel region in the corn ear image into four query points according to the encoded feature map, while retaining the original query points of the sparse corn kernel region in the corn ear image.

[0057] The Transformer decoder is used to calculate the attention of each target query point based on the encoded multi-scale feature map to obtain the feature representation of each target query point; wherein, the target query point includes the split query point and the original query point that is retained;

[0058] The prediction head includes a classification branch and a regression branch; the classification branch is used to determine whether the target query point corresponds to a corn ear based on the feature representation of the target query point; the regression branch is used to predict the position offset of the target query point when the target query point corresponds to a corn ear, and add the position offset to the position coordinates of the target query point to obtain the position coordinates of the corresponding corn ear.

[0059] Application phase:

[0060] The image of the corn ear to be detected is input into the trained corn ear kernel counting model to obtain the counting result and location coordinates of the corn ear kernels.

[0061] The structure and training process of the corn ear kernel counting model are described below.

[0062] First, a dataset containing images of corn ears and their labeled data was collected, taken with a mobile phone in everyday life scenes, field scenes, and with a seed testing machine in solid-color scenes. The labeled data includes the center point location and number of kernels on the corn ear, such as... Figure 4-9As shown, this dataset was used to train a deep learning network based on a corn ear kernel counting model, optimizing network parameters to minimize the difference between the predicted results and the true labels. The loss functions used during training included classification loss and localization loss. Network weights were updated through backpropagation until the model converged, learning the feature representation and spatial distribution patterns of corn ears. The classification loss used the cross-entropy loss function to determine whether a query point is a corn ear kernel; the localization loss used the Smooth L1 loss function to optimize the difference between the predicted and true locations. Through training, the model learned the feature representation and spatial distribution patterns of corn ears, providing support for subsequent counting and localization tasks.

[0063] The corn ear kernel counting model includes a feature extraction module, a Transformer encoder, a query point initialization module, a quadtree splitter, a Transformer decoder, and a prediction head.

[0064] 1. Feature Extraction Module

[0065] Before using the feature extraction module to extract features from the corn cob image, the process also includes initial preprocessing of the corn cob image, including operations such as cropping, rotation, and normalization, to increase data diversity and improve generalization ability.

[0066] The feature extraction module is used to extract multi-scale features of the image. It can use any existing feature extraction module, but preferably a feature extraction module based on a CNN backbone network (e.g., VGG16 convolutional neural network).

[0067] The preprocessed image is input into a feature extraction module based on a CNN backbone network to generate feature maps at multiple resolutions. The feature extraction module uses a VGG16 convolutional neural network as its backbone network, extracting feature maps layer by layer at resolutions of 1 / 2, 1 / 4, 1 / 8, 1 / 16, and 1 / 32, providing the corn ear kernel counting model with comprehensive information from low-level local details to high-level global context. These feature maps are then input into the encoder.

[0068] like Figure 1 As shown, to obtain a sufficiently large receptive field, the encoder encodes the preprocessed image through five encoding stages (1 / 2, 1 / 4, 1 / 8, 1 / 16, and 1 / 32) of the VGG16 network, generating multi-scale feature maps. These feature maps provide rich local details and global contextual information for subsequent point queries and the decoder.

[0069] 2. Encoder

[0070] The encoder, based on the Transformer architecture, is used to further extract and integrate contextual information. The encoder's output is a set of encoded feature representations that not only preserve the visual information extracted by the CNN backbone network but also enhance the long-range dependencies between features and global contextual information, providing richer semantic information for subsequent point queries and the Transformer decoding module.

[0071] As a preferred approach, the encoder employs a progressive rectangular window attention mechanism to extract and integrate contextual information. The encoder first calculates self-attention within a large rectangular window to capture global contextual information in the image. As the number of encoder layers increases, the size of the rectangular window gradually decreases, allowing the model to focus on smaller local regions and capture more detailed features of the corn kernels. The encoder's self-attention calculation formula is as follows:

[0072]

[0073] Where x l-1 The output features of the encoder at layer (l-1) are... For the features after self-attention processing via a rectangular window, x l This represents the final output feature of the encoder's l-th layer. RectWin-SA represents the rectangular window self-attention mechanism, FFN is the feedforward network, and LN is the layer normalization. Through this progressive attention calculation method, the encoder can gradually understand the image content from global to local perspectives, outputting a set of encoded feature representations F. These feature representations not only retain the visual information extracted by the CNN backbone network but also enhance the long-distance dependencies and global contextual information between features, providing richer semantic information for subsequent point queries and Transformer decoding modules.

[0074] 3. Query point initialization module and quadtree splitter

[0075] The query point initialization module evenly distributes sparse query points on the corn ear image, with an initial distribution interval of K pixels.

[0076] A quadtree splitter analyzes the contextual features of the encoder output to determine which regions in the corn ear image are dense corn kernel regions and which are sparse regions. The quadtree splitter outputs a split map, which determines which query points need further splitting. In dense regions, quadtree nodes are recursively split, dividing a query point into four new query points to improve the counting accuracy of dense regions. In sparse regions, the original set of query points is preserved.

[0077] That is, such as Figure 2As shown, the query point initialization module initializes a set of sparse query points on the corn ear image, and then inputs the initialized corn ear image into a quadtree splitter. This splitter consists of an average pooling layer, a 1×1 convolutional layer, and a sigmoid activation function. The average pooling layer is used to reduce the spatial dimension of the feature map, the 1×1 convolutional layer is responsible for extracting region density features, and the sigmoid function maps the output value to the [0,1] interval to obtain the split map M. s .

[0078] M S =σ(Conv 1×1 (AvgPool(F)))

[0079] Where σ is the sigmoid function, Conv 1×1 This is a 1×1 convolution operation, and AvgPool is an average pooling operation. M s The closer each element value is to 1, the higher the density of corn kernels in the corresponding region, requiring query point splitting; conversely, the closer the value is to 0, the lower the region density, and no splitting is needed. According to M... s The corn kernel counting model performs quadtree splitting on query points in dense regions, subdividing one query point into four new query points. This increases the query point density within a local area, allowing for a more refined capture of the distribution of corn kernels. The final result is a set of query points adapted to different density regions.

[0080] 4. Transformer Decoder

[0081] The generated set of query points and the contextual features output by the encoder are jointly input into the decoder. The decoder, based on a Transformer architecture, preferably employs a Progressive Rectangle Window Attention mechanism to process the local features of the query points. Attention is computed within a larger rectangular window for sparse query points and within a smaller rectangular window for dense query points, thereby improving counting accuracy without increasing computational complexity. The decoder outputs a feature representation for each query point, which integrates contextual information and local features, providing rich semantic information for subsequent classification and localization tasks.

[0082] In the decoding phase of the Transformer, the decoder receives input consisting of a point query quadtree and encoded image features, and outputs the decoded point queries. Specifically, it infers relationships between point queries by calculating attention within a local rectangular window, enabling it to determine whether each query point represents a kernel of corn and to pinpoint its location, guided by the image context. Sparse query points undergo self-attention calculation within a larger rectangular window, while dense query points undergo it within a smaller rectangular window. The decoder's attention calculation involves not only self-attention but also cross-attention with encoder features. The decoder's attention calculation formula is:

[0083]

[0084] Among them, z l-1 The output features of the decoder at layer l-1, For the features after self-attention and cross-attention processing, z l This represents the final output feature of the decoder's layer l. RectWin-SA represents the rectangular window self-attention mechanism, and RectWin-CA represents the rectangular window cross-attention mechanism. x N This represents the final output features of the encoder. Through this local rectangular window attention computation, the decoder is able to output feature representations for each query point. These feature representations fuse local and global contextual information, providing rich semantic information for subsequent classification and localization tasks.

[0085] 5. Predicting the head

[0086] The feature representation output from the decoder is input into the prediction head, which uses a multilayer perceptron (MLP) structure to classify and locate each query point. The prediction head contains two main branches: a classification branch determines whether each query point corresponds to a corn ear and outputs a classification probability; a regression branch predicts the positional offset of the query point, thus accurately locating the actual position of the corn ear. Specifically, the classification branch maps the decoded features to classification probabilities using the MLP network, and uses a sigmoid activation function to limit the output value to the range [0,1], representing the probability that the query point is a corn ear. The regression branch also uses the MLP network to predict the positional offset (Δx, Δy) of the query point and adds these offsets to the original position of the query point to calculate the final position coordinates (x, y) of the corn ear. Finally, accurate corn ear count and location information are generated as the model's output.

[0087] Specifically, the classification branch maps the decoded features to classification probabilities using a multilayer perceptron (MLP), and uses a sigmoid activation function to restrict the output value to the range [0,1], representing the probability that the query point is a corn kernel. The formula for calculating the classification probability is:

[0088] c i =σ(W c ·MLP(z decode )+b c )

[0089] Among them, c i W is the classification probability of the i-th query point, σ is the Sigmoid activation function, and W is the classification probability of the ith query point. c and b c These are the weights and bias parameters of the classification branches, z decode It is the feature representation output by the decoder.

[0090] The regression branch also predicts the location offset of the query point using a multilayer perceptron (MLP) and adds these offsets to the original location of the query point to calculate the final location coordinates of the corn kernels. The formula for predicting the location offset is:

[0091] Δp i =W p ·MLP(z decode )+b p

[0092] Where, Δp i =(Δx) i ,Δy i ) is the position offset, W p and b p These are the weights and bias parameters of the regression branch, (x i ,y i ) is the initial position of the query point.

[0093] In the output stage of the corn ear kernel count model, each query point is assigned a classification probability c. i and a normalized pixel position p i =(x i +Δx i ,y i +Δy i The model sets a probability threshold (e.g., 0.5) and classifies query points with a probability greater than this threshold as corn kernels, marking their locations in the image. Then, the corn kernel counting model aggregates all query points identified as corn kernels to determine the total number of kernels in the image. Finally, it generates a file showing the coordinates of the corn kernels and visualizes the count, as shown below. Figure 10-11As shown.

[0094] This invention provides a method for detecting the number of rows and kernels per ear of maize, such as... Figure 12 As shown, it includes:

[0095] S1. Based on the coordinates (x, y) of all corn kernels in the corn ear image to be detected, perform principal component analysis on all points to calculate the direction of the first principal axis, rotate all points to make the first principal axis parallel to the x-axis, and sort them according to the x-coordinate from smallest to largest to form a point set.

[0096] Specifically, based on the coordinates of all corn kernels in the corn ear image to be detected, principal component analysis (PCA) is performed on all points to calculate the direction of the first principal axis. All points are then rotated until the first principal axis is parallel to the x-axis, and all points are sorted in ascending order according to their x-coordinates to form a point set. The rotation process utilizes the calculated principal axis angle and the mean of the point set.

[0097] The above PCA and rotation processes include:

[0098] Calculate the mean and principal axis direction of the point set;

[0099] Construct a rotation matrix to rotate the point set to a horizontal position, and record the rotation angle and mean coordinates for subsequent recovery operations.

[0100] S2, the point set is cropped in the x direction according to a preset cropping ratio, and the point set is divided into a first point set and a second point set; wherein, the points in the first point set are the points located in the middle part of the point set, and the points in the second point set are the points located in other positions in the point set.

[0101] Specifically, the rotated point set is cropped in the x-direction according to the set cropping ratio, retaining only the points in the middle; the preset cropping ratio is between 0 and 1 and can be set according to requirements.

[0102] For example, if the preset cropping ratio is 0.12, the rotated point set is cropped according to the cropping ratio. The center point of the x-axis coordinate of the point set is found, and the width of the cropping area is expanded to both sides based on this center point. The start and end x-coordinates (x1 and x2) of the cropping area are determined. The points located within the cropping area are selected, and the point set formed by them is taken as the first point set. The point set formed by the remaining points is taken as the second point set.

[0103] S3, cluster the first point set, and take the cluster number with the highest silhouette coefficient as the optimal cluster number. Perform cluster analysis on the first point set according to the optimal cluster number, generate cluster labels, and assign the label of its row to each point in the first point set.

[0104] Specifically, for example, the maximum number of clusters is set to 10 and the minimum number of clusters is 2. Any clustering algorithm can be used here, such as the K-Means clustering algorithm. The K-Means clustering algorithm is used to cluster the clipped point set into 2 to 10 clusters respectively. The number of clusters with the highest silhouette coefficient among all clustering results is selected as the optimal number of clusters for the clipped point set. The optimal number of clusters is used to perform cluster analysis on the standardized point set, generate cluster labels, and assign a label to each point for its row.

[0105] S4. Using a vector-based point-finding method, find points in the second point set that are in the same row as each row of points in the first point set and assign a label to the row they belong to, until all points in the second point set have been traversed.

[0106] The vector-based point-finding method is as follows: the point with the smallest and largest x-coordinate in each row of the first point set is taken as the left and right boundary points of that row, respectively.

[0107] When searching for points to the left, an initial vector is set horizontally to the left, starting from the left boundary point of each row. Points in the first point set that are to the left of the second point set are selected as points to be processed in descending order of their x-coordinates, and the following operations are performed: a new vector is obtained by connecting the left boundary point of each row to the point to be processed, and it is determined whether the angle between the initial vector and the new vector of each row is outside the preset angle range. If so, the point to be processed is considered a noise point; otherwise, the point to be processed is assigned the label of the row containing the point with the smallest angle, and the corresponding new vector is used as the initial vector of that row for the next operation; moving to the right... Similarly, when finding points along the edge, an initial vector is set horizontally to the right starting point of each row. Points in the first point set that are to the right of the second point set are selected in ascending order of their x-coordinates as points to be processed, and the following operations are performed: a new vector is obtained by connecting the right boundary point of each row to the point to be processed, and it is determined whether the angle between the initial vector and the new vector of each row is not within the preset angle range. If so, the point to be processed is regarded as a noise point; otherwise, the point to be processed is assigned the label of the row with the smallest angle, and the corresponding new vector is used as the initial vector of the row point in the next operation.

[0108] Specifically, the vector-based point-finding method is as follows: For one of the clustered points in S3, the two points with the largest and smallest x-coordinates of that cluster are designated as the right and left boundary points. When finding points to the left, an initial horizontal vector (hereinafter referred to as the original vector) is assigned to each left boundary point. For each point in the second point set to the left of the first point set, the following operation is performed in descending order of x-coordinates: a new vector is obtained by connecting the left boundary point of each cluster to the point. The angle between the new vector and the original vector is checked to see if it is within the set angle error (e.g., ±20°). If it is within the error range, the cluster with the smallest angle is taken as the category of the point, the left boundary point of that category is updated to the point, and the new vector obtained is used as the original vector for the next operation; if no cluster meets the angle requirement, the point is considered a noise point; the same principle applies when finding points to the right.

[0109] S5 outputs the detection results of the number of rows and kernels per ear of corn.

[0110] The optimal cluster number is the number of rows of corn ears, which is the number of kernels per ear and the corresponding number of kernels per ear row after finding the points using a vector-based point-finding method.

[0111] It is understood that the coordinates of all corn kernels in the corn ear image to be detected can be obtained using any existing detection method. To improve the recognition accuracy, the corn ear kernel counting method provided in this embodiment of the invention is preferred.

[0112] Preferably, after traversing all points in the set of points, the method further includes:

[0113] S6, determine whether there are any unlabeled points in the point set. If so, take the set of unlabeled points as the updated point set and return to S2; otherwise, proceed to S5.

[0114] Specifically, considering that photography can only capture one side of the corn, some rows of corn may be rotated or bent. This means that when searching for points from the middle to both ends, rows on the back of the corn that the camera could not capture may rotate to the front of the corn, resulting in extra half rows. Due to symmetry, extra half rows may appear at both ends of the corn, i.e., two half rows. Since these half rows are not in the rows determined by the previous clustering, they cannot be identified and classified by the vector method. Therefore, additional S2-S4 pruning, clustering, and point-finding operations are required to cluster and identify the two half rows before point-finding. Finally, the classification results of the half rows are merged with the previous classification results to obtain the overall classification result. If there are still unlabeled points, they are regarded as error points (noise points). Finally, the complete results of the detection of the number of rows and kernels per ear are output.

[0115] That is, the remaining unlabeled points are grouped into new standardized motors for further trimming, K-means clustering, and point finding operations. The half-row cases caused by rotation are handled, the remaining points are labeled, and the results are merged with the original classification results. The number of corn kernels in each category can be obtained by counting the points in each category. Finally, the points are rotated back to their original direction to generate the detection results for the number of ear rows and kernels per ear row, as shown below. Figure 13-15 As shown.

[0116] Preferably, after step S2 and before step S3, the method further includes: compressing the x-coordinates of each point in the cropped point set by a factor of N, while keeping the y-coordinates unchanged;

[0117] After step S3 and before step S4, the process also includes: stretching the x-coordinates of each point in the clustered point set by a factor of N, while keeping the y-coordinates unchanged, to restore the original scale.

[0118] Specifically, after step S2 and before S3, the cropped point set is proportionally stretched to eliminate coordinate scale differences. Based on the fact that the row containing the corn kernels approximately conforms to a straight line, and after previously rotating the row to horizontal, the x-coordinate of the point set is proportionally reduced, for example, by a factor of 200. The y-coordinate remains unchanged, so that points on the same row are compressed together as much as possible, eliminating coordinate scale differences and significantly enhancing the subsequent clustering effect. After step S3 and before S4, the x-coordinate of the clustered (cropped) point set is proportionally stretched, for example, by a factor of 200, to restore it to its original scale.

[0119] This invention provides an electronic device, including: a computer-readable storage medium and a processor;

[0120] The computer-readable storage medium is used to store executable instructions;

[0121] The processor is used to read executable instructions stored in the computer-readable storage medium and execute the corn ear kernel counting method or the corn ear row number and kernel count detection method as described in any of the above embodiments.

[0122] This invention provides a computer-readable storage medium storing computer instructions for causing a processor to execute a corn ear kernel counting method or a corn ear row number and kernel count detection method as described in any of the above embodiments.

[0123] This invention provides a computer program product, including a computer program or instructions, which, when executed by a processor, implement the corn ear kernel count method or the corn ear row count and kernel count detection method as described in any of the above embodiments.

[0124] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for detecting the number of rows and kernels per ear of maize, characterized in that, include: S1. Based on the coordinates of all corn kernels in the corn ear image to be detected, perform principal component analysis on all points to calculate the direction of the first principal axis, rotate all points to make the first principal axis parallel to the x-axis, and sort them according to the x-coordinate from smallest to largest to form a point set; S2, the point set is clipped in the x direction according to a preset clipping ratio, and the point set is divided into a first point set and a second point set; wherein, the points in the first point set are the points located in the middle part of the point set, and the points in the second point set are the points located in other positions in the point set. S3, cluster the first point set, take the number of clusters with the highest silhouette coefficient as the optimal number of clusters, perform cluster analysis on the first point set according to the optimal number of clusters, generate cluster labels, and assign the label of its row to each point in the first point set. S4. Using a vector-based point-finding method, find points in the second point set that are in the same row as each row of points in the first point set and assign a label to the row they belong to, until all points in the second point set have been traversed. The vector-based point-finding method is as follows: the point with the smallest and largest x-coordinate in each row of the first point set is taken as the left and right boundary points of that row, respectively. When searching for points to the left, an initial vector is set horizontally to the left, starting from the left boundary point of each row. Points in the first point set that are to the left of the second point set are selected as points to be processed in descending order of their x-coordinates. The following operations are performed: a new vector is obtained by connecting the left boundary point of each row to the points to be processed. It is then determined whether the angle between the initial vector and the new vector of each row is outside the preset angle range. If so, the point to be processed is considered a noise point. Otherwise, the point to be processed is assigned the label of the row containing the point with the smallest angle, and the corresponding new vector is used as the initial vector of that row for the next operation. The same principle applies when searching for points to the right. S5 outputs the detection results of the number of rows and kernels per ear of corn.

2. The method as described in claim 1, characterized in that, After traversing all points in the second point set, the process also includes: S6, determine whether there are any unlabeled points in the second point set. If so, take the set of unlabeled points as the updated first point set and return to S3; otherwise, proceed to S5.

3. The method as described in claim 1, characterized in that, After step S2 and before step S3, the process also includes: compressing the x-coordinates of each point in the cropped point set by a factor of N, while keeping the y-coordinates unchanged; After step S3 and before step S4, the operation also includes: stretching the x-coordinate of each point in the clustered point set by a factor of N, while keeping the y-coordinate unchanged.

4. The method as described in claim 1, characterized in that, The coordinates of all corn kernels in the corn ear image to be detected are obtained using the following method for counting corn kernels per ear: Training phase: A corn ear kernel counting model was constructed and trained using a dataset to obtain a trained corn ear kernel counting model. The dataset includes corn ear images and their annotation data collected in residential, field, and solid color scenarios, respectively. The annotation data includes the number of corn kernels per ear and the location coordinates of each kernel. The corn ear kernel counting model includes: The feature extraction module is used to extract multi-scale feature maps from corn ear images; A Transformer encoder is used to encode the multi-scale feature map; The query point initialization module is used to process the corn ear image to evenly distribute the original query points. A quadtree splitter is used to split the original query points of the dense corn kernel region in the corn ear image into four query points according to the encoded feature map, while retaining the original query points of the sparse corn kernel region in the corn ear image. The Transformer decoder is used to calculate the attention of each target query point based on the encoded multi-scale feature map to obtain the feature representation of each target query point; wherein, the target query point includes the split query point and the original query point that is retained; The prediction head includes a classification branch and a regression branch; the classification branch is used to determine whether the target query point corresponds to a corn ear based on its feature representation; the regression branch is used to predict the position offset of the target query point if it corresponds to a corn ear, so as to obtain the position coordinates of the corn ear. Application phase: The image of the corn ear to be detected is input into the trained corn ear kernel counting model to obtain the counting result and location coordinates of the corn ear kernels.

5. The method as described in claim 4, characterized in that, The Transformer encoder encodes the multi-scale features based on a progressive rectangular window attention mechanism, and the calculation formula is as follows: in, For encoder number The output features of the layer The features are those after self-attention processing through a rectangular window. For encoder number The final output features of the layer This represents a rectangular window self-attention mechanism, where FFN is a feedforward network and LN is a layer normalization.

6. The method as described in claim 4 or 5, characterized in that, The Transformer decoder calculates the attention of each target query point based on an attention mechanism within a rectangular window. The calculation formula is as follows: in, For decoder number The output features of the layer Features are those processed by self-attention and cross-attention. For decoder number The final output features of the layer; This represents the self-attention mechanism of a rectangular window. This represents the cross-attention mechanism of a rectangular window. This represents the final output feature of the encoder.

7. An electronic device, characterized in that, include: Computer-readable storage media and processors; The computer-readable storage medium is used to store executable instructions; The processor is used to read executable instructions stored in the computer-readable storage medium and execute the corn ear row number and ear row number detection method as described in any one of claims 1-6.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing a processor to execute the method for detecting the number of rows and kernels per ear of corn as described in any one of claims 1-6.

9. A computer program product, comprising a computer program or instructions, characterized in that, The computer program or instructions are executed by a processor according to any one of claims 1-6, the method for detecting the number of rows and kernels per ear of corn as described in claims 1-6.

Images