Automatic seed testing method and system based on corn ear panoramic image

Abstract

The invention provides an automatic seed testing method and system based on a corn ear panoramic image, and belongs to the technical field of corn ear seed testing. Line scanning panoramic splicing can still keep continuity and geometric consistency under the condition of rotation eccentricity / slippage, and the negative influence of stripe distortion on subsequent grain detection and ear row analysis is remarkably reduced. Therefore, on the premise of not depending on high-cost three-dimensional equipment, not only can the advantage of ear full-surface information acquisition be kept, but also stripe error distortion in the scene can be corrected by utilizing an error compensation mechanism, so that the stability and precision of ear row number, row grain number and total grain number calculation are improved; the problems that strip error distortion is prone to being generated in special scenes such as ear rotation eccentricity or slippage in existing line scanning panoramic splicing, and then the ear row number and the grain statistical accuracy are reduced are solved.

Description

Technical Field

[0001] This invention relates to the field of corn ear seed testing technology, specifically to an automated seed testing method and system based on panoramic images of corn ears. Background Technology

[0002] As an important food crop, maize's yield and quality traits directly affect food security and the development of the breeding industry. The maize ear, as the direct carrier of yield and quality, has phenotypic parameters such as the number of rows per ear, the number of kernels per row, and the total number of kernels as key indicators for testing variety specificity, uniformity, and stability, as well as in the process of new variety breeding. Currently, the testing of the number of rows per ear and the number of kernels still generally relies on manual counting or threshing counters, which is prone to problems such as low efficiency, high subjectivity, poor repeatability, and difficulty in maintaining consistency under large-scale sample conditions.

[0003] With the development of computer vision technology, image analysis-based seed selection methods are gradually increasing. Existing approaches typically include: seed selection methods based on single 2D images, full-surface reconstruction methods based on multi-view image sequences, and seed selection methods based on 3D point clouds or 3D model reconstruction. It's easy to understand, but it's important to note that a single 2D image can only reflect information from one side of the ear, inherently leading to systematic biases due to incomplete information. While multi-view or 3D reconstruction provides more complete information, it significantly increases the requirements for hardware, lighting conditions, and algorithm complexity, making it difficult to simultaneously meet the cost, efficiency, and stability requirements of large-scale breeding scenarios.

[0004] To balance information integrity and system feasibility, line-scan panoramic imaging is an important approach. This involves using an industrial line-scan camera to rotate around the ear of grain, acquiring a strip sequence, and stitching it together to create a panoramic unfolded image, thus obtaining a high-resolution image covering the entire surface of the ear. However, a special scenario that is easily overlooked but significantly impacts the results exists in practical applications: when the ear of grain experiences eccentricity, slippage, or slight shaking during rotation, the relative displacement between adjacent strips changes over time and accumulates, leading to strip errors, longitudinal distortion, local omissions, or duplicate coverage in the panoramic stitching. These distortions are further amplified by texture repetition, reflections, or local occlusion, causing missed detections and duplicate counts of grains, which in turn leads to instability in the ear row resolution direction and increased deviations in the statistical discrepancies between the number of ear rows and the number of grains per row.

[0005] Therefore, how to maintain the advantage of acquiring full surface information of the ear without relying on high-cost 3D equipment, and how to compensate for strip error distortion in the above scenarios in order to improve the stability and accuracy of the calculation of the number of rows, number of grains per row and total number of grains, has become a technical problem that urgently needs to be solved in this field. Summary of the Invention

[0006] The purpose of this invention is to provide an automated seed assessment method based on panoramic images of corn ears, so as to at least solve the problem that existing line scan panoramic stitching is prone to strip error distortion in special scenarios such as ear rotation, eccentricity, or slippage, which leads to a decrease in the accuracy of ear row count and grain statistics.

[0007] To achieve the above objectives, the first aspect of the present invention provides an automated seed assessment method based on panoramic images of maize ears, the method comprising:

[0008] Acquire the raw strip sequence data of the line scan of a target corn ear rotating once, and generate a strip pixel matrix set based on the raw strip sequence data of the line scan;

[0009] A panoramic image of corn ears is constructed based on the strip pixel matrix set, a scanning error compensation tensor corresponding to each strip of the strip pixel matrix set is generated, and non-rigid stitching correction is performed on the panoramic image to obtain a corrected panoramic image.

[0010] Obtain seed annotation data for the corrected panoramic image, and train a seed target detection model based on the seed annotation data;

[0011] The corrected panoramic image is input into the grain target detection model to obtain a grain detection result set, and a grain spatial feature point set is generated based on the grain detection result set;

[0012] Based on the set of spatial feature points of the grains, a spatial graph of the grains is constructed and the row analysis and row grain statistics are performed to output the seed evaluation results containing the number of rows of the ears, the number of grains per row, and the total number of grains.

[0013] Optionally, the process involves acquiring raw strip sequence data of a line scan of a target corn ear rotating once, and generating a strip pixel matrix set based on the raw strip sequence data, including:

[0014] Acquire the line scan pixel sequence output by the line scan camera during the rotation of the target corn ear, and record each output line scan pixel sequence as strip scan row data;

[0015] The strip scan line data are indexed and numbered according to the strip acquisition order to form a strip sequence index table containing a one-to-one correspondence between strip numbers and pixel values;

[0016] The data from multiple consecutive strip scan rows are accumulated and stitched together according to a preset pixel width to generate a corresponding strip pixel matrix, and all strip pixel matrices are combined to form the strip pixel matrix set.

[0017] The integrity check of the strip pixel matrix set is performed based on the strip sequence index table, so as to output the strip pixel matrix set when the check satisfies the preset width and height constraints.

[0018] Optionally, a panoramic image of a corn ear is constructed based on the strip pixel matrix set, a scanning error compensation tensor corresponding to each strip of the strip pixel matrix set is generated, and non-rigid stitching correction is performed on the panoramic image to obtain a corrected panoramic image, including:

[0019] For adjacent strip pixel matrices in the strip pixel matrix set, extract the corresponding strip overlap window data, and calculate the original displacement estimation vector between strips based on the strip overlap window data;

[0020] Temporal smoothing is performed on the original displacement estimation vector to obtain the strip-corrected displacement vector, and all strip-corrected displacement vectors are stacked according to the strip number to form a scan error compensation tensor;

[0021] Based on the scanning error compensation tensor, perform non-rigid remapping of the strip pixel matrix set strip by strip, and then accumulate and stitch the remapped strip pixel matrix set according to the strip number order to generate the corrected panoramic image.

[0022] Optionally, obtaining seed annotation data for the corrected panoramic image includes:

[0023] Obtain a set of corrected panoramic images, and generate corresponding image identification information for each image in the set of corrected panoramic images;

[0024] For each corrected panoramic image, a set of rectangular bounding boxes covering all grain regions in the corrected panoramic image is obtained, and each rectangular bounding box is associated with the corresponding grain category label to form grain labeling data;

[0025] The seed annotation data is paired and encapsulated with image labeling information to construct a dataset for model training, and overlap is allowed between the rectangular annotation boxes to preserve the seed occlusion relationship representation.

[0026] Optionally, a grain target detection model is trained based on the labeled grain data, including:

[0027] The dataset is divided into a training set and a validation set to obtain a training sample set and a validation sample set.

[0028] Image scale normalization and data augmentation are performed on the training sample set to generate input tensors for model training;

[0029] The input tensor is iteratively optimized and trained based on a preset deep learning object detection architecture, and the training process is evaluated for accuracy based on the validation sample set to output model weight parameters that meet the preset evaluation threshold, thus forming the seed object detection model.

[0030] Optionally, the corrected panoramic image is input into the grain target detection model to obtain a grain detection result set, and a grain spatial feature point set is generated based on the grain detection result set, including:

[0031] The corrected panoramic image is divided into multiple detection sub-images according to a preset window size, and seed target detection is performed on each detection sub-image to obtain a set of sub-image detection boxes;

[0032] Based on the spatial index relationship of each detection sub-image, the set of detection boxes of the sub-images is mapped to the calibrated panoramic image coordinate system to generate a panoramic detection box set;

[0033] Non-maximum suppression is performed on the set of panoramic detection boxes to obtain a set of seed detection results, and the coordinates of the seed center point are calculated based on the geometric center of each detection box in the set of seed detection results to form the set of seed spatial feature points.

[0034] Optionally, constructing a grain spatial map based on the set of grain spatial feature points includes:

[0035] Delaunay triangulation is performed based on the seed space feature point set to generate an adjacent edge set, and the seed space graph is constructed by using the seed space feature point set as a node set and the adjacent edge set as an edge set.

[0036] For each adjacent edge in the grain space graph, calculate the edge orientation angle and edge length, and generate the edge cost based on the edge orientation angle and edge length;

[0037] Based on the edge cost, Bellman-Ford path optimization is performed on the grain space graph to obtain the optimal edge sequence set under the preset cost criterion, and the ear row direction parameters are determined based on the direction statistics of the optimal edge sequence set.

[0038] Optionally, perform ear row analysis and row grain statistics to output the seed evaluation results, which include the number of ear rows, the number of grains per row, and the total number of grains, including:

[0039] Based on the ear row direction parameters, the coordinates of the grain center point are projected onto the ear row orthogonal axis to obtain a row index projection value sequence;

[0040] Perform one-dimensional clustering on the row index projection value sequence to obtain multiple ear row clusters, and determine the number of ear row clusters as the number of ear rows;

[0041] For each ear row cluster, the number of kernels per row is determined based on the number of kernel center points within that ear row cluster, and the total number of kernels is calculated based on the number of kernels per row and the number of ear rows.

[0042] Optionally, before outputting the seed evaluation results, the method further includes performing consistency verification on the total number of seeds and the row seed count, and performing a re-examination of abnormal sections if the consistency verification fails to meet preset conditions, including:

[0043] The detection count value is determined based on the grain detection result set, and the statistical count value is determined based on the number of rows in the ear and the number of grains per row;

[0044] A consistency score is calculated based on the detection count value and the statistical count value, and if the consistency score is greater than a preset threshold, it is determined that the consistency verification does not meet the preset conditions.

[0045] When the consistency verification fails to meet the preset conditions, an abnormal segment mask is generated based on the spatial distribution of the low-confidence detection boxes in the seed detection result set, and re-detection inference is performed on the image region defined by the abnormal segment mask to update the seed detection result set.

[0046] The seed testing results are updated and output based on the updated seed detection result set.

[0047] A second aspect of the present invention provides an automated crop testing system based on panoramic images of maize ears, the system being used to execute the aforementioned automated crop testing method based on panoramic images of maize ears, the system comprising:

[0048] The acquisition unit is used to acquire the original strip sequence data of the line scan of the target corn ear rotating once, and to generate a strip pixel matrix set based on the original strip sequence data of the line scan.

[0049] The construction unit is used to construct a panoramic image of corn ears based on the strip pixel matrix set, generate a scanning error compensation tensor corresponding to each strip of the strip pixel matrix set, and perform non-rigid stitching correction on the panoramic image to obtain a corrected panoramic image.

[0050] The training unit is used to acquire seed annotation data for the corrected panoramic image and train a seed target detection model based on the seed annotation data.

[0051] The generation unit is used to input the corrected panoramic image into the grain target detection model to obtain a grain detection result set, and generate a grain spatial feature point set based on the grain detection result set;

[0052] The output unit is used to construct a grain spatial map based on the set of grain spatial feature points and perform ear row analysis and row grain statistics to output the seed evaluation results including the number of ear rows, the number of grains per row, and the total number of grains.

[0053] Beneficial Effects: Through the above technical solution, this invention proposes an automated seed assessment method and system based on panoramic images of maize ears. By introducing an error compensation mechanism while ensuring complete acquisition of information across the entire ear surface, the line-scan panoramic stitching maintains continuity and geometric consistency even under rotational eccentricity / slippage conditions. This significantly reduces the negative impact of strip distortion on subsequent grain detection and ear row analysis, thereby improving the stability and accuracy of ear row number, row grain number, and total grain count calculations. Simultaneously, this invention retains the high efficiency and low hardware cost advantages of line-scan imaging, making it suitable for large-scale breeding practices and DUS testing scenarios.

[0054] Other features and advantages of the embodiments of the present invention will be described in detail in the following detailed description section. Attached Figure Description

[0055] The accompanying drawings are provided to further illustrate embodiments of the present invention and form part of the specification. They are used together with the following detailed description to explain the embodiments of the present invention, but do not constitute a limitation thereof. In the drawings:

[0056] Figure 1 This is a flowchart of the steps of an automated seed assessment method based on panoramic images of corn ears provided by one embodiment of the present invention.

[0057] Figure 2 This is a panoramic schematic diagram of a corn ear provided in one embodiment of the present invention.

[0058] Figure 3 This is a schematic diagram of seed labeling provided in one embodiment of the present invention.

[0059] Figure 4 This is a schematic diagram of the recognition result provided by one embodiment of the present invention.

[0060] Figure 5 This is a system structure diagram of an automated seed evaluation system based on panoramic images of corn ears, provided by one embodiment of the present invention. Detailed Implementation

[0061] The specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are for illustration and explanation only and are not intended to limit the present invention.

[0062] like Figure 1 As shown, this invention provides an automated seed assessment method based on panoramic images of maize ears, the method comprising:

[0063] S10: Obtain the original strip sequence data of the line scan of the target corn ear rotating once, and generate a strip pixel matrix set based on the original strip sequence data of the line scan.

[0064] Specifically, the process involves acquiring the line scan pixel sequence output sequentially by the line scan camera during the rotation of the target corn ear, and recording each output line scan pixel sequence as a strip scan row data; indexing and numbering the strip scan row data according to the strip acquisition order to form a strip sequence index table containing a one-to-one correspondence between strip numbers and pixel values; accumulating and splicing multiple consecutive strip scan row data according to a preset pixel width to generate a corresponding strip pixel matrix, and constructing the strip pixel matrix set from all strip pixel matrices; performing an integrity check on the strip pixel matrix set based on the strip sequence index table, and outputting the strip pixel matrix set when the check meets the preset width and height constraints.

[0065] In this embodiment of the invention, step S10 is used to form the basic data input for subsequent panoramic construction and correction. It should be noted that this invention does not limit the use of a specific hardware structure for data acquisition. However, in one feasible implementation, an industrial-grade line scan camera can be configured in a sealed acquisition box with a constant light source. This allows the pixel values ​​output by the camera to have better stability over time, thereby reducing brightness errors and texture inconsistencies between stripes caused by changes in ambient light. Figure 2 As shown.

[0066] In practical applications, line scan cameras can output line scan pixel sequences sequentially at a fixed sampling rate during one rotation of the fruit ear. Specifically, each output line scan pixel sequence can be considered as strip scan line data, and each strip scan line data can be represented in terms of data structure as a one-dimensional array or vector, whose elements are pixel values ​​arranged in scanning order. Furthermore, in one executable implementation, a timestamp or strip number can be appended to each output to form a strip sequence index table, establishing a one-to-one correspondence between the strip number and its pixel sequence. This index table is subsequently used to determine the splicing order and to perform integrity checks in case of lost, duplicate, or abnormal frames, thereby ensuring the availability and traceability of the strip pixel matrix set.

[0067] It should be further noted that the strip pixel matrix set is not simply a direct stacking of all line scan lines, but rather constrains the horizontal width and panoramic unfolding height of each strip at the data level to ensure that the panoramic image obtained by subsequent stitching meets the preset target in terms of size. Specifically, in one exemplary implementation, the line scan camera collects 2 pixels per scan, and collects 1600 times in one revolution around the ear of fruit, thereby forming a panoramic unfolding width of 3200 pixels. At the same time, the panoramic height can be set to 4090 pixels to meet the unfolding requirements of covering the surface of the ear of fruit. It should be noted that the above values ​​are only used to illustrate a feasible parameter configuration and do not limit the present invention to using a fixed number of sampling pixels, sampling times, or height parameters. Those skilled in the art can make equivalent adjustments according to the size of the ear of fruit, the field of view of the lens, the target resolution, and the computing power requirements.

[0068] In one executable implementation, multiple consecutive strip scan rows of data can be accumulated and stitched together according to a preset pixel width to generate a two-dimensional strip pixel matrix. That is, one-dimensional pixel rows on the time series are stacked sequentially to form a matrix representation with horizontal and vertical pixel indices as coordinates, and all strip pixel matrices are collected to form a strip pixel matrix set. Each strip pixel matrix retains its strip number and its relative order information in the sequence, so that subsequent steps can accurately access the required strip data by using the strip number as an index.

[0069] S20: Construct a panoramic image of corn ears based on the strip pixel matrix set, generate a scanning error compensation tensor corresponding to each strip of the strip pixel matrix set, and perform non-rigid stitching correction on the panoramic image to obtain a corrected panoramic image.

[0070] Specifically, for adjacent strip pixel matrices in the strip pixel matrix set, corresponding strip overlap window data is extracted, and the original displacement estimation vector between strips is calculated based on the strip overlap window data; temporal smoothing is performed on the original displacement estimation vector to obtain strip correction displacement vectors, and all strip correction displacement vectors are stacked into a scan error compensation tensor according to the strip number; non-rigid remapping is performed on the strip pixel matrix set based on the scan error compensation tensor, and the remapped strip pixel matrix set is accumulated and stitched in strip number order to generate the corrected panoramic image.

[0071] In this embodiment of the invention, step S20 is used to convert the strip pixel matrix set into a corrected panoramic image that can be used for detection and analysis. It should be noted that in a normal scenario where the ear of fruit rotates stably and the axis is aligned, the relative displacement between adjacent strips is approximately constant. A panoramic unfolded image covering the entire surface of the ear of fruit can be obtained by directly stitching the strips in order of their numbers, and the effect of virtually no overlap or omissions can be achieved.

[0072] However, in scenarios involving ear rotation, slippage, or slight shaking, the relative displacement between adjacent strips changes over time and accumulates errors, leading to longitudinal distortion, local omissions, or overlapping in the stitched panoramic image. Furthermore, when the ear surface has reflections, repetitive textures, or local occlusion, local matching between strips is more prone to erroneous displacement estimation, further exacerbating stitching distortion and reducing the stability of subsequent grain detection and ear row analysis. To address this technical problem, this invention introduces a scanning error compensation tensor to characterize the two-dimensional displacement variation of the strip sequence over time, and accordingly performs a correction process on the strip data that differs from conventional stitching.

[0073] In this embodiment of the invention, the construction of the scanning error compensation tensor is not a one-time global registration of the entire image, but rather a correlation matching of adjacent stripes within a local overlap window, thereby obtaining the original displacement estimation vector of each strip relative to the previous strip. Specifically, in one executable implementation, for strip t and strip (t-1), strip overlap window data can be extracted from their boundary, and the correlation function can be calculated:

[0074] ;

[0075] ;

[0076] Where t is the strip number; Let be the pixel value of the overlapping window of the t-th strip; The pixel value after the overlapping window of the (t-1)th strip is shifted. This is the translation amount; This is the original displacement estimation vector. It should be noted that this invention does not limit the correlation function to the above form; normalized cross-correlation, phase correlation, or other equivalent forms can also be used. However, regardless of the implementation, the output must form an original estimation vector that can characterize the two-dimensional displacement between strips for subsequent time series smoothing and tensor construction.

[0077] After obtaining the original displacement estimation vector, this embodiment of the invention further suppresses the influence of noise and mismatches on the displacement sequence through temporal smoothing, making the displacement changes more consistent with the physical prior that the rotation error is continuously changing, thereby reducing the abrupt propagation of displacement jumps between strips. Specifically, the following recursive update relationship can be adopted:

[0078] ;

[0079] ;

[0080] in, To correct the displacement vector; is the smoothing coefficient; N is the total number of strips; T is the scan error compensation tensor. It should be noted that this invention uses the term "tensor" to emphasize that it is structured displacement data oriented towards strip sequences, containing not only the two-dimensional displacement components of each strip but also implicitly including the time-dimensional index of the strip numbering order, thus enabling subsequent non-rigid remapping to stably retrieve the corresponding displacement information according to the strip number.

[0081] After constructing the scanning error compensation tensor, this embodiment of the invention further performs a non-rigid remapping of the strip pixel matrix set. It should be noted that conventional stitching often only performs rigid translation alignment. However, when errors accumulate over time and cause local distortion, rigid translation alone cannot simultaneously satisfy global continuity and local consistency. This invention introduces the following data processing action through tensor-based strip remapping: For each strip, according to its corresponding... Perform coordinate resampling.

[0082] In a more feasible implementation, the strips can be segmented along the height direction, and within each segment, non-rigid methods such as segmented affine mapping or thin-plate splines can be used for local remapping. This ensures that the strip boundaries remain continuous with adjacent strips in the global coordinate system, thereby significantly reducing strip error distortion in eccentric / slip scenarios. After remapping, the corrected panoramic image can be obtained by accumulating and stitching the strips in numerical order.

[0083] S30: Obtain the seed annotation data for the corrected panoramic image, and train a seed target detection model based on the seed annotation data.

[0084] Specifically, a set of calibrated panoramic images is acquired, and corresponding image labeling information is generated for each image in the set. For each calibrated panoramic image, a set of rectangular bounding boxes covering all seed regions in the image is acquired, and each bounding box is associated with a corresponding seed category label to form seed labeling data. The seed labeling data and image labeling information are paired and encapsulated to construct a dataset for model training, allowing overlap between the bounding boxes to preserve the seed occlusion relationship representation. The dataset is then divided into a training set and a validation set to obtain a training sample set and a validation sample set. Image scale normalization and data augmentation are performed on the training sample set to generate an input tensor for model training. Iterative optimization training is performed on the input tensor based on a preset deep learning object detection architecture, and the training process is evaluated for accuracy based on the validation sample set to output model weight parameters that meet a preset evaluation threshold, thus forming the seed object detection model.

[0085] In this embodiment of the invention, step S30 is used to establish a detection model capable of stably locating kernels in a panoramic image. It should be noted that corn kernels in a panoramic unfolded image typically exhibit dense arrangement and natural occlusion features, and different varieties, maturity levels, and lighting conditions will result in differences in color and texture. Therefore, relying solely on a fixed threshold or traditional segmentation methods is insufficient to balance accuracy and generalization. To address this, this invention employs supervised learning to acquire kernel annotation data and train the target detection model.

[0086] In one executable implementation, such as Figure 3 As shown, a labeling tool can be used to obtain rectangular bounding boxes for the grains in a set of corrected panoramic images, and each bounding box can be associated with a grain category label to form grain labeling data. It should be noted that, to avoid loss of labeling information due to occlusion, this invention allows appropriate overlap between bounding boxes and requires each bounding box to tightly surround the grain region, so that the model can simultaneously obtain the appearance features of the grain boundaries and the contextual features of occlusion overlap during the learning process. In one exemplary implementation, 610 panoramic images of ears of grain can be labeled, accumulating more than 300,000 valid targets, thus forming a dataset with sufficient coverage. It should be noted that the above scale is only an example and does not limit the invention to using a fixed number of samples or targets.

[0087] During the training phase, in this embodiment of the invention, the dataset can be divided into a training set and a validation set, and scale normalization and data augmentation processing can be performed on the training samples to obtain the input tensor. Further, in one executable implementation, a YOLOv8 series model can be selected as the object detection architecture. Iterative optimization training is performed on the input tensor, and the accuracy is evaluated on the validation set until the output model weight parameters meet a preset evaluation threshold, forming a seed object detection model. It should be noted that this invention does not limit the use of a specific version of the YOLO model or a fixed network width and depth configuration. The model structure can be equivalently replaced according to computational power and accuracy requirements, as long as its output can generate a seed detection result set and support subsequent spatial feature point construction.

[0088] S40: Input the corrected panoramic image into the grain target detection model to obtain a grain detection result set, and generate a grain spatial feature point set based on the grain detection result set.

[0089] Specifically, the corrected panoramic image is divided into multiple detection sub-images according to a preset window size, and seed target detection is performed on each detection sub-image to obtain a set of sub-image detection boxes; the set of sub-image detection boxes is mapped to the coordinate system of the corrected panoramic image based on the spatial index relationship of each detection sub-image to generate a panoramic detection box set; non-maximum suppression is performed on the panoramic detection box set to obtain a set of seed detection results, and the coordinates of the seed center point are calculated based on the geometric center of each detection box in the seed detection result set to form the set of seed spatial feature points.

[0090] In this embodiment of the invention, step S40 is used to convert the corrected panoramic image into a set of seed center points that can be used for graph structure analysis. It should be noted that the corrected panoramic image is typically a high-resolution image. Directly inputting the entire image into the detection network may lead to excessive memory usage or a decrease in inference speed. Furthermore, under extremely high resolution conditions, it may introduce scale mismatch, resulting in fluctuations in detection performance. Therefore, in one executable implementation, this invention employs a windowed detection strategy, dividing the corrected panoramic image into multiple detection sub-images according to a preset window size, and performing seed target detection on each detection sub-image to obtain a set of sub-image detection boxes.

[0091] In practical applications, to ensure that the detection results remain consistent in the panoramic coordinate system, it is necessary to maintain the spatial index relationship of each detection sub-image relative to the panoramic coordinate system. Specifically, the coordinates of the sub-image detection boxes can be mapped back to the corrected panoramic image coordinate system through translation and scaling, thereby obtaining a set of panoramic detection boxes. Furthermore, in one executable implementation, non-maximum suppression can be performed on the set of panoramic detection boxes to remove overlapping and duplicate boxes, outputting a set of seed detection results.

[0092] After obtaining the grain detection result set, this embodiment of the invention further calculates the geometric center of each detection box and uses the coordinates of the geometric center as the grain spatial feature points. It should be noted that the grain spatial feature point set is more suitable for ear row analysis than the detection boxes, because ear row analysis focuses on the topological arrangement of grains on the unfolded plane. Geometric center points can carry this arrangement information in a simpler data structure, while reducing the interference of frame scale differences on graph structure construction. The resulting grain spatial feature point set will be used as the direct input to step S50 and as a structural prior to assist in locating abnormal regions in the consistency verification and abnormal segment re-detection in step S60.

[0093] S50: Construct a grain spatial map based on the set of grain spatial feature points and perform panicle row analysis and row grain statistics to output the seed evaluation results containing the number of panicle rows, the number of grains per row, and the total number of grains.

[0094] Specifically, Delaunay triangulation is performed based on the set of spatial feature points of the grains to generate a set of adjacent edges. The set of spatial feature points of the grains is used as a set of nodes, and the set of adjacent edges is used as a set of edges to construct a grain space graph. For each adjacent edge in the grain space graph, the edge direction angle and edge length are calculated, and an edge cost is generated based on the edge direction angle and edge length. Based on the edge cost, Bellman-Ford path optimization is performed on the grain space graph to obtain an optimal edge sequence set under a preset cost criterion, and the ear row direction parameters are determined based on the direction statistics of the optimal edge sequence set. Based on the ear row direction parameters, the coordinates of the grain center points are projected onto the ear row orthogonal axis to obtain a row index projection value sequence. One-dimensional clustering is performed on the row index projection value sequence to obtain multiple ear row clusters, and the number of ear row clusters is determined as the number of ear rows. For each ear row cluster, the number of grains per row is determined based on the number of grain center points in the ear row cluster, and the total number of grains is calculated based on the number of grains per row and the number of ear rows.

[0095] In this embodiment of the invention, step S50 is used to recover the ear row direction from the set of spatial feature points of grains and complete the counting statistics. It should be noted that if the stripe or row structure is detected directly in the pixel domain, it is easily affected by differences in grain morphology, occlusion, missing grains and changes in light; therefore, this invention shifts the analysis object to the level of grain center points, and expresses the adjacency relationship through graph structure, thereby estimating the ear row direction at a more stable topological level.

[0096] In one executable implementation, Delaunay triangulation is performed on the set of feature points in the grain space to generate a set of adjacent edges, and a grain space graph is constructed using the center point as a node and the adjacent edges as edges. Subsequently, the edge orientation angle and edge length are calculated for each adjacent edge, and the edge cost is constructed accordingly. The calculation of the edge orientation angle satisfies:

[0097] ;

[0098] in, and The coordinates of the center point, Here, represents the direction angle. It should be noted that the edge cost can be designed in an equivalent form where higher direction consistency results in lower cost or higher direction consistency results in higher benefit. This invention does not limit the specific construction formula of the edge cost, but it limits it to be based on the direction angle and length information, so that path optimization tends to select the edge sequence that satisfies the grain-row orientation.

[0099] In a further executable implementation, Bellman-Ford path optimization can be performed on the grain space graph to obtain the optimal edge sequence set under a preset cost criterion. The directional statistics of the optimal edge sequence set can then be summarized to obtain the ear row direction parameters. It should be noted that the Bellman-Ford algorithm is used for global path cost optimization in this invention. Its result can be the shortest path, the maximum consistency path, or other equivalent optimal path forms, as long as its output can stably represent the direction of the grain rows.

[0100] After obtaining the ear row orientation parameters, in order to convert the two-dimensional point distribution into a clusterable one-dimensional row index, this invention projects the coordinates of the grain center point onto the orthogonal axis of the ear row, obtaining a sequence of row index projection values:

[0101] ;

[0102] in, Let be the projection value of the i-th center point onto the orthogonal axis. Then, perform one-dimensional clustering on the projection sequence to obtain multiple ear row clusters, and use the number of ear row clusters as the ear row number R. Within each ear row cluster, count the number of center points to obtain the corresponding row grain number. The final total number of grains is:

[0103] ;

[0104] It should be noted that in one alternative implementation, when the ear row structure is regular and the difference in the number of grains per row is small, the total number of grains can also be approximately obtained by multiplying R by the average number of grains per row. However, this method can still be equivalently transformed into summing the number of grains per row. Therefore, this invention uses summation statistics as a unified expression to correspond with the subsequent consistency verification mechanism.

[0105] In one exemplary implementation, such as Figure 4 As shown, the trained detection model can complete the count of ear rows and grains in a panoramic image within approximately 5 seconds. Furthermore, for panoramic images with a regular grain distribution, the accuracy rate for identifying the number of ear rows and grains can reach over 92%. It should be noted that the above performance indicators are related to hardware configuration, model size, and sample coverage, and do not constitute a limitation of the present invention.

[0106] Following this, a detection count value is determined based on the grain detection result set, and a statistical count value is determined based on the number of rows in the ear and the number of grains per row. A consistency score is calculated based on the detection count value and the statistical count value, and if the consistency score is greater than a preset threshold, it is determined that the consistency verification does not meet the preset conditions. When it is determined that the consistency verification does not meet the preset conditions, an abnormal segment mask is generated based on the spatial distribution of low-confidence detection boxes in the grain detection result set, and re-detection inference is performed on the image region defined by the abnormal segment mask to update the grain detection result set. Based on the updated grain detection result set, the seed evaluation result is updated and output.

[0107] In this embodiment of the invention, consistency verification and abnormal segment re-detection output are also included to further improve the reliability of the results. It should be noted that the total number of grains can be directly obtained from the detection result set as the detection count value. Statistical counts can also be indirectly obtained from ear row analysis and row grain statistics. When there are missed / false detections due to splicing distortion, local reflections, occlusion, or texture repetition, these two types of counting often produce deviations. Therefore, this invention introduces a consistency score to quantify the degree of difference:

[0108] ;

[0109] Where C is the consistency score, and the denominator is... Used to avoid a denominator of 0.

[0110] In this embodiment of the invention, when the consistency score is greater than a preset threshold, it is determined that the consistency verification does not meet the preset conditions, and at this time, the abnormal segment re-examination process is triggered. It should be noted that this re-examination process is different from the conventional inference process. It is not executed indiscriminately on all samples, but is only activated in the case of consistency anomalies, thereby achieving the goal of not increasing the overhead in normal scenarios and significantly improving the reliability in abnormal scenarios.

[0111] In one executable implementation, an anomaly mask can be generated based on the spatial distribution of low-confidence detection boxes. Specifically, the low-confidence detection boxes can be spatially aggregated in a panoramic coordinate system, and the aggregated region can be expanded to form the mask range, thereby obtaining the anomaly mask. Subsequently, re-detection inference is performed only on the image region defined by the anomaly mask, and the re-detection result is merged with the original detection result to update the seed detection result set. Furthermore, based on the updated detection result set, the graph construction, orientation resolution, and row-grain statistics process in step S50 are repeated to obtain the updated... and The consistency verification is then performed again until the consistency score meets the preset conditions or reaches the preset limit for the number of re-examinations, thus outputting the final examination result. It should be noted that the re-examination inference can adopt different threshold strategies, resolution strategies, or enhancement strategies than the initial inference. For example, the resolution can be increased in abnormal segments, the confidence threshold can be reduced, or stronger data augmentation methods can be used to improve the detection recall of occluded, reflective, and texture repetitive areas.

[0112] Therefore, this invention eliminates special distortions in the splicing stage by using a scanning error compensation tensor, and corrects residual uncertainties in the detection and statistical stages by using consistency verification and re-detection of abnormal sections, thereby significantly improving the stability and reproducibility of automated testing results.

[0113] like Figure 5As shown, this invention provides an automated crop testing system based on panoramic images of corn ears. The system is used to execute the aforementioned automated crop testing method based on panoramic images of corn ears. The system includes:

[0114] The acquisition unit is used to acquire the original strip sequence data of the line scan of the target corn ear rotating once, and to generate a strip pixel matrix set based on the original strip sequence data of the line scan.

[0115] The construction unit is used to construct a panoramic image of corn ears based on the strip pixel matrix set, generate a scanning error compensation tensor corresponding to each strip of the strip pixel matrix set, and perform non-rigid stitching correction on the panoramic image to obtain a corrected panoramic image.

[0116] The training unit is used to acquire seed annotation data for the corrected panoramic image and train a seed target detection model based on the seed annotation data.

[0117] The generation unit is used to input the corrected panoramic image into the grain target detection model to obtain a grain detection result set, and generate a grain spatial feature point set based on the grain detection result set;

[0118] The output unit is used to construct a grain spatial map based on the set of grain spatial feature points and perform ear row analysis and row grain statistics to output the seed evaluation results including the number of ear rows, the number of grains per row, and the total number of grains.

[0119] Those skilled in the art will understand that all or part of the steps in the methods of the above embodiments can be implemented by a program instructing related hardware. This program is stored in a storage medium and includes several instructions to cause a microcontroller, chip, or processor to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as a USB flash drive, a portable hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0120] The optional embodiments of the present invention have been described in detail above with reference to the accompanying drawings. However, the embodiments of the present invention are not limited to the specific details described above. Within the scope of the technical concept of the embodiments of the present invention, various simple modifications can be made to the technical solutions of the embodiments of the present invention, and these simple modifications all fall within the protection scope of the embodiments of the present invention. It should also be noted that the various specific technical features described in the above specific embodiments can be combined in any suitable manner without contradiction. To avoid unnecessary repetition, the embodiments of the present invention will not further describe the various possible combinations.

[0121] Furthermore, various different embodiments of the present invention can be combined in any way, as long as they do not violate the spirit of the embodiments of the present invention, they should also be regarded as the content disclosed by the embodiments of the present invention.

Claims

1. An automated method for variety assessment based on panoramic images of maize ears, characterized in that, The method includes: Acquire the raw strip sequence data of the line scan of a target corn ear rotating once, and generate a strip pixel matrix set based on the raw strip sequence data of the line scan; A panoramic image of corn ears is constructed based on the strip pixel matrix set, a scanning error compensation tensor corresponding to each strip of the strip pixel matrix set is generated, and non-rigid stitching correction is performed on the panoramic image to obtain a corrected panoramic image. Obtain seed annotation data for the corrected panoramic image, and train a seed target detection model based on the seed annotation data; The corrected panoramic image is input into the grain target detection model to obtain a grain detection result set, and a grain spatial feature point set is generated based on the grain detection result set; Based on the set of spatial feature points of the grains, a spatial graph of the grains is constructed and the row analysis and row grain statistics are performed to output the seed evaluation results containing the number of rows of the ears, the number of grains per row, and the total number of grains.

2. The automated seed assessment method based on panoramic images of maize ears according to claim 1, characterized in that, Acquire raw strip sequence data of a line scan of a target corn ear rotating once, and generate a strip pixel matrix set based on the raw strip sequence data, including: Acquire the line scan pixel sequence output by the line scan camera during the rotation of the target corn ear, and record each output line scan pixel sequence as strip scan row data; The strip scan line data are indexed and numbered according to the strip acquisition order to form a strip sequence index table containing a one-to-one correspondence between strip numbers and pixel values; The data from multiple consecutive strip scan rows are accumulated and stitched together according to a preset pixel width to generate a corresponding strip pixel matrix, and all strip pixel matrices are combined to form the strip pixel matrix set. The integrity check of the strip pixel matrix set is performed based on the strip sequence index table, so as to output the strip pixel matrix set when the check satisfies the preset width and height constraints.

3. The automated seed assessment method based on panoramic images of maize ears according to claim 1, characterized in that, A panoramic image of a corn ear is constructed based on the strip pixel matrix set. A scanning error compensation tensor corresponding to each strip of the strip pixel matrix set is generated. Non-rigid stitching correction is then performed on the panoramic image to obtain a corrected panoramic image, including: For adjacent strip pixel matrices in the strip pixel matrix set, extract the corresponding strip overlap window data, and calculate the original displacement estimation vector between strips based on the strip overlap window data; Temporal smoothing is performed on the original displacement estimation vector to obtain the strip-corrected displacement vector, and all strip-corrected displacement vectors are stacked according to the strip number to form a scan error compensation tensor; Based on the scanning error compensation tensor, perform non-rigid remapping of the strip pixel matrix set strip by strip, and then accumulate and stitch the remapped strip pixel matrix set according to the strip number order to generate the corrected panoramic image.

4. The automated seed assessment method based on panoramic images of maize ears according to claim 1, characterized in that, Obtaining seed annotation data for the corrected panoramic image includes: Obtain a set of corrected panoramic images, and generate corresponding image identification information for each image in the set of corrected panoramic images; For each corrected panoramic image, a set of rectangular bounding boxes covering all grain regions in the corrected panoramic image is obtained, and each rectangular bounding box is associated with the corresponding grain category label to form grain labeling data; The seed annotation data is paired and encapsulated with image labeling information to construct a dataset for model training, and overlap is allowed between the rectangular annotation boxes to preserve the seed occlusion relationship representation.

5. The automated seed assessment method based on panoramic images of maize ears according to claim 4, characterized in that, A grain target detection model is trained based on the aforementioned grain annotation data, including: The dataset is divided into a training set and a validation set to obtain a training sample set and a validation sample set. Image scale normalization and data augmentation are performed on the training sample set to generate input tensors for model training; The input tensor is iteratively optimized and trained based on a preset deep learning object detection architecture, and the training process is evaluated for accuracy based on the validation sample set to output model weight parameters that meet the preset evaluation threshold, thus forming the seed object detection model.

6. The automated seed assessment method based on panoramic images of maize ears according to claim 1, characterized in that, The corrected panoramic image is input into the grain target detection model to obtain a grain detection result set, and a grain spatial feature point set is generated based on the grain detection result set, including: The corrected panoramic image is divided into multiple detection sub-images according to a preset window size, and seed target detection is performed on each detection sub-image to obtain a set of sub-image detection boxes; Based on the spatial index relationship of each detection sub-image, the set of detection boxes of the sub-images is mapped to the calibrated panoramic image coordinate system to generate a panoramic detection box set; Non-maximum suppression is performed on the set of panoramic detection boxes to obtain a set of seed detection results, and the coordinates of the seed center point are calculated based on the geometric center of each detection box in the set of seed detection results to form the set of seed spatial feature points.

7. The automated seed assessment method based on panoramic images of maize ears according to claim 6, characterized in that, Constructing a grain spatial map based on the set of grain spatial feature points includes: Based on the set of feature points in the grain space, Delaunay triangulation is performed to generate a set of adjacent edges, and the set of feature points in the grain space is used as a set of nodes and the set of adjacent edges is used as a set of edges to construct a grain space graph; For each adjacent edge in the grain space graph, calculate the edge orientation angle and edge length, and generate the edge cost based on the edge orientation angle and edge length; Based on the edge cost, Bellman-Ford path optimization is performed on the grain space graph to obtain the optimal edge sequence set under the preset cost criterion, and the ear row direction parameters are determined based on the direction statistics of the optimal edge sequence set.

8. The automated seed assessment method based on panoramic images of maize ears according to claim 7, characterized in that, Perform ear row analysis and row grain statistics to output the seed evaluation results, including the number of ear rows, the number of grains per row, and the total number of grains. Based on the ear row direction parameters, the coordinates of the grain center point are projected onto the ear row orthogonal axis to obtain a row index projection value sequence; Perform one-dimensional clustering on the row index projection value sequence to obtain multiple ear row clusters, and determine the number of ear row clusters as the number of ear rows; For each ear row cluster, the number of kernels per row is determined based on the number of kernel center points within that ear row cluster, and the total number of kernels is calculated based on the number of kernels per row and the number of ear rows.

9. The automated seed assessment method based on panoramic images of maize ears according to claim 8, characterized in that, Before outputting the test results, the method further includes performing consistency verification on the total number of grains and the row grain count, and performing abnormal segment re-examination if the consistency verification does not meet the preset conditions, including: The detection count value is determined based on the grain detection result set, and the statistical count value is determined based on the number of rows in the ear and the number of grains per row; A consistency score is calculated based on the detection count value and the statistical count value, and if the consistency score is greater than a preset threshold, it is determined that the consistency verification does not meet the preset conditions. When the consistency verification fails to meet the preset conditions, an abnormal segment mask is generated based on the spatial distribution of the low-confidence detection boxes in the seed detection result set, and re-detection inference is performed on the image region defined by the abnormal segment mask to update the seed detection result set. The seed testing results are updated and output based on the updated seed detection result set.

10. An automated seed evaluation system based on panoramic images of maize ears, characterized in that, The system is used to execute the automated seed assessment method based on panoramic images of maize ears as described in any one of claims 1-9, and the system comprises: The acquisition unit is used to acquire the original strip sequence data of the line scan of the target corn ear rotating once, and to generate a strip pixel matrix set based on the original strip sequence data of the line scan. The construction unit is used to construct a panoramic image of corn ears based on the strip pixel matrix set, generate a scanning error compensation tensor corresponding to each strip of the strip pixel matrix set, and perform non-rigid stitching correction on the panoramic image to obtain a corrected panoramic image. The training unit is used to acquire seed annotation data for the corrected panoramic image and train a seed target detection model based on the seed annotation data. The generation unit is used to input the corrected panoramic image into the grain target detection model to obtain a grain detection result set, and generate a grain spatial feature point set based on the grain detection result set; The output unit is used to construct a grain spatial map based on the set of grain spatial feature points and perform ear row analysis and row grain statistics to output the seed evaluation results including the number of ear rows, the number of grains per row, and the total number of grains.

Images