A corn ear grain character image recognition method and system based on deep learning

Abstract

The application discloses a kind of corn ear grain character image recognition method and system based on deep learning, belong to crop phenotype image recognition technical field.The method of the present application is: collecting corn ear panorama image covering the distribution of grain around ear, after background segmentation, illumination correction and cylindrical development geometric regulation, generate regular ear image and pixel physical size mapping parameter;According to regular ear image, extract grain row and column boundary saliency features and construct initial grid topology structure, while extracting multi-scale grain deep semantic features by convolutional neural network;After feature fusion, generate grain instance segmentation mask, complete grain center pixel correction and generate multi-attribute integrated representation graph;Based on multi-attribute integrated representation graph, solve single grain character data, and complete corn ear grain population derived index estimation according to all single grain character data.The present application has high detection precision, and the parameters cover the whole, which can be applied to corn breeding high-throughput test breeding scene.

Description

Technical Field

[0001] This invention relates to the field of crop phenotypic image recognition technology, specifically to a method and system for recognizing maize ear and kernel traits based on deep learning. Background Technology

[0002] The characteristics of maize ears and kernels are core indicators for variety selection, approval, and yield measurement. They encompass individual kernel traits such as kernel length and width, as well as population-derived indicators like total kernel count and uniformity. Traditional maize variety assessment relies primarily on manual measurement and counting, which suffers from low efficiency, high subjectivity, and large repeated measurement errors, making it unsuitable for the high-throughput, large-scale testing demands of modern maize breeding. With the development of machine vision and deep learning technologies, image-based automatic identification methods for maize ear and kernel traits are gradually becoming the mainstream research and application direction in the industry.

[0003] Most existing corn ear image recognition solutions use single-view or limited-view acquisition of ear images, which cannot completely cover the kernel distribution around the entire ear. This can easily lead to errors in ear row count and insufficient accuracy in total kernel estimation due to kernel occlusion and view omissions. Some multi-view acquisition solutions lack accurate cylindrical unfolding geometric regularization processing. The deformation caused by the cylindrical imaging of the ear can lead to distortion in kernel size measurement. The mapping accuracy between pixels and physical size is low, which directly affects the accuracy of trait parameter calculation.

[0004] At the level of kernel segmentation and trait calculation, existing methods mostly directly use general instance segmentation networks for kernel detection, failing to fully utilize the orderly arrangement of maize kernel rows and columns as a priori constraint. When faced with scenarios such as densely clustered kernels and blurred boundaries, problems such as over-segmentation, under-segmentation, and boundary missegmentation easily occur, resulting in insufficient accuracy in kernel center localization. Kernel recognition schemes based on traditional image processing rely on manually designed feature operators, which have poor robustness to different maize varieties, kernel colors, and imaging environments, and limited generalization ability. In addition, existing recognition methods mostly focus on single-index detection and lack a unified pixel-level multi-attribute representation framework, making it difficult to simultaneously and accurately output the geometric traits of individual kernels and the derived indicators of the entire ear population, thus failing to fully meet the needs of high-throughput, multi-parameter integrated seed evaluation applications in maize breeding. Summary of the Invention

[0005] The purpose of this invention is to provide a method and system for image recognition of maize ear and kernel traits based on deep learning, so as to solve the problems of ear imaging distortion, insufficient kernel segmentation accuracy, and difficulty in integrated high-throughput detection of multiple kernel traits mentioned in the background art.

[0006] To address the aforementioned technical problems, this invention provides a deep learning-based image recognition method for maize ear kernel traits, comprising the following steps:

[0007] S1. Acquire a panoramic image of corn ears covering the entire circumference of the ear and the distribution of kernels. Perform background segmentation, illumination correction and cylindrical unfolding geometric regularization on the panoramic image of corn ears in sequence to generate regularized ear images and pixel physical size mapping parameters.

[0008] S2. Based on the growth characteristics of orderly arrangement of corn kernel rows and columns, extract the salient features of kernel row and column boundaries from regular ear images, and detect candidate kernel center points and construct an initial grid topology.

[0009] S3. Extract multi-scale deep semantic feature maps of grains from regular ear images using a convolutional neural network;

[0010] S4. Integrate the saliency features of grain row and column boundaries with multi-scale deep semantic features of grains to generate a grain instance segmentation mask. Integrate the candidate points of grain center in the topology with multi-scale deep semantic feature maps of grains to complete the correction of grain center pixels and construct pixel relative coordinate representation. Combine the pixel physical size mapping parameters to generate a multi-attribute integrated representation map.

[0011] S5. Calculate single grain trait data based on multi-attribute integrated characterization graph, and estimate the population-derived indices of maize ear grains based on all single grain trait data.

[0012] Preferably, S1 specifically comprises:

[0013] S11. Convert the panoramic image of corn ears to the HSV color space, extract the main body region of the ears by color threshold segmentation to generate an effective region mask, and extract the effective region of the ears from the panoramic image of corn ears based on the effective region mask.

[0014] S12. The homomorphic filtering algorithm is used to perform illumination compensation on the effective area of ​​the ear of corn to eliminate the light and dark gradient and local shadows caused by the curved surface imaging of the corn ear of corn, so as to obtain the illumination-corrected image of the corn ear of corn.

[0015] S13. Based on the cylindrical projection geometric model, the illumination-corrected corn ear image is subjected to cylindrical unfolding processing to convert the three-dimensional cylindrical grain distribution projection into a two-dimensional rectangular panoramic image.

[0016] S14. Perform axial alignment and size normalization on the two-dimensional panoramic image, calculate the pixel physical size mapping parameters in combination with the camera calibration parameters, and output a regular ear of fruit image with matching pixel physical size mapping parameters.

[0017] Preferably, the method for extracting the salient features of the grain row and column boundaries is as follows:

[0018] A vertical edge detection operator is used to extract the circumferential column boundary response of grains in a regular ear image, and a horizontal edge detection operator is used to extract the axial row boundary response of grains in a regular ear image.

[0019] Based on the growth prior of circumferential periodicity of corn ear rows and axial continuity of kernel columns, the boundary responses of kernel axial rows and circumferential columns are filtered for noise reduction and feature enhancement.

[0020] The enhanced grain axial row boundary response and the grain circumferential column boundary response are normalized and fused to generate salient features of grain row and column boundaries.

[0021] Preferably, the method for detecting candidate points at the center of the grain and constructing an initial grid topology is as follows:

[0022] The Gaussian difference speckle detection algorithm is used to extract local brightness extreme points in regular ear images as initial candidate points for grain centers;

[0023] Circumferential clustering is performed on each initial candidate grain center to determine the number of ear rows and complete the ear row division. Within each ear row, the grain centers are sorted axially and missing initial candidate grain centers are filled in. Based on this, a set of candidate grain center points with row and column affiliation is obtained.

[0024] The row and column arrangement of the candidate grain center point set is topologically verified by combining the growth characteristics of the staggered arrangement of grains in adjacent ear rows, and an initial grid topology structure with row and column numbers is generated.

[0025] Preferably, the backbone network is MobileNetV3, combined with a feature pyramid network structure. The backbone network extracts four hierarchical feature maps of different resolutions from the regular ear of grain image, corresponding to shallow detail features to deep semantic features. The feature pyramid network completes the hierarchical fusion of multi-scale features through top-down upsampling and lateral skip connections, and outputs a multi-scale deep semantic feature map of grains with the same spatial size as the input regular ear of grain image.

[0026] Preferably, the method for obtaining the seed instance segmentation mask is as follows:

[0027] The saliency features of grain row and column boundaries are used as spatial attention weights and fused with multi-scale grain deep semantic feature maps in a channel-by-channel weighted manner to obtain multi-scale fused feature maps.

[0028] A lightweight segmentation convolutional head is used to decode the multi-scale fused feature map to generate a boundary probability map. The boundary probability map is used to characterize the probability that a pixel belongs to the inside of a seed, the seed boundary, and the background, providing boundary constraints for seed instance segmentation.

[0029] Using the candidate points of the seed center in the initial grid topology as seed points, seed-guided watershed segmentation is performed on the boundary probability map to obtain the seed instance segmentation mask.

[0030] Preferably, the method for correcting the seed center pixel and constructing the pixel relative coordinate representation is as follows:

[0031] Using the candidate point of the seed center in the initial grid topology as the anchor point, the center pixel offset is regressed in the corresponding local feature region of the multi-scale seed deep semantic feature map. The pixel coordinates of the candidate point of the seed center are corrected at the pixel level according to the center pixel offset to obtain the seed center coordinates that match the actual geometric center of the seed.

[0032] Using the center coordinates of the grain as a reference, the relative position parameters of each effective pixel region corresponding to each grain are calculated and stored relative to the center of the grain, generating a pixel relative coordinate representation. The relative position parameters include rectangular relative coordinates and polar relative coordinates.

[0033] Preferably, the method for generating the multi-attribute integrated representation map is as follows:

[0034] Unify the pixel physical size mapping parameters, seed instance segmentation mask and pixel relative coordinate representation into the same coordinate system, and perform channel splicing with the seed identity as the association key;

[0035] Each pixel is assigned a seed identity attribute, a relative position attribute, a boundary attribute, and a geometric mapping attribute to form a multi-attribute integrated representation map in which each pixel carries multi-dimensional seed association information.

[0036] Preferably, S5 specifically includes:

[0037] Based on the multi-attribute integrated characterization map, the grain length, grain width, projected area, grain fullness and grain shape index of a single grain are obtained through geometric calculation. At the same time, the weight of a single grain is estimated by a pre-trained regression model with grain length, grain width, projected area and fullness as input features.

[0038] Collect data on the traits of all individual kernels in the whole ear of maize, and calculate and output the population-derived indicators of maize ear kernels, including the total number of kernels in the ear, the weight of kernels per ear, the number of rows in the ear, the number of kernels per row, and the uniformity of kernel traits.

[0039] This invention also provides a deep learning-based image recognition system for maize ear and kernel traits, used to implement the aforementioned deep learning-based image recognition method for maize ear and kernel traits, comprising:

[0040] The image acquisition and preprocessing module is used to acquire panoramic images of corn ears covering the entire circumference of the ear and the distribution of kernels. The panoramic images of corn ears are then subjected to background segmentation, illumination correction and cylindrical unfolding geometric regularization to generate regularized ear images and pixel physical size mapping parameters.

[0041] The feature extraction and topology construction module is used to extract the salient features of kernel row and column boundaries from regular ear images based on the growth characteristics of orderly arrangement of corn kernel rows and columns, while detecting candidate kernel center points and constructing an initial grid topology structure.

[0042] The semantic extraction module is used to extract multi-scale deep semantic feature maps of grains from regular ear images using a convolutional neural network.

[0043] The representation generation module is used to fuse the saliency features of grain row and column boundaries with multi-scale deep semantic features of grains to generate a grain instance segmentation mask, fuse the candidate points of grain center in the topology with multi-scale deep semantic feature maps of grains to complete the correction of grain center pixels and construct pixel relative coordinate representation, and combine the pixel physical size mapping parameters to generate a multi-attribute integrated representation map.

[0044] The trait calculation and statistics module is used to calculate the single grain trait data based on the multi-attribute integrated characterization graph, and to estimate the population-derived indicators of maize ear grains based on all single grain trait data.

[0045] Due to the adoption of the above technical solution, the technical progress achieved by this invention compared to the prior art is as follows:

[0046] 1. This invention adopts a full-circle panoramic image acquisition scheme, combined with HSV color space background segmentation, homomorphic filtering illumination correction and cylindrical unfolding geometric regularization process, to convert the three-dimensional cylindrical grain distribution into a distortion-free two-dimensional regular image. At the same time, it generates accurate pixel physical size mapping parameters, completely restores the true shape of all grains in the whole ear, eliminates measurement errors caused by missing viewpoints, surface deformation and uneven brightness from the image source, and improves the basic accuracy of grain size measurement.

[0047] 2. This invention integrates the saliency prior of grain row and column boundaries extracted based on crop growth characteristics with multi-scale deep semantic features of MobileNetV3-FPN. It uses Gaussian difference spot detection to build a grid topology structure as a watershed to segment seeds, and then uses feature offset regression to accurately correct the center pixel coordinates of the grains. Combining crop growth priors and deep learning features, it can effectively distinguish adhered grains, and has stronger adaptability to maize ears under multiple varieties and complex lighting conditions. The accuracy of grain instance segmentation and center localization is greatly improved.

[0048] 3. This invention integrates pixel physical size mapping parameters, grain instance segmentation masks, and pixel relative coordinate representations to construct a multi-attribute integrated representation map, enabling each pixel to carry multi-dimensional grain association information. Based on this representation system, it can not only calculate single grain traits such as grain length, grain width, projected area, and single grain weight, but also batch count population-derived indicators such as total number of grains per ear, number of rows per ear, and number of grains per row, realizing integrated and simultaneous detection of multiple grain traits, and adapting to the actual needs of large-scale, multi-parameter rapid seed evaluation in maize breeding. Attached Figure Description

[0049] Figure 1 This is a schematic diagram of the method flow of the present invention. Detailed Implementation

[0050] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the following description is provided in conjunction with the appendix. Figure 1 The present invention will be further described in detail below with reference to specific embodiments. It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of the invention.

[0051] like Figure 1 As shown in the figure, this embodiment provides a deep learning-based image recognition method for corn ear kernel traits, which is implemented according to the following steps:

[0052] S1. Panoramic Image Acquisition and Geometric Regularization of Corn Ears

[0053] A panoramic image of a corn ear covering the entire circumference of the ear and the distribution of kernels was acquired. Background segmentation, illumination correction, and cylindrical unfolding geometric regularization were then performed on the panoramic image to generate a regularized ear image and pixel physical size mapping parameters. The specific steps are as follows:

[0054] S11. Background Segmentation and Effective Region Extraction: A 2048×2048 resolution CMOS industrial camera was used with a uniformly rotating stage. The corn ear was vertically fixed at the center of the stage. During one rotation of the stage, 20 side view images were continuously acquired. A cylindrical stitching algorithm was used to generate a panoramic image of the corn ear covering the entire circumference of the ear and ensuring that all kernels on the surface of the corn ear were presented without obstruction. Subsequently, the panoramic image of the corn ear was converted from the RGB color space to the HSV color space, and a threshold range was set according to the typical color characteristics of the corn ear: hue H∈[15°,45°], saturation S∈[0.2,1.0], brightness V∈[0.15,0.95], the pixels of the panoramic image of corn ears are binarized to obtain an initial ear mask. Then, morphological opening operations are performed on the initial ear mask in sequence using 3×3 rectangular structuring elements to remove isolated noise points and morphological closing operations to fill internal holes, to obtain the final effective region mask. The panoramic image of corn ears is cropped along the minimum bounding rectangle of the effective region mask to remove large background areas, resulting in an effective ear region that only contains the main body of the ear.

[0055] S12. Homomorphic Filtering Illumination Correction: A Gaussian homomorphic filtering algorithm is used to compensate for illumination in the effective area of ​​the ear. First, the effective area of ​​the ear is converted to the logarithmic domain and decomposed into illuminance and reflection components. The Gaussian homomorphic filter is used to suppress low frequencies of the illuminance component and enhance high frequencies of the reflection component. The parameters of the Gaussian homomorphic filter can be set as follows: low frequency gain 0.5, high frequency gain 2.0, and cutoff frequency 80. After filtering, the image is converted back to the spatial domain to eliminate the "bright center, dark edge" brightness gradient caused by the cylindrical imaging of the corn ear, as well as the local shadows in the kernel depressions, thus obtaining the illumination-corrected corn ear image.

[0056] S13. Cylindrical Unfolding Geometric Transformation: Based on the cylindrical projection geometric model, cylindrical unfolding is performed on the illumination-corrected corn ear image. First, the longitudinal central axis of the effective area of ​​the ear is extracted as the central axis of the cylinder. The radial contour of the ear is fitted row by row along the axis to obtain the function r(x) of the ear radius changing with the axial coordinate. A cylindrical unfolding coordinate system is established with the axial direction as the vertical axis and the circumferential unfolding angle as the horizontal axis. According to the cylindrical orthographic projection mapping relationship, each pixel (u,v) in the illumination-corrected corn ear image is mapped to the unfolded two-dimensional coordinates (u',v'). Based on this, the three-dimensional cylindrical grain distribution is converted into a two-dimensional rectangular panoramic image after cylindrical unfolding through pixel-by-pixel coordinate mapping. The invalid edge areas of the two-dimensional rectangular panoramic image are filled by bilinear interpolation to ensure the texture continuity of the two-dimensional rectangular panoramic image.

[0057] S14. Axial alignment and size normalization mapping: Fit the central axis of the ear of fruit using the least squares method, rotate the two-dimensional rectangular panoramic image to make the axis of the ear of fruit parallel to the vertical direction of the image, and complete the axial alignment; normalize the two-dimensional rectangular panoramic image after axial alignment to a fixed size with a height of 1024 pixels, and the width is adaptively adjusted according to the circumferential length of the ear of fruit.

[0058] Combining the camera intrinsic parameters and object distance parameters obtained through pre-calibration using a checkerboard pattern, as well as the actual radial dimensions of the fruit ears, the actual physical dimensions of each pixel in the two-dimensional rectangular panoramic image after axial alignment and size normalization are calculated, generating a pixel physical dimension mapping parameter table that corresponds one-to-one with the image pixels; finally, a regular fruit ear image with matching pixel physical dimension mapping parameters is output.

[0059] S2. Prior Feature Extraction and Topology Construction

[0060] Based on the growth characteristics of orderly row and column arrangement of maize kernels, the salient features of kernel row and column boundaries are extracted from regular ear images. At the same time, candidate kernel center points are detected and an initial grid topology is constructed. The specific steps are as follows:

[0061] S21. Extraction of salient features of grain row and column boundaries

[0062] A 3×3 vertical Sobel operator is used to convolve the regular ear image to obtain the circumferential column boundary response map of the grains, which can reflect the salience of the vertical boundaries between grain columns; a 3×3 horizontal Sobel operator is used to convolve the image to obtain the axial row boundary response map of the grains, which can reflect the salience of the horizontal boundaries between grain rows.

[0063] Based on the growth prior of the circumferential periodicity of maize ear rows, namely the inherent regularity of the circumferential arrangement of maize kernels along the circumferential direction and the circumferential repetition of the ear row boundaries, a one-dimensional periodic filter is applied to the kernel circumferential column boundary response map. The ear row period width is calculated through autocorrelation analysis, and the boundary response within the period is weighted to enhance and suppress non-periodic random noise. Based on the growth prior of the axial continuity of kernel rows, namely the sequential and continuous arrangement of kernels in the same ear row along the ear axis and the axial extension of the row boundaries, an axial one-dimensional Gaussian filter is applied to the kernel axial row boundary response map to strengthen the response intensity of continuous row boundaries and weaken discontinuous pseudo-boundaries.

[0064] The enhanced seed circumferential row and column boundary response maps are normalized to the [0,1] interval, and then fused point by point according to the maximum value of the pixel to generate a seed row and column boundary saliency feature map. The higher the pixel value in the seed row and column boundary saliency feature map, the greater the probability that the position belongs to the seed boundary.

[0065] S22. Seed Center Candidate Point Detection and Initial Mesh Topology Construction

[0066] The Gaussian difference speckle detection algorithm is used to extract local brightness extrema in regular ear images. Specifically, a 5-layer Gaussian scale pyramid is constructed, and the difference between adjacent Gaussian images is used to obtain a difference Gaussian image. Local extrema are detected in scale space and image space, and extrema with response values ​​higher than the threshold T=0.12 are selected as initial candidate points for grain center. Each initial candidate point for grain center corresponds to the preliminary center position of a grain.

[0067] K-means clustering is performed on all initial candidate points of grain centers along the horizontal (circumferential) direction of the regular ear image. The optimal number of clusters is determined by the silhouette coefficient method, which is the number of rows of the ear. Each cluster corresponds to one row of ears. Then, within each row of ears, the initial candidate points of grain centers are sorted in ascending order according to the vertical (axial) direction of the regular ear image. The axial average distance between adjacent initial candidate points of grain centers is calculated. When the axial average distance is greater than 1.5 times the average distance, it is determined to be a missing grain. Virtual initial candidate points of grain centers are filled in at the midpoint of the distance. Finally, a set of candidate points of grain centers with the same number of grains in each row and row and column affiliation is obtained.

[0068] Since corn ears and kernels naturally exhibit an ordered grid-like growth pattern, this study combines the staggered arrangement of kernels in adjacent rows of corn ears to verify the circumferential offset of the kernel centers in adjacent rows. This circumferential offset represents the positional deviation of the kernel center at the corresponding position in the adjacent row in the circumferential direction. If the circumferential offset is less than 1 / 3 of the row spacing, it is determined to be a topological anomaly, which is corrected by fine-tuning the position of the candidate kernel center point. Finally, a unique row number and column number are assigned to each candidate kernel center point to generate an initial grid topology structure with row and column numbers.

[0069] S3. Multi-scale deep semantic feature extraction

[0070] Multi-scale deep semantic feature maps of grains are extracted from regular ear images using a convolutional neural network. Specifically, MobileNetV3-large is used as the backbone network, combined with a feature pyramid network (FPN) structure. The backbone network contains four downsampling feature levels with strides of 2, 4, 8, and 16, respectively. After inputting a regular ear image of size H×W, the backbone network outputs four feature maps of decreasing resolution: C1 (H / 2×W / 2), C2 (H / 4×W / 4), C3 (H / 8×W / 8), and C4 (H / 16×W / 16). C1 and C2 focus on shallow details such as grain edges and textures, while C3 and C4 focus on deep semantic features such as grain type and overall morphology. H and W are...

[0071] The feature pyramid network adopts a top-down fusion path. The deepest feature map C4 is upsampled by 2 times bilinearly and fused with C3 element-wise to obtain P3. P3 is upsampled by 2 times and fused with C2 to obtain P2. P2 is upsampled by 2 times and fused with C1 to obtain P1. Finally, the number of channels of P1 is adjusted by 1×1 convolution, and the output is a multi-scale deep semantic feature map of grains with the same spatial size as the input regular ear image, realizing multi-scale fusion of shallow details and deep semantics.

[0072] S4. Generation of Multi-Attribute Integrated Representation Map

[0073] A seed instance segmentation mask is generated by fusing seed row and column boundary saliency features with multi-scale seed deep semantic features. Seed center candidate points based on topological structure are fused with multi-scale seed deep semantic feature maps to correct seed center pixels and construct pixel relative coordinate representations. Finally, a multi-attribute integrated representation map is generated by combining pixel physical size mapping parameters. The specific steps are as follows:

[0074] S41. Generation of Seed Instance Segmentation Mask

[0075] The saliency feature map of the grain row and column boundaries is expanded to the same number of channels as the multi-scale deep semantic feature map of grains through channel replication, and used as the spatial attention weight map. The spatial attention weight map and the multi-scale deep semantic feature map of grains are then multiplied pixel by pixel in a channel-wise weighted manner to obtain a multi-scale fusion feature map, which strengthens the feature response of the grain boundary region and suppresses irrelevant features of the background and the inside of the grains.

[0076] A lightweight segmentation convolutional head is used to decode the multi-scale fused feature map. This lightweight segmentation convolutional head consists of three 3×3 convolutional layers and one 1×1 output convolutional layer, outputting a 3-channel boundary probability map. The three channels correspond to the probability values ​​of three types of pixels: inside the seed, the seed boundary, and the background, respectively, with a value range of [0,1], providing boundary constraints for seed instance segmentation.

[0077] Using each candidate seed center point in the initial grid topology as an independent seed point, a watershed transformation is performed on the seed boundary channel of the boundary probability map. Using the pixel region with high boundary probability as the segmentation line, all image pixel regions corresponding to the entire boundary probability map are divided into seed regions with the same number of seed points. Each independent closed segmentation region corresponds to a seed instance. Finally, a binary seed instance segmentation mask is output, in which each seed region has a unique instance number.

[0078] S42. Grain center pixel correction and pixel relative coordinate representation

[0079] Using each candidate seed center point in the initial grid topology as an anchor point, a local feature region of size 32×32 centered on the anchor point is extracted from the multi-scale seed deep semantic feature map. The deep semantic feature vector of the local feature region is regressed through two fully connected layers to output the center pixel offset of the anchor point: the horizontal offset Δx and the vertical offset Δy. The pixel coordinates (x, y) of the candidate seed center point are added to the center pixel offset to obtain the corrected seed center coordinates (x+Δx, y+Δy), realizing pixel-level center localization correction so that the center coordinates match the actual geometric center of the seed.

[0080] For each seed instance, taking the corrected seed center coordinates as the origin, the relative position parameters are calculated pixel by pixel for all valid pixels within the segmentation mask of the seed instance. The relative position parameters include rectangular relative coordinates and polar relative coordinates. The rectangular relative coordinates are the horizontal and vertical differences between the absolute coordinates of the pixel and the coordinates of the seed center. The polar relative coordinates are the polar radius distance from the pixel to the seed center and the polar angle of the pixel relative to the seed center. The above relative coordinate parameters are stored one-to-one with the pixels to generate a pixel relative coordinate representation map covering the entire image. Each pixel carries the relative position information of its seed.

[0081] S43. Generation of Multi-Attribute Integrated Representation Map

[0082] The pixel physical size mapping parameters, grain instance segmentation mask, and pixel relative coordinate representation are unified under the pixel coordinate system of the regularized ear image. Using the grain instance number (grain identity) as the association key, they are stitched together along the channel dimension to construct a multi-channel, multi-attribute integrated representation map. Each pixel in this multi-attribute integrated representation map contains the following four-dimensional attributes:

[0083] Seed identity attribute: Corresponds to the seed instance number, identifying the individual seed to which the pixel belongs;

[0084] Boundary attributes: indicate whether a pixel belongs to the inside of a seed, the boundary of a seed, or the background;

[0085] Relative position attribute: includes Cartesian relative coordinates and polar relative coordinates, representing the positional relationship of a pixel relative to the center of its seed;

[0086] Geometric mapping attribute: The corresponding pixel physical size mapping parameter, which represents the actual physical size of the pixel.

[0087] This multi-attribute integrated characterization map allows for the direct acquisition of all associated attributes of any grain through pixel indexing, providing a unified data foundation for subsequent trait calculations.

[0088] S5. Grain trait calculation and population index estimation

[0089] The single grain trait data are calculated based on the multi-attribute integrated characterization graph, and the population-derived indexes of maize ears and grains are estimated based on all single grain trait data.

[0090] The specific calculation of single grain trait data is as follows: For each grain instance, combined with the geometric mapping parameters of the multi-attribute integrated characterization map, the corresponding single grain trait data is obtained through geometric calculation and model regression. The single grain trait data includes grain length, grain width, projected area, grain plumpness, and grain shape index. The grain length is obtained by extracting the two endpoints corresponding to the maximum extreme diameter on the grain outline, calculating the pixel distance between the two points, and multiplying it by the pixel physical size mapping parameter at the corresponding position. The grain width is obtained by calculating the maximum pixel distance on the grain outline in the direction perpendicular to the grain length and multiplying it by the pixel physical size mapping parameter. The projected area is obtained by statistically analyzing the effective images within the grain segmentation mask. The total number of pixels is multiplied by the physical area corresponding to a single pixel. The kernel fullness is obtained by calculating the ratio of the kernel outline area to the area of ​​the kernel's smallest bounding rectangle; a higher value indicates better kernel fullness. The kernel shape index is obtained by calculating the ratio of kernel length to kernel width, which is used to characterize the length-to-width ratio of the kernel. The single kernel weight is obtained by using a pre-trained random forest regression model, with kernel length, kernel width, projected area, and fullness as input features, and outputting an estimated value of the single kernel weight. This random forest regression model is pre-trained with at least 12 maize varieties and at least 6,000 kernels as measured samples, and has good generalization ability for different maize varieties.

[0091] The estimation of corn ear kernel population derived indices is specifically as follows: Collect data on the traits of all individual kernels in the entire ear, calculate and output corn ear kernel population derived indices including total number of kernels, single ear kernel weight, number of rows in the ear, number of kernels per row, and kernel trait evenness; the total number of kernels in the ear is obtained by counting the total number of kernel instances; the single ear kernel weight is obtained by summing the estimated single kernel weights of all individual kernels, representing the total weight of all kernels in a single corn ear; the number of rows in the ear is directly output from the clustering results of the initial grid topology, i.e., the total number of rows in the ear; the number of kernels per row is obtained by counting the number of kernels in each row in the ear, outputting the average number of kernels per row for all rows, and simultaneously outputting the maximum and minimum values ​​of the number of kernels per row; for kernel trait evenness, calculate the coefficient of variation (standard deviation / mean) of kernel length, kernel width, and single kernel weight for all kernels in the entire ear. The smaller the coefficient of variation, the higher the trait evenness, and output the evenness values ​​of each trait simultaneously.

[0092] The aforementioned population-derived indicators and single-kernel trait data for maize ears and kernels can comprehensively support high-throughput variety testing, varietal trait evaluation, and selection of superior lines in the maize breeding process.

[0093] The above description describes specific embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any modifications or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.

Claims

1. A method for image recognition of maize ear kernel traits based on deep learning, characterized in that, Includes the following steps: S1. Acquire a panoramic image of corn ears covering the entire circumference of the ear and the distribution of kernels. Perform background segmentation, illumination correction and cylindrical unfolding geometric regularization on the panoramic image of corn ears in sequence to generate regularized ear images and pixel physical size mapping parameters. S2. Based on the growth characteristics of orderly arrangement of corn kernel rows and columns, extract the salient features of kernel row and column boundaries from regular ear images, and at the same time detect candidate kernel center points and construct an initial grid topology. S3. Extract multi-scale deep semantic feature maps of grains from regular ear images using a convolutional neural network; S4. Integrate the saliency features of grain row and column boundaries with multi-scale deep semantic features of grains to generate a grain instance segmentation mask. Integrate the candidate points of grain center in the topology with multi-scale deep semantic feature maps of grains to complete the correction of grain center pixels and construct pixel relative coordinate representation. Combine the pixel physical size mapping parameters to generate a multi-attribute integrated representation map. S5. Calculate single grain trait data based on multi-attribute integrated characterization graph, and estimate the population-derived indices of maize ear grains based on all single grain trait data.

2. The method for image recognition of corn ear kernel traits according to claim 1, characterized in that, Specifically, S1 is: S11. Convert the panoramic image of corn ears to the HSV color space, extract the main body region of the ears by color threshold segmentation to generate an effective region mask, and extract the effective region of the ears from the panoramic image of corn ears based on the effective region mask. S12. The homomorphic filtering algorithm is used to perform illumination compensation on the effective area of ​​the ear of corn to eliminate the light and dark gradient and local shadows caused by the curved surface imaging of the corn ear of corn, so as to obtain the illumination-corrected image of the corn ear of corn. S13. Based on the cylindrical projection geometric model, the illumination-corrected corn ear image is subjected to cylindrical unfolding processing to convert the three-dimensional cylindrical grain distribution projection into a two-dimensional rectangular panoramic image. S14. Perform axial alignment and size normalization on the two-dimensional panoramic image, calculate the pixel physical size mapping parameters in combination with the camera calibration parameters, and output a regular ear of fruit image with matching pixel physical size mapping parameters.

3. The method for image recognition of corn ear kernel traits according to claim 2, characterized in that, The method for extracting the salient features of the grain row and column boundaries is as follows: A vertical edge detection operator is used to extract the circumferential column boundary response of grains in a regular ear image, and a horizontal edge detection operator is used to extract the axial row boundary response of grains in a regular ear image. Based on the growth prior of circumferential periodicity of corn ear rows and axial continuity of kernel columns, the boundary responses of kernel axial rows and circumferential columns are filtered for noise reduction and feature enhancement. The enhanced grain axial row boundary response and the grain circumferential column boundary response are normalized and fused to generate salient features of grain row and column boundaries.

4. The method for image recognition of corn ear kernel traits according to claim 2, characterized in that, The method for detecting candidate points at the center of grains and constructing an initial grid topology is as follows: The Gaussian difference speckle detection algorithm is used to extract local brightness extreme points in regular ear images as initial candidate points for grain centers; Circumferential clustering is performed on each initial candidate grain center to determine the number of ear rows and complete the ear row division. Within each ear row, the grain centers are sorted axially and missing initial candidate grain centers are filled in. Based on this, a set of candidate grain center points with row and column affiliation is obtained. The row and column arrangement of the candidate grain center point set is topologically verified by combining the growth characteristics of the staggered arrangement of grains in adjacent ear rows, and an initial grid topology structure with row and column numbers is generated.

5. The method for image recognition of corn ear kernel traits according to claim 2, characterized in that, The system uses MobileNetV3 as its backbone network and incorporates a feature pyramid network structure. The backbone network extracts four hierarchical feature maps of different resolutions from the regular ear of grain image, corresponding to shallow detail features to deep semantic features. The feature pyramid network completes the hierarchical fusion of multi-scale features through top-down upsampling and lateral skip connections, and outputs a multi-scale deep semantic feature map of grains with the same spatial size as the input regular ear of grain image.

6. The method for image recognition of maize ear kernel traits according to claim 3 or 5, characterized in that, The method for obtaining the seed instance segmentation mask is as follows: The saliency features of grain row and column boundaries are used as spatial attention weights and fused with multi-scale grain deep semantic feature maps in a channel-by-channel weighted manner to obtain multi-scale fused feature maps. A lightweight segmentation convolutional head is used to decode the multi-scale fused feature map to generate a boundary probability map. The boundary probability map is used to characterize the probability that a pixel belongs to the inside of a seed, the seed boundary, and the background, providing boundary constraints for seed instance segmentation. Using the candidate points of the seed center in the initial grid topology as seed points, seed-guided watershed segmentation is performed on the boundary probability map to obtain the seed instance segmentation mask.

7. The method for image recognition of corn ear kernel traits according to claim 4 or 5, characterized in that, The method for correcting the seed center pixel and constructing the pixel relative coordinate representation is as follows: Using the candidate point of the seed center in the initial grid topology as the anchor point, the center pixel offset is regressed in the corresponding local feature region of the multi-scale seed deep semantic feature map. The pixel coordinates of the candidate point of the seed center are corrected at the pixel level according to the center pixel offset to obtain the seed center coordinates that match the actual geometric center of the seed. Using the center coordinates of the grain as a reference, the relative position parameters of each effective pixel region corresponding to each grain are calculated and stored relative to the center of the grain, generating a pixel relative coordinate representation. The relative position parameters include rectangular relative coordinates and polar relative coordinates.

8. The method for image recognition of corn ear kernel traits according to claim 1, characterized in that, The method for generating the multi-attribute integrated representation map is as follows: Unify the pixel physical size mapping parameters, seed instance segmentation mask and pixel relative coordinate representation into the same coordinate system, and perform channel splicing with the seed identity as the association key; Each pixel is assigned a seed identity attribute, a relative position attribute, a boundary attribute, and a geometric mapping attribute to form a multi-attribute integrated representation map in which each pixel carries multi-dimensional seed association information.

9. The method for image recognition of maize ear kernel traits according to claim 8, characterized in that, Specifically, S5 is: Based on the multi-attribute integrated characterization map, the grain length, grain width, projected area, grain fullness and grain shape index of a single grain are obtained through geometric calculation. At the same time, the weight of a single grain is estimated by a pre-trained regression model with grain length, grain width, projected area and fullness as input features. Collect data on the traits of all individual kernels in the whole ear of maize, and calculate and output the population-derived indicators of maize ear kernels, including the total number of kernels in the ear, the weight of kernels per ear, the number of rows in the ear, the number of kernels per row, and the uniformity of kernel traits.

10. A deep learning-based image recognition system for maize ear and kernel traits, the system being used to implement the deep learning-based image recognition method for maize ear and kernel traits as described in claim 1, comprising: The image acquisition and preprocessing module is used to acquire panoramic images of corn ears covering the entire circumference of the ear and the distribution of kernels. The panoramic images of corn ears are then subjected to background segmentation, illumination correction and cylindrical unfolding geometric regularization to generate regularized ear images and pixel physical size mapping parameters. The feature extraction and topology construction module is used to extract the salient features of kernel row and column boundaries from regular ear images based on the growth characteristics of orderly arrangement of corn kernel rows and columns, while detecting candidate kernel center points and constructing an initial grid topology structure. The semantic extraction module is used to extract multi-scale deep semantic feature maps of grains from regular ear images using a convolutional neural network. The characterization generation module is used to fuse the saliency features of grain row and column boundaries with multi-scale deep semantic features of grains to generate a grain instance segmentation mask, fuse the candidate points of grain center in the topology with multi-scale deep semantic feature maps of grains to complete the correction of grain center pixels and construct pixel relative coordinate characterization, and combine the pixel physical size mapping parameters to generate a multi-attribute integrated characterization map. The trait calculation and statistics module is used to calculate the single grain trait data based on the multi-attribute integrated characterization graph, and to estimate the population-derived indicators of maize ear grains based on all single grain trait data.

Images