Whole ear image processing method and device

Abstract

The present invention provides a method for processing a whole ear image, which includes a complete whole ear. The processing method includes: executing an extraction algorithm on the whole ear image to obtain a whole ear mask; executing an extraction algorithm on the whole ear mask to obtain a grain region mask; superimposing the grain region mask and the red channel image of the whole ear image to obtain a grain region red channel grayscale image; executing superpixel segmentation on the grain region red channel grayscale image to obtain a grain mask; and outputting grain statistical analysis results based on the grain mask.

Description

Technical Field

[0001] The present invention relates to the field of crop phenotype detection, and in particular to a method and device for processing whole ear images to perform grain number and grain shape statistics per ear. Background Art

[0002] As my country's leading staple food crop, rice production plays a crucial role in food security and the stable development of China's and the world's economies. Key factors influencing rice yield include grain number per panicle and grain shape. In current basic rice research, phenotypic data on grain number per panicle and grain shape are crucial indicators for identifying gene function and demonstrating biological significance. In rice breeding and large-scale rice field production, grain number per panicle and grain shape are essential prerequisites for estimating rice yield. Grain number per panicle refers to the total number of grains per panicle per active tiller. Grain shape, which includes grain length and width, is a crucial indicator for estimating thousand-grain weight.

[0003] The trait of grain number per panicle has traditionally been counted manually. While a few methods exist for analyzing grain number per panicle by scanning evenly laid rice panicles, scanning even neatly laid panicles is time-consuming and labor-intensive, resulting in low efficiency. Grain shape phenotyping requires threshing, spreading the grains on a scanner, and then analyzing the data using software. These processes involve significant manual labor, making the counting of these traits time-consuming, labor-intensive, and costly. Currently, no method exists for measuring grain number and shape per panicle directly from the entire panicle without threshing.

[0004] Therefore, there is an urgent need for a high-throughput method that can directly and simultaneously count the number of grains per ear and the grain shape before threshing. Summary of the Invention

[0005] The following is a brief summary of one or more aspects to provide a basic understanding of these aspects. This summary is not an exhaustive overview of all conceivable aspects and is neither intended to identify key or critical elements of all aspects nor to define the scope of any or all aspects. Its sole purpose is to present some concepts of one or more aspects in a simplified form as a prelude to the more detailed description that will be provided later.

[0006] According to one aspect of the present invention, a method for processing a whole ear image is provided, wherein the whole ear image includes a complete whole ear, and the processing method comprises:

[0007] performing an extraction algorithm on the whole ear image to obtain a whole ear mask;

[0008] performing an extraction algorithm on the whole ear mask to obtain an ear-kernel region mask;

[0009] Superimposing the ear-kernel region mask and the red channel image of the whole ear image to obtain a red channel grayscale image of the ear-kernel region;

[0010] Perform superpixel segmentation on the red channel grayscale image of the spikelet region to obtain a spikelet mask; and

[0011] The spike-grain statistical analysis results are output based on the spike-grain mask.

[0012] In one embodiment, performing an extraction algorithm on the whole ear image to obtain a whole ear mask includes:

[0013] The whole ear image is processed using a global automatic threshold algorithm to obtain a subject whole ear mask;

[0014] Using a local adaptive threshold algorithm to process the red channel grayscale image of the whole ear image to obtain a local whole ear mask; and

[0015] The main whole ear mask and the local whole ear mask are superimposed to obtain the whole ear mask.

[0016] In one embodiment, after executing the extraction algorithm on the whole ear image, the method further comprises:

[0017] Performing scaling adjustment on the whole ear image to calibrate a uniform resolution; and / or

[0018] A histogram matching algorithm is performed on the whole ear image to calibrate uniform exposure and contrast.

[0019] In one embodiment, performing an extraction algorithm on the whole ear mask to obtain a kernel region mask includes:

[0020] Performing an edge detection algorithm on the whole ear mask to extract a boundary mask of the whole ear mask;

[0021] Performing an XOR calculation on the boundary mask and the whole ear mask to obtain a preliminary ear-grain region mask;

[0022] Using a global threshold algorithm to process the a channel image of the LAB color space of the whole ear image to obtain a first ear branch stalk mask;

[0023] Extracting a second ear branch mask from the whole ear mask based on a two-dimensional skeleton algorithm; and

[0024] An exclusive OR operation is performed on the first spike mask, the second spike mask, and the preliminary kernel region mask to obtain the kernel region mask.

[0025] In one embodiment, extracting the second ear branch mask from the whole ear mask based on a two-dimensional skeleton algorithm includes:

[0026] Extracting an initial two-dimensional skeleton of the whole ear mask from the whole ear mask by an image thinning algorithm;

[0027] Analyze endpoints and branch nodes of the initial two-dimensional skeleton to obtain a refined two-dimensional skeleton; and perform a morphological dilation algorithm on the refined two-dimensional skeleton to obtain the second spike and stalk mask.

[0028] In one embodiment, the method further comprises:

[0029] Before performing the superpixel segmentation, the red channel grayscale image of the spikelet region is mapped with a pseudo color map to add pseudo color;

[0030] Then, the superpixel segmentation is performed on the mapped red channel grayscale image of the spikelet region.

[0031] In one embodiment, the superpixel segmentation is performed using a superpixel segmentation algorithm based on the Filsenzwaber significance map.

[0032] In one embodiment, in the Filsenzwaber significance map-based superpixel segmentation algorithm, the segmentation scale parameter is set equal to one two hundredth of the pixel area of ​​the grain region mask.

[0033] In one embodiment, the method further comprises:

[0034] The misdetected portion in the grain mask is segmented using a progressive watershed or random walk image segmentation algorithm to obtain a final grain mask.

[0035] In one embodiment, outputting the spike-to-grain statistical analysis results based on the spike-to-grain mask includes:

[0036] A database is constructed based on the kernel mask, and the database stores records about each kernel. The records of each kernel include the following multiple entries: boundary range, length, width, height, area, density, grayscale pixel group, and mask array of each kernel.

[0037] In one embodiment, the outputting of the spike-to-grain statistical analysis results based on the spike-to-grain mask further includes:

[0038] The mask array of each record and the whole ear image are superimposed to obtain the RGB image of each ear kernel, and the RGB image of each ear kernel is stored in the entry information of each ear kernel in the form of an image array.

[0039] In one embodiment, the outputting of the spike-to-grain statistical analysis results based on the spike-to-grain mask further includes:

[0040] An image thinning algorithm is executed on the mask array of each record to extract the two-dimensional skeleton of each kernel, and the two-dimensional skeleton of each kernel is stored in the entry information of each kernel in the form of an array.

[0041] In one embodiment, the outputting of the spike-to-grain statistical analysis results based on the spike-to-grain mask further includes:

[0042] The spike-grain mask is filtered based on the number of endpoints of the two-dimensional skeleton of each record, and the records with a two-dimensional skeleton endpoint number of 2 are retained.

[0043] According to another aspect of the present invention, there is also provided a device for processing an entire ear image, comprising:

[0044] Memory; and

[0045] A processor configured to execute the above method.

[0046] According to the present invention, a computer-readable medium is provided, on which computer-executable instructions are stored. When the computer-executable instructions are executed by a processor, the above method is implemented. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] The above features and advantages of the present invention will be better understood after reading the detailed description of the embodiments of the present disclosure in conjunction with the following drawings. In the drawings, the components are not necessarily drawn to scale, and components with similar related properties or characteristics may have the same or similar reference numerals.

[0048] Figure 1 A flow chart of a whole ear image processing method according to one aspect of the present invention is shown;

[0049] Figure 2a-2d An example of the collected spike image morphology and its calibration according to one aspect of the present invention is shown;

[0050] Figure 3 A flow chart of a method for extracting a whole ear mask according to one aspect of the present invention is shown;

[0051] Figure 4a-4e Schematic diagram of each stage in the whole ear mask extraction process according to one aspect of the present invention;

[0052] Figure 5 A flowchart of a method for extracting a grain region mask according to an embodiment of the present invention is shown;

[0053] Figure 6a-6d A schematic diagram showing various stages in the initial retention process of the grain region according to one aspect of the present invention is shown;

[0054] Figure 7a-7c Schematic diagram of each stage in the process of extracting spike and branch stalk masks and optimizing the retention of spike kernel regions according to one aspect of the present invention;

[0055] Figure 8a-8dSchematic diagrams of various stages of a process for removing main branches and stalks based on a whole-ear two-dimensional skeleton according to one aspect of the present invention are shown;

[0056] Figure 9a-9d Schematic diagram showing various stages in the process of detecting and labeling individual kernels using a superpixel algorithm according to one aspect of the present invention;

[0057] Figure 10a-Figure 10c A schematic diagram showing various stages of further dividing the kernels according to an aspect of the present invention; and

[0058] Figure 11a-Figure 11c An RGB image and a two-dimensional skeleton image of the screened kernels according to one aspect of the present invention are shown. DETAILED DESCRIPTION

[0059] The present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. Note that the various aspects described below with reference to the accompanying drawings and specific embodiments are merely exemplary and should not be construed as limiting the scope of protection of the present invention.

[0060] According to one aspect of the present invention, a method for processing images of whole ears is provided. It is only necessary to slightly separate the primary branches of the ears before taking pictures so that they are not stuck together, and there is no need to keep the ears in a very neat and uniform state. According to the scheme of the present invention, the number of all grains in the image and the length and width of all intact grains that are not blocked can be counted, and the average value of the grain shape can be automatically calculated. The scheme of the present invention can quickly read and analyze the number of grains per ear and the size of grain shape in multiple photos at a time with high throughput, which will greatly reduce the time-consuming and inefficient simple repetitive work of scientific researchers, improve the scientific research initiative of scientific researchers, and at the same time can count the number of grains in all ears of a single rice plant in a short time, ensuring more accurate data while saving a lot of time, greatly improving efficiency, and greatly reducing the related manual analysis costs.

[0061] Figure 1 FIG. 1 is a flow chart showing a method 100 for processing an entire ear image according to an aspect of the present invention. Figure 1 As shown, the method 100 may include the following steps.

[0062] At step 110 , an extraction algorithm is performed on the whole ear image to obtain a whole ear mask.

[0063] The term "mask" comes from basic image processing applications, meaning "a board covering something." If we don't want a certain area to appear in the final image, we can use a mask to cover it. In this case, the mask used represents white areas defined as retained, while black represents areas not to be retained. When the mask is superimposed on the image, only the image corresponding to the white areas of the mask is retained, while the image corresponding to the black areas is eliminated.

[0064] The whole ear image is an image of a whole rice ear. The different postures of the rice ear have a great influence on image recognition. Generally, the stalks of a single rice plant are manually spread out according to artificial shaping. Since the water content of the rice ear is low during maturity, the stalks will be deformed by force. The main stalk and the stalks are not completely separated, and the ear image is taken in a naturally unfolded state. When collecting images, the sample is placed on a black light-absorbing cloth and the image is captured using a digital camera or scanner, such as Figure 2a As shown. Figure 2a A rice plant B10 is shown in FIG.

[0065] In addition, this solution is also applicable to images collected using mobile phones, such as Figure 2b Compared to scanning and photographing, capturing images with a mobile phone faces several limitations, such as complex lighting conditions, low integrated camera resolution, and inaccurate focus. Therefore, analyzing images captured with a mobile phone requires not only the same process as scanning and photographing images, but also preprocessing.

[0066] In one embodiment, this preprocessing includes image calibration. Because images with excessively high resolutions consume excessive computation time, making high-throughput image analysis difficult, image preprocessing and calibration are necessary. To this end, a scaling algorithm can be used to automatically set the image height to, for example, 1024 pixels. This improves program processing efficiency without significantly reducing accuracy. After resolution calibration, images participating in trait analysis are uniformly sized to 1024 pixels, facilitating subsequent automated analysis.

[0067] Preferably, when calibrating an image captured by a mobile phone, it is necessary to calibrate not only the resolution but also the exposure of the image. By performing histogram matching with the reference image (Figure 2c), the image after histogram matching is obtained ( Figure 2d ), thereby reducing the impact of overexposure on image quality and reducing the difficulty of subsequent processing.

[0068] Figure 3 FIG. 3 is a flow chart of a method 300 for extracting a whole ear mask according to an embodiment. Figure 3 As shown, the method 300 may include the following steps.

[0069] In step 310 , a global automatic threshold algorithm is used to process the whole ear image to obtain a subject whole ear mask.

[0070] Threshold segmentation is a region-based image segmentation technology. It is the most commonly used and basic image segmentation method. It can divide a set of pixels according to grayscale. The resulting subsets correspond to regions in the real scene, and each region has consistent properties. Therefore, it is particularly suitable for images where the target and background occupy different grayscale ranges. When selecting the threshold, it can be selected based on the properties of the image pixel itself (i.e., global threshold), or based on the properties of the local area formed by the pixel and the neighborhood of the point (i.e., local threshold). It is also possible to combine the properties of the image pixel itself or the properties of the local area of ​​the image with the spatial coordinates of the pixel point when selecting the threshold, so that the resulting threshold is related to the coordinates of the pixel point (i.e., global dynamic threshold or local adaptive threshold).

[0071] like Figure 2a-2b As shown in the figure, the foreground and background of the image have obvious contrast, so the global dynamic threshold method can be used to easily extract the foreground mask of the image. The foreground mask extracted using the dynamic global automatic threshold algorithm is shown in the figure. Figure 4a Preferably, the size (such as length, width, area, etc.) of each connected domain (i.e., the white block in the figure) of the foreground mask can be screened, and blocks with too small area and too long and thin shape (i.e., small aspect ratio) can be removed to obtain regions of interest (ROI, such as Figure 4b As shown), that is, the main ear mask.

[0072] Step 320 : Using a local adaptive threshold algorithm, the red channel grayscale image of the whole ear image is processed to obtain a local whole ear mask.

[0073] Since the kernels are golden yellow, the R (red) channel of the calibrated image (the scanned image is the calibrated resolution when scanning, and the mobile phone image is the calibrated resolution and exposure when taking the image) is segmented using a local adaptive threshold to obtain a local whole-ear mask, such as Figure 4c As shown in Figure 2, the local adaptive threshold algorithm can more finely characterize the whole ear mask based on the detailed features in the local area of ​​the image.

[0074] Step 330: superimpose the main whole ear mask and the local whole ear mask to obtain the whole ear mask.

[0075] Local threshold segmentation cannot directly classify large black backgrounds like global threshold segmentation, and can only characterize the difference between the whole ear and its local background. Therefore, superimposing the main whole ear mask and the local whole ear mask can only retain the mask information within the region of interest, thereby obtaining the whole ear mask of rice, such as Figure 4d In practice, the rice panicle mask can be saved in the form of a binary array.

[0076] By superimposing the whole ear mask on the calibrated image, the background can be removed and the visible light (RGB) image of the whole rice ear can be extracted, as shown in Figure 4e shown.

[0077] Returning to method 100 , at step 120 , an extraction algorithm is performed on the whole ear mask to obtain a kernel region mask. Figure 5 FIG. 5 is a flow chart showing a method 500 for extracting a grain region mask according to an embodiment of the present invention. Figure 5 As shown, the method 500 may include the following steps.

[0078] At step 510 , an edge detection algorithm is performed on the whole ear mask to extract a boundary mask of the whole ear mask.

[0079] In one example, the Sobel edge detection algorithm can be used to extract the boundary of the whole ear mask, such as Figure 6a shown.

[0080] In step 520 , an XOR operation is performed on the boundary mask and the whole ear mask to obtain a preliminary ear-kernel region mask.

[0081] The extracted boundary is XORed with the whole ear mask to cut off the boundary, thereby disconnecting the whole ear's cob and stalk, as shown in Figure 6b shown.

[0082] Preferably, the size of each connected domain (white block) can be adjusted. Figure 6b After the blocks are disconnected from the branches, the blocks are screened and the connected domains with too small an area, for example, those smaller than a preset threshold, are removed to obtain the mask of the disconnected ear area and branches, such as Figure 6c shown.

[0083] Further preferably, the size and tilt angle of each connected domain can be adjusted. Figure 6c The blocks in the image are screened to remove those with too small an area, such as those smaller than a preset threshold, and those with an inclination angle too close to 90°, such as those exceeding a preset threshold. The remaining parts are used as preliminary grain area masks, such as Figure 6d As shown, it can be stored in array form.

[0084] In step 530 , a global threshold algorithm is used to process the a-channel image of the LAB color space of the whole ear image to obtain a first ear stalk mask.

[0085] Compared with the grain area, the color of the spikelets tends to be green. Therefore, according to one aspect of the present invention, the features of the spikelets of the whole ear can be extracted based on the color. First, the whole ear image with calibrated resolution and exposure can be converted to Lab color space, and the a channel image can be extracted, such as Figure 7aAs shown in , the a-channel image reflects the nonlinear spectral change from green to red. Then, the global threshold segmentation method can be used to extract the spikelet mask from the a-channel image, which can be called the first spikelet mask, as shown in Figure 7b The extracted spikelet mask can be used to take XOR with the initial spikelet region mask retained above to further remove the spikelets in the spikelet region, and obtain Figure 7c The spike-grain region mask shown was used for further analysis.

[0086] In step 540 , a second ear branch and stalk mask is extracted from the whole ear mask based on a two-dimensional skeleton algorithm.

[0087] Furthermore, in images of certain rice varieties, the main stems and stalks of the entire ear are relatively wide, with insufficient separation between the branches during capture. Furthermore, the main stems and stalks are similar in color to the grain area. This type of imagery can lead to accuracy deviations in the edge and color analysis algorithms mentioned above.

[0088] In view of this, according to one aspect of the present invention, a main stem and branch removal algorithm based on a two-dimensional skeleton is designed when extracting the grain area.

[0089] In step 541 , an initial two-dimensional skeleton of the whole ear mask is extracted from the whole ear mask by an image thinning algorithm.

[0090] Figure 8a A whole ear mask is shown. By using the image thinning algorithm, the extracted two-dimensional skeleton can be Figure 8b shown.

[0091] In step 542, the endpoints and branch nodes of the initial two-dimensional skeleton are analyzed to obtain a refined two-dimensional skeleton.

[0092] The two-dimensional skeleton obtained above is only an initial two-dimensional skeleton. It is also necessary to analyze the endpoints and branch nodes of the main trunk and branches. Ideally, the endpoints and branch nodes closest to the bottom of the image should be used as the main trunk and branch endpoints and branch nodes; however, in actual analysis, due to the presence of noise (burrs) in the skeleton, it may cause misjudgment of the main trunk, branch, and branch nodes, such as Figure 8b Point A in the image. Considering that the skeleton length between the endpoints and branch nodes of the burr is particularly short, the algorithm considers not only the distance between the branch node and the bottom of the image, but also the skeleton length between the candidate node and the nearest skeleton endpoint when analyzing the main branch node, thereby eliminating the influence of noise on the two-dimensional skeleton. For example, Figure 8bIn the example, the 2D skeleton length between the candidate trunk-branch node (point A) and its nearest skeleton endpoint (point B) is too short, so this candidate node is excluded as a false positive. After eliminating the noise points in the candidate nodes, the most reliable trunk-branch endpoint and branch node locations (i.e., points C and D in 8C) are obtained. Subsequently, based on the endpoints and branch nodes, the skeleton between these two points is extracted, which is the 2D skeleton of the trunk-branch, or refined 2D skeleton.

[0093] In step 543, a morphological dilation algorithm is performed on the refined two-dimensional skeleton to obtain a second spike mask.

[0094] After morphological expansion of the above-mentioned refined two-dimensional skeleton, a second spikelet mask can be obtained. This second spikelet mask can be used to perform logical XOR with the preliminary grain region mask to remove the main branches that are difficult to remove based on edge and color analysis algorithms, such as Figure 8d shown

[0095] In step 550 , an XOR operation is performed on the first stalk mask, the second stalk mask, and the preliminary kernel region mask to obtain a kernel region mask.

[0096] By performing an XOR calculation on the first stalk mask, the second stalk mask and the preliminary grain region mask, the stalk portion in the preliminary grain region mask can be removed to obtain a pure grain region mask.

[0097] Returning to method 100, in step 130, the red channel image of the whole ear image and the ear region mask are superimposed to obtain a red channel grayscale image of the ear region, such as Figure 9a shown.

[0098] In one example, a pseudo color image (such as jet) can be used to map the grayscale image of the grain area, thereby enhancing the contrast and distinction between the grains. Figure 9b shown.

[0099] In step 140 , superpixel segmentation is performed on the red channel grayscale image of the grain region to obtain a grain mask.

[0100] In one example, a superpixel segmentation algorithm based on Felzenszwalb's efficient graph can be applied to the mapped false-color image of the kernel region. The detected kernel mask, a set of masks representing each detected kernel, is extracted. By counting the masks representing each detected kernel, the number of kernels in the image can be estimated.

[0101] Compared with other commonly used superpixel segmentation algorithms, the Filsenzwaber algorithm can make it more sensitive to smaller detail targets by reducing the segmentation scale (min_size) parameter, and is therefore more suitable for the detection and segmentation of grains. In this algorithm, in order to balance time complexity and segmentation accuracy, the segmentation scale parameter value is dynamically set to two hundredths of the pixel area of ​​the grain region mask.

[0102] In order to intuitively present the detection results of the superpixel segmentation algorithm, Figure 9c Shows individual grain masks from superpixel detection results differentiated by color. Figure 9d The superpixel detection results of the kernels are superimposed on the preprocessed whole-ear image in the form of a bounding box and numbered to more intuitively display the detection results of the superpixel algorithm.

[0103] From the above results, it can be seen that there are cases where multiple kernels are identified as one. Therefore, if more accurate segmentation is required, these misdetected kernels must be screened out and further processed. Figure 10a In one example, the grains can be segmented using image segmentation algorithms such as progressive watershed or random walk. The segmentation results after processing are shown in FIG. Figure 10b As shown. After the segmentation is completed, the detection results of the further segmentation ( Figure 10b ) is merged into the detection results of the superpixel algorithm to obtain more accurate detection and counting results, such as Figure 10c shown.

[0104] In step 150 , the spike-to-grain statistical analysis results are output based on the spike-to-grain mask.

[0105] As mentioned above, the detected kernel mask is extracted, that is, a set of masks of each detected kernel. By counting the number of masks of the detected kernels, the number of kernels in the image can be estimated.

[0106] As part of the statistical analysis of the grains, a database can be constructed based on the results of the aforementioned grain detection and counting, and the detection results can be stored in the database. The content mainly includes: the boundary range of each grain (vertex coordinates of the bounding box), length (major_axis_length), width (minor_axis_length), area (area), density (solidity = number of pixels in the area ÷ number of pixels in the convex hull), grayscale image array (intensity_image), mask array (mask) and other data, as shown in Table 1 below.

[0107]

[0108] In order to reduce the amount of calculation and possible errors, the rows (records) corresponding to the areas with too small connected domain area and density can be deleted from the database. The results are shown in Table 2.

[0109]

[0110] The mask arrays (i.e., the data in the mask field (column)) within each record in the database were then superimposed on the preprocessed whole-ear image to extract the RGB image of each kernel. This image was then stored as an array in the data cube, as shown in the "image_color" field (column) in Table 3 below. Furthermore, an image thinning algorithm was applied to the mask data for each record to derive the 2D skeleton of each kernel, which was also stored as an array in the data cube, as shown in the "skeleton" field (column) in Table 3 below.

[0111]

[0112] After the above pre-screening and extraction operations, the retained RGB images of the spikelets and the corresponding two-dimensional skeletons of each spikelet are as follows: Figure 11a 、 11b shown.

[0113] Complete kernels often have regular shapes and edges, features that can be reflected in a 2D skeleton. Therefore, records (rows) in the dataset can be further filtered based on the properties of the kernel's 2D skeleton. Based on the kernel's 2D skeleton stored in the dataset (the "skeleton" field (column)), the number of endpoints for each skeleton is calculated and stored in the "end points num" field (column) in the dataset, as shown in Table 4 below.

[0114]

[0115] The records with the two-dimensional skeleton endpoint number of 2 are retained to obtain the final complete kernel screening results, and the measurement data of the kernels per ear are saved in the database. Figure 11c Images of the retained kernels are shown, and Table 5 below shows the results of the retained kernel analysis.

[0116] Although the above methods are illustrated and described as a series of acts for simplicity of explanation, it is to be understood and appreciated that these methods are not limited by the order of the acts, as some acts may occur in a different order and / or concurrently with other acts from those illustrated and described herein or not illustrated and described herein but understandable to those skilled in the art according to one or more embodiments.

[0117] The method for counting grain numbers and grain shapes per panicle according to the present invention can analyze images collected in two ways. The first method involves arranging rice panicles neatly, ensuring that the stalks do not overlap, and then photographing or scanning them using a mobile phone, camera, or scanner. This method produces neat and beautiful photos, but it takes time to arrange the rice panicles and separate the stalks. While this method accurately counts grain numbers per panicle, it cannot calculate the length and width phenotype of the grains.

[0118] The second image collection method doesn't require neatly arranging the rice ears; it simply requires that the individual stalks don't overlap. This method not only allows for an accurate count of the number of grains per ear, but also simultaneously calculates data such as grain length and width, achieving two goals at once.

[0119] In summary, this new software method for counting rice grain number and shape per panicle is convenient and fast, significantly simplifying and accelerating the rice phenotyping process, saving both labor and time. In practical applications, this method can be used for phenotyping genes associated with important agronomic traits and for efficiently estimating individual rice yield during rice breeding. This provides researchers with a fast and efficient means of phenotyping, bringing efficiency and accuracy to a new level.

[0120] According to one aspect of the present invention, there is also provided an apparatus for processing whole ear images, comprising a memory and a processor, wherein the processor can be used to execute the above software method.

[0121] Those skilled in the art will appreciate that information, signals, and data may be represented using any of a variety of different technologies and techniques. For example, data, instructions, commands, information, signals, bits, symbols, and chips cited throughout the foregoing description may be represented by voltages, currents, electromagnetic waves, magnetic fields or particles, optical fields or particles, or any combination thereof.

[0122] Those skilled in the art will further appreciate that the various illustrative logic blocks, modules, circuits, and algorithmic steps described in conjunction with the embodiments disclosed herein can be implemented as electronic hardware, computer software, or a combination of the two. To clearly illustrate this interchangeability of hardware and software, various illustrative components, blocks, modules, circuits, and steps are generally described above in terms of their functionality. Whether such functionality is implemented as hardware or software depends on the specific application and the design constraints imposed on the overall system. A skilled person may implement the described functionality in different ways for each specific application, but such implementation decisions should not be interpreted as resulting in a departure from the scope of the present invention.

[0123] The various illustrative logic modules and circuits described in conjunction with the embodiments disclosed herein may be implemented or executed using a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein. A general-purpose processor may be a microprocessor, but in the alternative, the processor may be any conventional processor, controller, microcontroller, or state machine. A processor may also be implemented as a combination of computing devices, such as a combination of a DSP and a microprocessor, a plurality of microprocessors, one or more microprocessors in conjunction with a DSP core, or any other such configuration.

[0124] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein may be embodied directly in hardware, in a software module executed by a processor, or in a combination of the two. The software module may reside in RAM memory, flash memory, ROM memory, EPROM memory, EEPROM memory, registers, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor so that the processor can read and write information from / to the storage medium. In an alternative, the storage medium may be integrated into the processor. The processor and storage medium may reside in an ASIC. The ASIC may reside in a user terminal. In an alternative, the processor and storage medium may reside in a user terminal as discrete components.

[0125] In one or more exemplary embodiments, the functions described may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software as a computer program product, the functions may be stored on or transmitted via a computer-readable medium as one or more instructions or codes. Computer-readable media include both computer storage media and communication media, including any media that facilitates the transfer of a computer program from one location to another. A storage medium may be any available medium that can be accessed by a computer. By way of example and not limitation, such computer-readable media may include RAM, ROM, EEPROM, CD-ROM or other optical disk storage, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a computer. Any connection is also properly referred to as a computer-readable medium. For example, if the software is transmitted from a website, server, or other remote source using a coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwaves, then the coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwaves are included in the definition of medium. As used herein, disk and disc include compact disc (CD), laser disc, optical disc, digital versatile disc (DVD), floppy disk and Blu-ray disc, where disks typically reproduce data magnetically, while discs reproduce data optically with lasers. Combinations of the above should also be included within the scope of computer-readable media.

[0126] The previous description of the disclosure is provided to enable any person skilled in the art to make or use the disclosure. Various modifications to the disclosure will be apparent to those skilled in the art, and the general principles defined herein may be applied to other variations without departing from the spirit or scope of the disclosure. Thus, the disclosure is not intended to be limited to the examples and designs described herein but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A method for processing a whole ear image, wherein the whole ear image includes a complete whole ear, the processing method comprising: executing an extraction algorithm on the whole ear image to obtain a whole ear mask; executing an extraction algorithm on the whole ear mask to obtain an ear-kernel region mask; Overlaying the ear-kernel region mask and the red channel image of the whole ear image to obtain a red channel grayscale image of the ear-kernel region; performing superpixel segmentation on the red channel grayscale image of the spikelet region to obtain a spikelet mask; as well as Output the spike-grain statistical analysis results based on the spike-grain mask, The step of performing an extraction algorithm on the whole ear mask to obtain an ear-kernel region mask comprises: executing an edge detection algorithm on the whole ear mask to extract a boundary mask of the whole ear mask; Performing an XOR calculation on the boundary mask and the whole ear mask to obtain a preliminary ear-grain region mask; Using a global threshold algorithm to process the a channel image of the LAB color space of the whole ear image to obtain a first ear branch mask; Extracting a second ear branch mask from the whole ear mask based on a two-dimensional skeleton algorithm; and An exclusive OR operation is performed on the first spikelet mask, the second spikelet mask, and the preliminary grain region mask to obtain the grain region mask.

2. The processing method according to claim 1, characterized in that The performing of the extraction algorithm on the whole ear image to obtain the whole ear mask comprises: Processing the whole ear image using a global automatic threshold algorithm to obtain a subject whole ear mask; Using a local adaptive threshold algorithm to process the red channel grayscale image of the whole ear image to obtain a local whole ear mask; and The main whole ear mask and the local whole ear mask are superimposed to obtain the whole ear mask.

3. The processing method according to claim 2, characterized in that Executing the extraction algorithm on the whole ear image also includes: Performing scaling adjustment on the whole ear image to calibrate a uniform resolution; and / or A histogram matching algorithm is performed on the whole ear image to calibrate uniform exposure and contrast.

4. The processing method according to claim 1, wherein The step of extracting the second ear branch mask from the whole ear mask based on a two-dimensional skeleton algorithm includes: Extracting an initial two-dimensional skeleton of the whole ear mask from the whole ear mask by an image thinning algorithm; Analyzing endpoints and branch nodes of the initial two-dimensional skeleton to obtain a refined two-dimensional skeleton; and A morphological dilation algorithm is performed on the refined two-dimensional skeleton to obtain the second spike and branch mask.

5. The processing method according to claim 1, wherein Also includes: Before performing the superpixel segmentation, mapping the red channel grayscale image of the spikelet region with a pseudo color map to add pseudo color; Then, the superpixel segmentation is performed on the mapped red channel grayscale image of the spikelet region.

6. The processing method according to claim 1, wherein The superpixel segmentation is performed using a superpixel segmentation algorithm based on the Filszentwaber significance map.

7. The processing method according to claim 6, characterized in that In the Filszentwaber significance map-based superpixel segmentation algorithm, the segmentation scale parameter is set equal to one two hundredth of the pixel area of ​​the grain region mask.

8. The processing method according to claim 1, wherein Also includes: A progressive watershed or random walk image segmentation algorithm is used to perform segmentation processing on the misdetected portion in the grain mask to obtain a final grain mask.

9. The processing method according to claim 1, wherein: Outputting the spike-grain statistical analysis results based on the spike-grain mask includes: A database is constructed based on the grain mask, and the database stores records about each grain. The record of each grain includes the following multiple items: boundary range, length, width, height, area, density, grayscale pixel group, and mask array of each grain.

10. The processing method according to claim 9, characterized in that The outputting of the spike-grain statistical analysis results based on the spike-grain mask further comprises: The mask array of each record and the whole ear image are superimposed to obtain the RGB image of each ear kernel, and the RGB image of each ear kernel is stored in the entry information of each ear kernel in the form of an image array.

11. The processing method according to claim 9, characterized in that The outputting of the spike-grain statistical analysis results based on the spike-grain mask further comprises: An image thinning algorithm is executed on the mask array of each record to extract the two-dimensional skeleton of each kernel, and the two-dimensional skeleton of each kernel is stored in the entry information of each kernel in the form of an array.

12. The processing method according to claim 11, characterized in that The outputting of the spike-grain statistical analysis results based on the spike-grain mask further comprises: The spike-grain mask is screened based on the number of endpoints of the two-dimensional skeleton of each record, and the records with two endpoints of the two-dimensional skeleton being 2 are retained.

13. A device for processing whole ear images, comprising: Memory; as well as A processor configured to process the method according to any one of claims 1 to 12.

14. A computer-readable medium having computer-executable instructions stored thereon, wherein the computer-executable instructions, when executed by a processor, implement the method according to any one of claims 1 to 12.

Images