Pheno 360 systems and methods for quantifying features on a plant part

Abstract

A system and methods are provided for analyzing an object. The method includes collecting image data of the object, where the image data includes a panoramic image capturing at least one entire 360-degree circumferential view of the object. The method also includes cropping the panoramic image to create a prepared image containing a single 360-degree view of the object, processing the prepared image to produce a processed image, and quantifying features of the object via the processed image.

Description

PHENO 360 SYSTEMS AND METHODS FOR QUANTIFYING FEATURES ON A PLANT PARTRELATED APPLICATIONS

[0001] This application claims priority under 35 U.S.C. § 119 to United States Provisional Patent Application No. 63 / 518,861 filed on August 10. 2023, the entire contents of which is incorporated herein by reference.STATEMENT REGARDING FEDERALLY SPONSORED RESEARCH

[0002] This invention was made with government support under grant number IOS- 1546792 awarded by the National Science Foundation. The government has certain rights in this invention.BACKGROUND

[0003] The rapidly advancing plant breeding field calls for high-throughput methods and technologies for phenotyping. Com is the number one ranking crop for production with 93 million acres grown in the United States in 2021, where tremendous breeding efforts have been made to improve com yield, drought tolerance, and nutritional values. Kernel traits, as components of yield, have been traditionally evaluated manually, which are labor-intensive, time-consuming, low efficient, and can be error prone.

[0004] There are several existing methods that phenotype com kernel traits in a high- throughput manner. Particularly, panoramic photography of com ears is useful to obtain information from the whole ear and avoids biases arising from a partial view. However, available methods to capture panoramic images of com ears often requires sophisticated setups and multiple steps of data transformation and / or algorithms to process and analyze images.

[0005] What is needed is an imaging system that is affordable and does not require much programming or statistical expertise. The availability and accessibility of such technology7globally across com grown environments at developed and developing countries would be beneficial.SUMMARY

[0006] Some embodiments of the invention provide a method for analyzing an object. The method includes collecting image data of the object, where the image data includes a panoramicimage capturing at least one entire 360-degree circumferential view of the object. The method also includes cropping the panoramic image to create a prepared image containing a single 360- degree view of the object, processing the prepared image to produce a processed image, and quantifying features of the object via the processed image.

[0007] In some embodiments, collecting the panoramic image of the object includes placing the object on a base unit configured to rotate at least 360 degrees and collecting the panoramic image with a stationary image collection device while rotating the base unit. In some embodiments, the base unit can be a turntable. Placing the object on the base unit can include securing the object to the base unit via a screw or placing the object in a clear container and setting the container on the base unit. Additionally, collecting the panoramic image of the object can include taking the panoramic image with a camera of a smart device.

[0008] In some embodiments, the object is an ear of com, and quantifying features of the object includes quantifying kernels. Quantifying kernels can include quantifying total kernels, quantifying kernels of a certain color, and / or quantifying kernels of a certain texture.

[0009] Some embodiments provide a system for analyzing features on a plant part imaged by an image collection device. The system includes a base unit configured to rotate the plant part while the plant part is imaged by the image collection device. The base unit includes a base, a top rotatable relative to the base, and a holder configured to receive the plant part, where the holder is positioned on the top so that the plant part rotates with the top.

[0010] In some embodiments, the holder includes a clear container that is integral with the top or otherwise coupled to the top. In some embodiments, the holder is a screw that extends upward from the top to receive the plant part. Additionally, in some embodiments, the base unit further comprises a rotating mechanism that rotates the top relative to the base and a controller configured to activate the rotating mechanism.

[0011] Some embodiments provide a method for quantifying kernels on a com ear. The method comprises collecting image data of the com ear, where the image data includes a panoramic image capturing at least one entire 360-degree circumferential view of the com ear, and cropping the panoramic image to create a prepared image containing a single 360-degree view of the com ear. The method also includes performing a color threshold function on the cropped image to identify pixels within a predetermined range within the prepared image and create a color threshold image, converting the color threshold image to a black and white binaryimage, and analyzing particles of a predetermined pixel range within the binary image to quantify kernels within the binary image.

[0012] In some embodiments, the method further includes segmenting the cropped image into separate classes using trained segmentation software prior to performing the color threshold function. The color threshold function can then be performed for each segmented image. In some embodiments, the classes include separate colors or separate textures.DESCRIPTION OF THE DRAWINGS

[0013] FIG. 1 A illustrates an example system for image analysis of a plant product, including panoramic image capturing as a com ear revolves and a com ear placed on base unit.

[0014] FIG. IB illustrates another view of the example system of FIG. 1A.

[0015] FIG. 2 illustrates a schematic diagram of an example system according to some implementations.

[0016] FIG. 3 illustrates a flowchart of an example method for image processing homogeneously patterned and mixed patterned com ears.

[0017] FIG. 4 illustrates an example of a fully prepared panoramic image with one full revolution using a landmark, including a com ear (FIG. 4A), one full revolution (FIG. 4C), and the raw panoramic image with one revolution marked (FIG. 4B).

[0018] FIG. 5 shows another example of a fully prepared panoramic image with one full revolution using 16 counted rows, including a com ear (FIG. 5 A), one full revolution (FIG. 5C), and the raw panoramic image with one revolution marked (FIG. 5B).

[0019] FIG. 6 illustrates an example graphical user interface for selecting kernels using a Color Threshold function, including an image after being Color Threshold-ed (left) and an example showing the defaults of color adjustment (right), including hue, saturation, and brightness.

[0020] FIG. 7 illustrates an example of total kernel count of a homogeneous patterned com ear. including a manual count of total number of kernels (left) and an automated count of total kernels from a processed black and white image (right).

[0021] FIG. 8 illustrates a first example mixed colored com ear sample (upper panel) used to train a Food Color Inspector, including an original image (left), segmented image (middle), and adjusted binary image (right) showing total kernels (black pixels); and a second example mixed colored com ear sample (lower panel) where the training was executed, including an original image (left), segmented image (middle), and adjusted binary image (right) showing total kernels (white pixels).

[0022] FIG. 9 illustrates an example graphical user interface of a Food Color Inspector window showing how to train a classifier using examples of the gaps between kernels as the background class, purple kernels as class 2, white kernels as class 3, and yellow kernels as class 4 with the original image and segmented image shown on the right.

[0023] FIG. 10 illustrates an example segmented split channel adjusted binary images for the classes purple (left), white (middle), and yellow (right) kernels of the first mixed colored com sample shown in FIG. 8.

[0024] FIG. 11 illustrates an example graphical user interface of purple kernel count using an Analyze Particle function, including results of Analyze Particle function with particles numbered (left) and a summary of counting results showing 170 kernels (right).

[0025] FIG. 12 illustrates example images of dried sticky coms (left), a sweet com (middle), and a starch com (right).

[0026] FIG. 13 illustrates an example image of a dried hybrid com ear resulting from a sweet x sticky cross whole view (left), a panoramic prepared image (middle), and a side view (right), where starch com is circled in green, sticky com circled in magenta, and sweet com is circled in white.

[0027] FIG. 14 illustrates a first mixed textured yellow com sample of different texture used to train a Food Color Inspector (upper panel), including an original image (left), segmented image (middle), and adjusted binary image (right) showing total kernels; and a second mixed textured yellow com ear sample (lower panel) where the training w as executed, including an original image (left), segmented image (middle), and adjusted binary image (right) showing total kernels.

[0028] FIG. 15 illustrates example segmented split channel adjusted binary images for the classes sticky (left), starch (middle), and sweet (right) kernels of the first mixed textured com sample shown in FIG. 14.

[0029] FIG. 16 shows an example panoramic image of a tree bark pattern.

[0030] FIG. 17 shows a schematic view of another example system for image analysis of a plant product, including panoramic image capturing of a live tree trunk.DETAILED DESCRIPTION

[0031] Before any embodiments of the invention are explained in detail, it is to be understood that the invention is not limited in its application to the details of construction and the arrangement of components set forth in the following description or illustrated in the following drawings. The invention is capable of other embodiments and of being practiced or of being earned out in various ways. Also, it is to be understood that the phraseology and terminology used herein is for the purpose of description and should not be regarded as limiting. The use of “including,” “comprising,” or “having” and variations thereof herein is meant to encompass the items listed thereafter and equivalents thereof as well as additional items. Unless specified or limited otherwise, the terms “mounted,” “connected,” “supported,” and “coupled” and variations thereof are used broadly and encompass both direct and indirect mountings, connections, supports, and couplings. Further, “connected” and “coupled” are not restricted to physical or mechanical connections or couplings.

[0032] The following discussion is presented to enable a person skilled in the art to make and use embodiments of the invention. Various modifications to the illustrated embodiments will be readily apparent to those skilled in the art. and the generic principles herein can be applied to other embodiments and applications without departing from embodiments of the invention. Thus, embodiments of the invention are not intended to be limited to embodiments shown, but are to be accorded the widest scope consistent with the principles and features disclosed herein. The following detailed description is to be read with reference to the figures, in which like elements in different figures have like reference numerals. The figures, which are not necessarily to scale, depict selected embodiments and are not intended to limit the scope of embodiments of the invention. Skilled artisans will recognize the examples provided herein have many useful alternatives and fall within the scope of embodiments of the invention.

[0033] Some embodiments provide an image collection system and method for image analysis of a plant product. Generally, the system includes an image collection device and a rotating base unit that receives a plant product. A panoramic image is taken while the plant product rotates at least one rotation (e.g., 360 degrees) on the rotating base unit. The image is then prepared to include exactly one full rotation of the plant product for further imageprocessing and analysis of surface paterns on the plant product. For example, some embodiments provide a system and method for counting com kernels, providing a high- throughput method to phenotype com kernel traits to estimate yield and to study their genetic inheritance. However, the system and methods of can be used for cylindrical object surface patern analysis of any type of plant product.

[0034] The system can provide an improvement over existing systems because it has a simple setup and is portable, easily accessible, and affordable. As illustrated in the corresponding figures and described in the examples below, the present methods are accurate and efficient for counting total kernels of a homogeneously paterned com ear and can be used to count segmented kernels of a com ear with mixed paterned kernels, including mixed colors and mixed textures. That is. the present systems and methods for counting can maintain a relatively high accuracy compared to conventional manual counting methods, with the potential to save vast amounts of time for com breeders and researchers who may or may not have programming and statistical expertise. For example, the methods of some embodiments can be semi-automatic and can further allow for batch-processing numerous images with custom macro scripts and a trained Food Color Inspector classifier. With different samples, depending on kernel density, the classifier and the macro scripts can be manually adjusted accordingly. The system of some embodiments can also be portable, which enables for panoramic images to be taken and prepared in real-time in the field. As such, the systems and methods described herein can be readily incorporated into breeding programs and / or genetic classrooms.

[0035] By way of example, FIGS. 1A and IB illustrate a system 10 according to some embodiments. The system 10 can include an image collection device 12 and a rotating base unit 14. The image collection device 12 can be a smart device capable of collecting panoramic images, such as a smart phone, a tablet, a laptop computer, a webcam, etc. The rotating base unit 14 can be configured to hold a plant object, such as a com ear 16. and rotate about 360 degrees.

[0036] In some embodiments, the base unit 14 can be a turntable. For example, the base unit 14 can include a static base 18 and a rotating top 20. The rotating top 20 can be configured to rotate relative to the base 18 at least about 360 degrees. In some embodiments, the base unit 14 can be manually rotatable such that a user can turn the rotating top 20 relative to the base 18. Additionally or alternatively, the base unit 14 can be automatically rotatable.

[0037] More specifically, as shown in the example schematic diagram of FIG. 2, the base unit 14 can include an internal controller 24 that controls a rotation mechanism 26, such as a motor, for the top 20 to automatically rotate relative to the static base 18. For example, a user can press a button 28 or other user input mechanism on the base unit 14 that causes the top 20 to rotate. As another example, the internal controller 24 can include a communication interface 30 (e.g., a wireless or wired communication port) that permits the controller 24 to communicate with the image collection device 12 or a separate computer system, where the image collection device 12 or computer system can control the internal controller 24 to activate the rotation mechanism 26 and rotate the top 20 relative to the base 18. The base unit 14 can be powered by an internal battery 32 or an external power source (e.g., by including an a / c adapter to be plugged into an outlet).

[0038] Referring to FIGS. 1A-2. the base unit 14 can further include a holder 22 configured to hold the plant part on the top 20 so that the plant part rotates with the top 20. Some examples of a holder 22 include, but are not limited to, a container or cup (as shown in FIG. 1), or a screw or pin, among others. With respect to a container-type holder 22, the holder 22 can receive the com ear 16 and can be clear (e.g., made of glass, plastic, acrylic, etc.) to avoid blocking any kernels during imaging. The holder 22 can be placed on the top 20. can be secured to the top 20 with a screw, nail, glue, Velcro, tape, or other fastening device, or can be integral with the top 20. With respect to a screw-ty pe holder 22, a screw or nail can extend up from the top 20 and can extend into the com ear 16 to maintain the com ear 16 in place on the top 20.

[0039] Generally, the holder 22 can be sized to receive a respective plant part (e.g., the com ear 16) and the top 20 can be sized to support the holder 22 and plant part. For example, the top 20 illustrated in FIG. 1 can be about 20 centimeters in diameter. This relatively small area and battery-pow ered equipment of some embodiments can allow for the system 10 to be easily portable for off-site locations. However, the size and / or shape of the holder 22 and / or the top 20 can vary based on the specific plant part to analyze (e.g.. com. tree trunk, cylindrical fruit, melon, etc.).

[0040] As noted above, the image collection device 12 can be a smart device capable of collecting panoramic images, such as a smart phone, a tablet, a laptop computer, a webcam, etc. In some embodiments, the image collection device 12 can be held by a user. In other embodiments, the image collection device 12 can rest on a table top alone or via a support (such as, for example, a tripod). Once the plant part is securely placed on the base unit 14, theimage collection device 12 can take a panoramic image while the top 20 rotates at least one full revolution. Accordingly, the image collection device 12 remains static while the base unit 14 rotates during image data collection.

[0041] Once image data is collected by the image collection device 12, the image data can be processed on the image collection device 12 and / or can be transmitted to a separate computing device for processing. For example, the image collection device 12, and / or a separate computing device, can include one or more programs stored in memory (e.g., a non- transitory computer-readable medium) and carried out by a processor to process the collected image data according to the methods described below. In some embodiments, image processing can be automatic or semi-automatic once image data is collected.

[0042] FIG. 3 illustrates a method 40 for collecting and analyzing plant image data according to some embodiments. While FIG. 3 is illustrated with a number of steps in a specific order, some applications may include more or fewer steps, different combinations of steps, or steps in a different order than what is shown. Additionally, one or more of the steps of the method 40 can be carried out by the image collection device 12 and / or a separate computing device (such as, but not limited to, a smart phone, a tablet, a laptop, a desktop computer, etc.). That is, the method 40 can be instructions stored on non-transitory computer- readable media to be carried out by a processor of a computing device, such as the image collection device 12.

[0043] As shown in FIG. 3. the method 40 can include image collection (step 42) and image cropping (step 44), wherein the cropped image may be a homogenously patterned plant part (step 46) or a mixed patterned plant part (step 48). Following step 46 for a homogenously patterned plant part, kernels are selected in the cropped image (step 50), the image is converted to binary (step 52), kernels are counted from the binary image (step 54). and the final kernel count is output (step 56). Following step 48 for a mixed patterned plant part, the image is segmented (step 58), individual segmented images are created (step 60), kernels are counted for each individual image (step 62), and the final kernel counts are output (step 64).

[0044] More specifically, referring still to FIG. 3, step 42 can include image collection. At step 42, a plant part, e.g., a com ear 16, can be received on the base unit 14. For example, the com ear 16 can be received in or coupled to the holder 22, which can be placed on or otherwise secured to the top 20 of the base unit 14, as shown in FIGS. 1 A and IB. The image collection device 12 can collect a raw panoramic image as a continuous array as the base unit14 rotates for at least one 360-degree revolution. Panoramic photography of com ears can be useful to obtain information from the whole ear and avoids biases arising from a partial view. In some applications, the base unit 14 can be configured to rotate at a rate of about 10 seconds per revolution.

[0045] Following image collection at step 42, the resulting image is cropped to include exactly one full revolution of the plant part at step 44. For example, a full revolution can be recognized by using a landmark on the com ear 16 and cropping the image when the landmark is recognized again. FIG. 4 shows an example of a fully prepared panoramic image with one full revolution using a landmark. More specifically, FIG. 4A illustrates a full, isometric view of a com ear 16, FIG. 4B illustrates a raw panoramic image 66 with one revolution marked at box 68 using a designated landmark, and FIG. 4C illustrates an image 70 corresponding to one full revolution of the com ear 16, i.e., corresponding to the box 68 in FIG. 4B. As another example, a full revolution can be recognized by counting a specific number of rows of com in the image that is equivalent to a full revolution such as, but not limited to, sixteen rows. FIG. 5 shows an example of a fully prepared panoramic image with one full revolution using sixteen counted rows. More specifically, FIG. 5A illustrates a full, isometric view of a com ear 16. FIG. 5B illustrates a raw panoramic image 72 with one revolution marked at box 74 by counting sixteen rows, and FIG. 5C illustrates an image 76 corresponding to one full revolution of the com ear 16, i.e., corresponding to the box 74 in FIG. 5B.

[0046] It should be noted that image cropping in step 44 can be executed automatically or manually using either of the examples described above with respect to FIGS. 4A-5C, or using other methods. Furthermore, in some applications, an automatically cropped image can be presented to a user for review and / or adjustment. Additionally, in some applications, once one full revolution of the com ear is present and / or recognized (e.g., automatically or by a user), the image may be fully prepared for kernel quantification and step 44 may be complete.

[0047] Referring back to the method 40 of FIG. 3, the method 40 can split depending on whether the plan part corresponds to homogenously patterned ears (step 46) or mixed patterned ears (step 48). This can be a manual selection by a user or may be automatically determined.

[0048] Following step 46, when homogenously patterned ears are recognized or selected, kernels are selected from the prepared image at step 50. For example, the prepared image can be processed to pull only pixels within a specified range, e.g., using an Image- Adjust-ColorThreshold function in ImageJ (an opensource image processing program). A balance is to be found when adjusting Color Threshold to keep darker kernels from blending together and lighter kernels from washing out, and manual adjustments may be made to determine a set pixel range. FIG. 6 shows selecting kernels using a Color Threshold function, including an image 80 after being Color Threshold-ed and an example user interface 82 (e.g., a graphical user interface) showing defaults of color adj ustment. including hue, saturation, and brightness.

[0049] Referring back to FIG. 3, at step 52, the image 80 output from step 50 is converted to binary to create a black and white image. For example, the image 80 is processed to convert pixels to black and white using a Process-Binary-Make Binary function in ImageJ. FIG. 7 shows an example black and white image 84 (right) and its corresponding original cropped image 86 (left). As a result, the black and white image 84 illustrates kernels as black and spacing between kernels as white. Optionally, following steps 50 or 52, further processing functions can be executed. As a first example, a Process-Binary-Watershed function can be used to separate any kernels that have blended together. More specifically, watershed separation is a technique for cutting apart connected components into separate ones. As another example, a Process-Binary-Fill Holes function can be used for areas that have the kernels washed out. That is, generally, the Fill Holes function removes holes inside particles in binary images.

[0050] From the processed black and white image 84 output at step 52, the kernels can be counted at step 54. For example, an Analyze Particle function in ImageJ can be used to accomplish kernel counting by counting particles within a defined pixel range. In some applications, noise in the image can be reduced or eliminated by restricting the size of the particles to be counted to be above, e.g.. 50-to-60 pixels.

[0051] Following step 54, the final kernel count is output at step 56. For example, a report may be generated that displays or otherwise communicates at least the kernel count to a user. This may be displayed at the image collection device 12 or another computing system.

[0052] Any of steps 44-56 can be executed automatically using image processing software on the image collection device 12 or a separate computing system. Furthermore, steps 44-56 can be repeated for a plurality of prepared images as a batch process. That is, when analyzing a batch of images with similar properties, a macro can be written to process a folder of prepared images, from adjusting the image to particle count. The following is an example ofa macro with steps to process of batch of prepared images to count the total number of kernels in a panorama image of a com ear:Macro 1 open(" C : / Users / preppedimage.j peg" ); runf'Color Threshold..."); / / setThreshold(255, 255); setOptionf'BlackBackground", false); runf'Convert to Mask"); runf'Watershed"); runf'Fill Holes"); run(" Analyze Particles...", "size=60-Infmity display clear summarize");

[0053] Referring back to the method 40 of FIG. 3, as noted above, once an image is cropped at step 44. the method 40 can split depending on whether the plan part corresponds to homogenously patterned ears (step 46) or mixed patterned ears (step 48). Following step 48, when mixed patterned ears are recognized or selected, a prepared image can be segmented using, for example, a machine learning classifier, such as with a Food Color Inspector classifier software (http: / / www.cofilab.com / portfolio / food-color-inspector / ). For example, one or more prepared images of a mixed color / pattem com ear sample can be opened in Food Color Inspector for training on representative samples categorizing purple kernels, white kernels, yellow kernels, and the gaps between the kernels as background. FIG. 8 shows a first example image set 88 of a mixed colored com ear sample used to train a Food Color Inspector, including an original image 90, segmented image 92, and adjusted binary image 94 showing total kernels (as black pixels). FIG. 8 further shoes a second example image set 96 of a mixed colored com ear sample, where the training was executed, including an original image 98, segmented image 100, and adjusted binary' image 102 showing total kernels (as white pixels). Additionally, FIG. 9 shows a graphical user interface 104 of a Food Color Inspector window showing how to train a classifier using examples of the gaps between kernels as the background class, purple kernels as class 2, white kernels as class 3, and yellow kernels as class 4 with the original image 90 and segmented image 92 (e.g., from the first sample set 88 from FIG. 8) shown on the right. While categories described herein include purple, white, yellow, and gaps, it should be noted that other additional categories can be made using the Food Color Inspector classifier.

[0054] At step 60, the obtained segmented image 92 from step 58 is color-thresholded, e.g., using default of HSB (hue-saturation-brightness) color space, to obtain an image of total kernels with kernels selected (as outlined above at step 50), and split into individualsegmented images of different-colored kernels (e.g., separate images for yellow kernels, purple kernels, and white kernels). For example, the color-thresholded image can be processed using a Color-Split Channels function in ImageJ to obtain individual segmented images of yellow kernels, purple kernels, and white kernels. Additionally, the adjusted images can be converted to binary, with Fill Holes or Watershed as needed, similar to that described above with respect to step 52. FIG. 10 shows the first sample image set 88 of a mixed colored com sample from FIG. 8, and a corresponding segmented split channel adjusted binary images for the three classes: purple kernels (left image 106), white kernels (middle image 108), and yellow kernels (right image 110).

[0055] Referring back to FIG. 3, at step 62, from the processed black and white images 106, 108, 110, the kernels can be counted. For example, similar to step 54 discussed above, the Analyze Particle function in ImageJ can be used to accomplish kernel counting. For example, FIG. 1 1 illustrates a graphical user interface 1 12 demonstrating the results of an Analyze Particles function with analyzed particles highlighted and numbered, giving a purple kernel count of 170. In some applications, to obtain a more accurate count, the size of the particles to be counted can be adjusted for the different images. As one example, during validation of the images of FIGS. 8 and 10, the size of particles to be counted was set to 1800 to infinity pixel. It should also be noted that, in some applications, total kernel count can be obtained before or after channel splitting.

[0056] Following step 62, the final kernel count is output at step 64. For example, a report may be generated that displays or otherw ise communicates at least the kernel counts for each colored kernel. This may be displayed at the image collection device 12 or another computing system and, in some implementations, can be a graphical user interface such as that shown in FIG. 11.

[0057] Any of steps 48-and 58-64 can be executed automatically using image processing software on the image collection device 12 or a separate computing system. Furthermore, steps 48-and 58-64 can be repeated for a plurality of prepared images as a batch process. For example, the Food Color Inspector classifier can work very well for batch segmentation of images, where one can train the classifier, save it, and then apply it to images that have similar categories. The following is an example of a macro for batch counting the number of kernels in a Food Color Inspector segmented image, including counting total kernels, splitting into separate segmented images, counting purple kernels, counting white kernels, and counting yellow:Macro 2 open("C: / Users / PurpleWhiteYellowPrepped_segmented.png "); runf'Color Threshold..."); runf'Close"); / / setThreshold(255, 255); setOption("BlackBackground", false); runf'Convert to Mask"); run("Fill Holes"); runf'Watershed"); run(" Analyze Particles...", "size=1800-Infinity show=[Overlay Masks] display clear summarize overlay"); closed: open("C: / Users / PurpleWhiteYellowPrepped_segmented.png "); runf'Split Channels"); selectWindow("PurpleWhiteYellowPrepped_segmented.png (blue)"); aveAs("Tiff","C: / Users / PurpleWhiteY ellowPrepped_segmentedCountedYellow.tif"); close(); saveAs("Tiff","C: / Users / P urpleWhiteYellowPrepped_segmentedCountedWhite.tif'); close(); saveAs("Tiff","C: / Users / P urpleWhiteYellowPrepped_segmentedCountedPurple.tif); closed; open("C: / Users / PurpleWhiteYellowPrepped_segmentedPurple.png "); run("Options...", "iterations=l count=l black do=Nothing"); runf'Convert to Mask"); runf'Watershed"); run(" Analyze Particles...", "size=1800-Infinity show=[Overlay Masks] display clear summarize overlay"); saveAsf'Tiff","C: / Users / Purpl eWhiteYellowPrepped_segmentedCountedPurple.tif"); closed; open("C: / Users / P urpleWhiteYellowPrepped_segmentedCountedWhite.tif'); runf'Fill Holes");run(" Analyze Particles...", "size=1800-Infmity show=[Overlay Masks] display clear summarize overlay"); saveAs("Tiff","C: / Users / P urpleWhiteYellowPrepped_segmentedCountedWhite.tif); close(); open("C: / Users / PurpleWhiteY ellowPrepped_segmentedCountedYellow.tif); runf'Fill Holes"); runfWatershed"); run(" Analyze Particles...", "size=1800-Infinity show=[Overlay Masks] display clear summarize overlay"); saveAs("Tiff', "C: / Users / yinxx / OneDrive / Desktop / Postdoc / com360 / PurpleWhiteYellowPrepped _segmentedCountedY ellow.tif'); close();

[0058] Accordingly, the above method 40 may be used to obtain a total kernel count of homogenously patterned kernels or to obtain a total kernel count for different colors of a mixed color / pattem com sample. In some applications, the above method 40 can be further applied to other plant parts, such as for analyzing tree trunks, circular fruit (e.g., melons or others), or for analyzing other characteristics of com.

[0059] As another example, in some embodiments, the method 40 can be used for segmenting kernels from mixed texture com samples into selected categories, e.g., using Food Color Inspector and steps 42, 44, 48 and 58-64 above. More specifically, FIG. 12 illustrates examples of sticky / waxy com 114, which can be yellow and opaque, dried sweet com 116, which can be yellow, shriveled and translucent, and starch com 118, which can be yellow and opaque with a translucent appearance. Furthermore, FIG. 13 illustrates various images of an example a dried hybrid com ear 120 resulting from a sweet x sticky cross, including a whole view of the sweet x sticky cross com ear 120 (left), a panoramic prepared image (middle), and a side view (right) are shown, where starch com is circled in green 119, sticky com circled in magenta 121, and sweet com is circled in white 123. For example, there can be up to three sweet genes in com, where one or two are mostly responsible for the sweetness in traditional sweet com, a recessive mutation of one of the sweet genes is responsible for sticky kernels, while other sticky genes can exist in the background of the sweet genes. The genetic inheritance of kernel composition traits can thus be presented using the methods described herein.

[0060] To be able to quantify the segregation ratio in a sweet com x sticky com cross, as shown in FIG. 13. would offer more insights into the genetic relationships of these genes. Accordingly, in applying the method 40 above to mixed texture com samples, steps 42, 44, 48 and 58-64 may be used, though Food Color Inspector may be trained to segment different textures rather than colors. For example, FIG. 14 shows a first example image set 122 of a mixed textured yellow com sample of different texture used to train a Food Color Inspector, including an original image 124, segmented image 126, and adjusted binary image 128 showing total kernels; and a second example image set 130 of a mixed textured yellow com ear sample where the training was executed, including an original image 132, segmented image 134, and adjusted binary image 136 showing total kernels. In this example, the size of the particles to be counted was 100 to infinity pixel for total kernel and starch kernel counts, and 350 to infinity and 300 to infinity pixel for sticky and sweet kernel counts, respectively.

[0061] FIG. 15 shows example segmented split channel adjusted binary images for the three separate classes: sticky kernels (left image 138), starch kernels (middle image 140), and sweet kernels (right image 142) of the first mixed textured com sample set 122 show n in FIG. 14. As another example, during validation of the images of FIG. 14 and 15, the size of particles to be counted w as 1000 to infinity pixel for kernel counts except for sticky kernel count, which was 1800 to infinity pixel.

[0062] Referring now to specific examples testing the method 40 above, in a first example, total kernel count of homogenously patterned kernels using the above method 40 (e.g., steps 42, 44, 46, and 50-56) was tested. That is, for com ears with homogeneously patterned kernels, the total kernel count was obtained via manual count (e.g., via counting the image 86 in FIG. 7) versus the automated count on a black and white (binary) image using method 40 (e.g., from the image 86 in FIG. 7). The method 40 showed an average accuracy of 93.7% compared with manual count, as shown below' in Table 1. Percent accuracy was calculated as Accuracy = - xm where xm is manual count and xa is the method 40 count.Furthermore, the method 40 takes an average of 33.75 seconds per each sample (e.g., the time it takes to run and modify the macro), while manual count takes 4 minutes. 13 seconds per sample, saving about 3 minutes, 40 seconds per image.Table 1 Total kernel count of different sample com ears using the ImageJ binary analysis count and manual countCom ear samples Replicate Kernel count _ %Accuracy Counting time _Manual ImageJ Al Manual ImageJ AlRed 1 423 421 99.5 4 min 2 s 17 sRed 2 611 602 98.5 5 min 27 s 1 min 2 sYellow 1 269 265 98.5 4 mm 32 s 54 sYellow 2 264 321 78.4 2 min 51 s 2 s

[0063] In a second example, kernel count of com ears of mixed colors using the above method 40 (e.g., steps 42, 44, 48, and 58-64) was tested. The accuracy for total kernel count (obtained from a black and white image before channel splitting) was 85.9% and average accuracy for segmented kernel counts was 84.8%, including purple, white, and yellow kernels having accuracies of 85.4%, 87%, and 79.3%, respectively, as shown in Table 2 below. The method 40 takes an average of 2 minutes, 37 seconds per category per sample, slightly longer than manual counting which takes 2 minutes, 3 seconds.Table 2 Total and segmented kernel count of mixed colored com ear sample (sample 1 from image set 88 in FIG. 8) used to train the classifier of the Food Color Inspector and total and segmented kernel count of mixed colored com sample (sample 2 from image set 96 in FIG. 8) where the classifier was executed.Kernel Sample Kernel count _ %Accuracy Counting time _Manual Al Manual AlTotal 1 381 285 74.8 3 min 21 s 5 min 51 sPurple 1 238 175 73.5 3 min 2 minWhite 1 103 83 80.6 1 min 41 s 3 minYellow 1 29 23 79.3 55 s 1 min 50 sTotal 2 301 310 97 2 min 24 s 2 min 20 sPurple 2 142 146 97.2 1 min 27 s 2 min 33 sWhite 2 151 141 93.4 1 min 33 s 48 s

[0064] Additionally, while the average accuracy for total kernel count was 85.9%, the method 40 for homogenous com (e.g., steps 42, 44, 46, and 50-56) was also tested on a mixed color ear and resulted in an average accuracy of 24.9% (data not shown). Accordingly, higher accuracies were obtained for mixed color / pattem com samples using the Food Color Inspector to first segment and then count the total kernels of the binary image than counting from the binary image directly.

[0065] In a third example, kernel count of com ears of mixed textures (in sweet x sticky hybrid samples) using the above method 40 (e.g., steps 42, 44, 48, and 58-64) was tested. The16SUBSTITUTE SHEET (RULE 26)average accuracy for counting total kernels was 93.9%, for segmented kernel count was 61.8%, for sweet kernels was 79.6%, for sticky and starch kernels was 28.5% and 77.4% respectively (Table 3). The average manual counting time was 2 minutes, 30 seconds, while method 40 counting time was 1 minute, 55 seconds, saving 35 seconds per image.Table 3 Total and segmented kernel count of mixed textured yellow com ear sample used to train the classifier of the Food Color Inspector and total and segmented kernel count of mixed textured yellow com sample where the classifier was executedKernel Sample Kernel count %Accuracy Counting time _Manual Al Manual AlTotal 3 341 307 90 2 min 52 s 4 minSolid opaque (sticky) 3 65 98 49.2 3 min 50 s 2 minShiny opaque (starch) 3 175 101 57.7 3 min 54 s 1 min 25 sShriveled translucent (sweet) 3 85 64 75.3 51 s 53 sTotal 4 318 311 97.8 2 min 41 s 2 min 18 sSolid opaque (sticky) 4 39 75 7.7 2 min 2 min 17 sShiny opaque (starch) 4 170 175 97.1 2 min 43 s 1 min 2 sShriveled translucent (sweet) 4 87 73 83.9 1 min 6 s 1 min 28 s

[0066] According to the second and third examples above, kernel counts of different categories of mixed patterned com samples can be obtained with relatively high accuracies (84.8% or 61.8%) using Food Color Inspector. Thus, the method 40 has the potential to save a vast amount of time to count segmented kernels of a mixed patterned com sample, especially as the number of images increases.

[0067] The above panoramic, high-throughput, cost-effective, easily accessible methods of some embodiments not only can assist in phenotyping for plant improvement, but also assist in furthering the understanding of genetic inheritance of certain traits. This is particularly true to study genes that contribute to the white, waxy, sweet, popcorn, high- amylose, high oil, and quality protein attributes in different types of com. For example, the yellow, red, and purple kernels were found to be controlled by four genes and produced by pigments synthesized in the carotenoid and antho-cyanin pathways, while the white kernels are a result of a lack of pigments produced from either pathway.

[0068] Additionally, as an example of using the system 10 and method 40 described herein to study genetic inheritance, with the sweet x sticky hybrid sample, a Chi-square test was conducted to test for the segregation ratio, where / 2=E ^^^Expected ^^ '17SUBSTITUTE SHEET (RULE 26)hypothesis was that starch:sweet:sticky kernel count fits a 9:4:3 ratio. In other words, if the hypothesis was true, the starchiness, sweetness, and stickiness of the kernels in this sample are controlled by two genes with epistatic effects. A dominant copy at both genes (A_B) are required to have starch kernels, a dominant copy at the first gene but not the second gene (A_bb) results in sticky kernels, no dominant copy at the first gene (aaB and aabb) results in sweet kernels. The gene A is very likely Sugaryl. Recessive sugaryl mutations have been shown to be responsible for sweetness in traditional sweet com, where the Sugaryl gene is responsible for amylopectin crystallinity and biosynthesis that is important for sticky com. A second gene B seems to be important for starch synthesis, where gene A plays an epistatic role, such that gene B needs gene A to produce starch and without gene A, a dominant copy of gene B still results in sweet kernels.. - . > (175-19181)2(85-85.2S)2(6S-63.94)2„ . .0069 The test resulted in: %2 = - 1- - - 1- - - = 1.49 with degrees of freedom of 2. This 1.49 value is smaller than the Chi-square value at alpha 0.05 which is 5.99. Thus, segmented kernel counts identified by the methods 40 herein can be used to study trait segregation and, as illustrated above, can demonstrate that a com sample with mixed starch: sweet: sticky kernels was segregating in a 9:4:3 ratio. This can help better inform the inheritance of these genes in a sweet x sticky germplasm, which can benefit breeders in making informed breeding decisions in the future and can also be used as a case study to teach Mendelian genetics and epistasis.

[0070] It should be noted that a system for carrying out the above methods 40 may include both the image collection device 12 and the base unit 14, such as the system 10 described above, may only include an image collection device 12, or may only include a base unit 14. For example, in some applications, the base unit 14 may be sold as a separate system, to be used with a smart phone or other image collection device. As another example, the methods 40 can be earned out using a panoramic image obtained by other means.

[0071] Furthermore, as noted above, the system 10 and / or method 40 can be used to analyze other cylindrical object surface patterns, such as on tree trunks, circular or cylindncal fruits, etc. Accordingly, with equipment modifications such as a rotating trunk-mounted tripod track or an aerial drone, the methods 40 can be used to analyze fruits and tree trunks in the field, for non-destructive assessment for diseased spots, growth rate, or bark pattern, etc. FIG. 16 shows an example panoramic image 144 of a tree bark pattern using a system in accordance with some embodiments, which may be analyzed according to a method similar18SUBSTITUTE SHEET (RULE 26)to the method 40 described above. Additionally, FIG. 17, described further below, shows another example system for use with live tree trunks to obtain panoramic images, which may be analyzed according to a method similar to the method 40 described above. Thus, reference to an ear of com or plant part throughout the disclosure can be applied to any object, where the object is a plant, or plant part thereof, including but not limited to a vegetative part (i. e. , stem, stalk, branch, root, petiole, cob, glume, shoot, bud, cotyledon, hypocotyl, plumule); a flower, or part thereof (i.e. flower, bud, petals, ovary, ovules, stamen, pistil, tassel, anther, pollen); a vegetable and / or fruit, or part thereof (i.e., ear or kernel of com, legume or bean pod, legume, tomato, raspberry, blackberry, blueberry, strawberry, melon, squash, cucumber, broccoli, cauliflower, onion, garlic); a nut and / or seed, or part thereof (i.e., seed, seed pod, grain, nut, nutshell, seed coat, embryo, endosperm, aleurone), atree, or part thereof (i.e., trunk, branch, stem, root, leaf).

[0072] Furthermore, while the system 10 and method 40 described herein are directed toward quantifying kernels, they can also be directed toward measuring other variables to provide a robust, low-cost way for com breeders (or other plant breeders) to screen through hundreds and thousands of varieties efficiently. For example, besides total kernel and segmented kernel counts, other traits such as com ear length, width, circumference, diameter, radius, row number, kernel area, and kernel size, as well as those similar to grape cluster traits such as aspect ratio (com ear radius / length), compactness, to color, 2-D or 3-D shape and / or dimensions can also be efficiently extracted using the panoramic imaging capturing system 10 and variations of the methods 40 described herein. Additionally, in some embodiments, the kernels may be further divided into parts, including but not limited to the pericarp, the aleurone, the endosperm, and the embryo, and the kernel parts are quantified and / or classified into different color categories. In further embodiments, the color or dimensions of the spaces between the kernels are measured.

[0073] In light of the above, the system 10 and methods 40 of some embodiments utilize image processing software that do not require programming or statistical expertise. As such, the system 10 and methods 40 provide a quick and direct method to assess kernel counts in a portable way for average breeders who may or may not have the programming and / or statistical expertise. However, the methods still allow for manual adjustment to be done on particle size selection on different samples to differentiate and count certain kernels.

[0074] Within these methods 40, several parameters may be recommended to be kept constant while a few parameters can be adjusted for each different sample. As a first example 19SUBSTITUTE SHEET (RULE 26)to train the classifier more precisely, more examples of the gaps between kernels as background can be selected, particularly when ImageJ has a hard time to differentiate adjacent same patterned kernels. As a second example, all images can be taken at the same distance to the com ears, under the same illumination, to avoid much of the manual tweaking of the image analysis process. With these parameters kept constant, once the image in cropped, the places that can be adjusted for each homogeneously patterned com ear are the Kernel Selection step for Watershed and Fill the Holes functions, and Color Threshold values, and the Kernel Counting step for particle size to be counted. In addition to adjustment of the image processing and counting steps, different patterned ears can be trained anew using the Food Color Inspector. The classifier can be trained on the com ear with the most patterns, and then apply it on com ears with any subset of the patterns. For example, the classifier trained using a com sample of purple, yellow, and white kernels can be applied to and segment a com sample with purple and white kernels.

[0075] In some applications, accuracy of the method 40 may be further improved through integration of programming-based image analysis pipelines. Another potential way to further improve accuracy includes enhancement of the kernel edges using the color deconvolution algorithm or the connected domain analysis.

[0076] As noted above, the system 10 and / or method 40 described herein can be used to analyze other cylindrical object surface patterns, such as on tree trunks, cylindrical fruits, etc. Furthermore, FIG. 17 illustrates another example sy stem 200 that can be used to analyze a plant part 210, such as fruits and tree trunks of any size in the field, for non-destructive assessment for diseased spots, growth rate, or bark pattern, natural tree bark development, other bark features, sunburn damage, etc. More generally, the system 200 can be used for any large immobile cylindrical structure surface analysis. In some applications, the system 200, along with the methods 40 described herein, can be used for forest tree surveys, after fire tree bark analyses, and / or disease infected and sunburn tree bark studies, among other analyses.

[0077] Referring still to FIG. 17, the system 200 can include a ring holder 202, a support bar 204, a circular track 206, and a device holder 208. The ring holder 202 can be configured to fit around a plant part, such as a tree trunk 210, and secured via a clip 212. In some embodiments, the ring holder 202 and / or the clip 212 can be adjustable to accommodate a range of circumferences of plant parts. However, in other embodiments, as shown in FIG. 17, different-sized ring holders 202 can be used for plant parts of different circumferences. The20SUBSTITUTE SHEET (RULE 26)ring holder 202 can secure the system 200 to the plant part 210, and can be coupled to the circular track 206 via one or more support bars 204. For example, the support bars 204 can be substantially rigid bars that extend radially outward and downward from the ring holder 202. As a result, the circular track 206 is positioned around a circumference of the plant part 210, a distance away from the plant part 210. The device holder 208 can be coupled to the circular track 206 and can be configured to hold an image collection device 214 (e.g., similar to the image collection device 12 described above). The device holder 208 can be coupled to the circular track 206 in a manner that allows it to be moved along the circular track 206, enabling the image collection device 214 to obtain a panoramic image around a circumference of the plant part 210. For example, the device holder 208 can include a micromotor 216 (e.g., optionally including a rechargeable battery unit and a remote sensor), that can drive the device holder 208 along the circular track 206 in one or both directions. In some embodiments, the micromotor 216 can be remotely controlled (e.g., via the image collection device 214 or another computing system) to move the device holder 208 around the circular track 206 for image data collection. Once image data of the plant part 210 is collected via the system 200, the image data may be analyzed according to a method similar to the method 40 described above.

[0078] It will be appreciated by those skilled in the art that while the invention has been described above in connection with particular embodiments and examples, the invention is not necessarily so limited, and that numerous other embodiments, examples, uses, modifications and departures from the embodiments, examples and uses are intended to be encompassed by the claims attached hereto. The entire disclosure of each patent and publication cited herein is incorporated by reference, as if each such patent or publication were individually incorporated by reference herein. Various features and advantages of the invention are set forth in the following claims.21SUBSTITUTE SHEET (RULE 26)

Claims

CLAIMS1. A method for analyzing an object, comprising: collecting image data of the object, the image data including a panoramic image capturing at least one entire 360-degree circumferential view of the object; cropping the panoramic image to create a prepared image containing a single 360- degree view of the object; processing the prepared image to produce a processed image; and quantifying features of the object via the processed image.

2. The method of claim 1, wherein collecting the panoramic image of the object includes: placing the object on a base unit configured to rotate at least 360 degrees; and collecting the panoramic image with a stationary' image collection device while rotating the base unit.

3. The method of claim 2, wherein the base unit is a turntable.

4. The method of claim 2, wherein placing the object on the base unit includes securing the object to the base unit via a screw.

5. The method of claim 2, wherein placing the object on the base unit includes placing the object in a clear container and setting the clear container on the base unit.

6. The method of claim 1, wherein collecting the panoramic image of the object includes taking the panoramic image with a camera of a smart device.

7. The method of claim 1, wherein the object is an ear of com, and quantifying features of the object includes quantifying kernels.

8. The method of claim 7, wherein quantifying kernels includes quantify ing total kernels.

9. The method of claim 7, wherein quantifying kernels includes quantifying kernels having a certain feature from the following list of features: a certain color or a certain texture.

10. The method of claim 1, wherein the object is a tree trunk, and quantifying features of the object includes quantifying tree bark features.

11. A system for analyzing features on a plant part imaged by an image collection device, the system comprising: a base unit configured to rotate the plant part while the plant part is imaged by the image collection device, the base unit comprising: a base; a top rotatable relative to the base; and a holder configured to receive the plant part, the holder positioned on the top so that the plant part rotates with the top.

12. The system of claim 11, wherein the holder includes a clear container.

13. The system of claim 12, wherein the holder is integral with the top.

14. The system of claim 12, wherein the holder is coupled to the top.

15. The system of claim 11, wherein the holder is a screw or pin that extends upward from the top to receive the plant part.

16. The system of claim 11, wherein the base unit further comprises a rotating mechanism that rotates the top relative to the base and a controller configured to activate the rotating mechanism.

17. A method for quantifying kernels on a com ear, comprising: collecting image data of the com ear, the image data including a panoramic image capturing at least one entire 360-degree circumferential view of the com ear; cropping the panoramic image to create a prepared image containing a single 360- degree view of the com ear; performing a color threshold function on the cropped image to identify7pixels within a predetermined range within the prepared image and create a color threshold image; converting the color threshold image to a black and white binary image; andanalyzing particles of a predetermined pixel range within the binary image to quantify kernels within the binary image.

18. The method of claim 17, further comprising segmenting the cropped image into separate classes using trained segmentation software prior to performing the color threshold function.

19. The method of claim 18, wherein performing the color threshold function includes performing the color threshold function for each segmented image.

20. The method of claim 18. wherein the classes include one of separate colors or separate textures.

Images