Egg sorting using three-class classifier and spectral data

Abstract

A method includes sensing light (402) coming from an egg (502) to produce spectral data (509) for the egg (502), and applying (406) at least some of the spectral data (513) to a three-class classifier (248) to identify one class of three classes for the egg, wherein the three classes consist of unfertilized, fertilized and male, and fertilized and female.

Description

U011.0416W01-1-EGG SORTING USING THREE-CLASS CLASSIFIER AND SPECTRAL DATABACKGROUND

[0001] Egg sorting is used to group eggs that have common characteristics. This often includes grouping eggs that are unfertilized into one group, grouping eggs that are fertilized and have male embryos into a second group and grouping eggs that are fertilized and have female embryos in a third group.SUMMARY

[0002] A method includes sensing light coming from an egg to produce spectral data for the egg. and applying at least some of the spectral data to a three-class classifier to identify one class of three classes for the egg, wherein the three classes consist of unfertilized, fertilized and male, and fertilized and female.

[0003] In accordance with a further embodiment, an egg sorter includes an inspection area for inspecting an egg, the inspection area having: a light source that applies light to the egg; and a light sensor that senses light from the egg and generates spectral values from the sensed light. A classifier uses the spectral values to perform a single classification of the egg into one of three classes consisting of unfertilized; fertilized and male; and fertilized and female and an actuator controller controls at least one actuator based on the single classification of the egg so as to move the egg to an area designated for eggs having the class of the egg.

[0004] In accordance with a still further embodiment, a method of sorting an egg includes applying spectral data obtained from light from the egg to a three-class classifier to assign the egg to one of three classes with a single classification and using the class to which the egg is assigned to control an actuator so as to place the egg with other eggs of the same class as the egg.

[0005] In some embodiments, the spectral data is filtered to form filtered spectral data, wherein the filtered data has fewer spectral bands than the spectral data. The filtered spectral data is applied to the three-class classifier to identify the class for the egg.

[0006] In some embodiments, the filtered spectral data comprises less than twenty six spectral bands.

[0007] In some embodiments, the spectral data comprises short wave infrared light.U011.0416W01-2-

[0008] In some embodiments, the spectral bands of the filtered spectral data are identified during training of the three-class classifier.

[0009] In some embodiments, the three-class classifier is a support vector machine.

[0010] In some embodiments, the classifier uses spectral values in the visible and shortwave infrared ranges.

[0011] In some embodiments, the filtered spectral bands are selected while training the classifier.

[0012] This Summary is provided to introduce a selection of concepts in a simplified form that are further described below in the Detailed Description. This Summary is not intended to identify key features or essential features of the claimed subject matter, nor is it intended to be used as an aid in determining the scope of the claimed subject matter.BRIEF DESCRIPTION OF THE DRAWINGS

[0013] FIG. 1 is a flow diagram of a method for training a three-class classifier.

[0014] FIG. 2 is a block diagram of a system used to perform the method of FIG. 1.

[0015] FIG. 3 is a perspective view of a light sensing system in accordance with one embodiment.

[0016] FIG. 4 is a method of sorting eggs.

[0017] FIG. 5 is a block diagram of an egg sorter.DETAILED DESCRIPTION

[0018] The number of eggs that an egg sorting machine can process in a day is largely dependent on how quickly the machine can determine the fertilization status of the egg and, if the egg is fertilized, the sex of the embryo. In the past, a two-stage system was used in which a determination of the fertilization status of the egg was made using an initial 2-class classifier and then a determination of the sex of the embryo was made using a second 2-class classifier. Having two classifiers was thought to provide better classification because each classifier could use different input features that are selected to optimize each classification. However, using two classifiers in series slows the sorting process.U011.0416W01-3-

[0019] In the embodiments described below, a three-class classifier is provided that sorts each egg into one of three classes: unfertilized, male and female. Counterintuitively, this three-class classifier provides high accuracy in both identifying whether an egg is fertilized and in identifying the sex of fertilized eggs using a common set of input features. In addition, because a single classifier is provided, the sorting process takes less time thereby allowing more eggs to be sorted in a work period.

[0020] FIG.l provides a flow diagram of a method of training a three-class classifier for egg sorting and FIG. 2 provides a block diagram of a system for implementing the method of FIG. 1.

[0021] In step 100 of FIG. 1, a spectrum of light is selected. In accordance with one embodiment, a spectrum of 400 - 1700 nm is selected in step 100, which includes visible light, Near Infrared (NIR) light, and Short Wave Infrared (SWIR) light.

[0022] In step 102, training data is obtained along the entire spectrum selected in step 100. In FIG. 2, the training data is obtained by placing an egg, such as egg 202, between a light source 204 and a light sensor 206 and measuring the intensity of light received by light sensor 206 from egg 202. In particular, a set of overlapping spectral bands that cover the entire selected spectrum is selected and the intensity of light for each spectral band is measured by light sensor 206.

[0023] FIG. 3 provides a perspective view of a spectral measurement system for obtaining spectral data in step 102 according to one embodiment. In the system of FIG. 3, two cameras 306 and 308 are used as light sensor 206. Camera 308 is a hyperspectral camera (400 - 950 nm) for the visual / near infrared spectral range such as the Pika L camera from Resonon®. Camera 308 operates as a push-broom imager that captures a slice of the object with each image. The camera includes a grating that separates the incoming light into different bands that are applied to separate sensors. Camera 308 has two hundred forty spectral bands with an average band width of 2.1nm. Thus, at each pixel position along the length of the slit, different sensors provide a respective sensed value for a spectral band associated with the sensor. To capture image data for the entire object, camera 308 is moved along a sliding track 310 by a motor (not shown) so that multiple image slices of the object are captured.

[0024] Camera 306 is a hyperspectral camera (950 - 1700 nm) for the Short Wave Infrared (SWIR) spectral range such as model 4400H from Hinalea®. Camera 306 captures a full hyperspectral image cube in a single shot and has 400 bands with an average bandwidth of 2 nm.U011.0416W01-4-

[0025] Both cameras capture hyperspectral images with 1 mm spatial resolution.

[0026] Cameras 306 and 308 are placed in a dark room 304 with the egg sample 302 placed in an egg holder 300 positioned on a floor 314 of dark room 304. (The other two walls and the ceiling of dark room 304 are not shown so that the interior of room 304 can be seen). A light source 312 is positioned below floor 314 and directs light upward such that cameras 306 and 308 receive light transmitted through egg sample 302. In accordance with one embodiment, light source 312 is a halogen light source that provides illumination across a broad spectral range. The position of light source 312 below floor 314 is adjustable to allow for fine-tuning of the lighting intensity provided to the egg to achieve optimal lighting such that sufficient illumination is provided for imaging without overexposure. Cooling fans (not shown) are used in darkroom 304 to prevent heat buildup from lighting and imaging, protecting the egg during the imaging process. The imaging time for both cameras is set to be 0.1 second, which is shorter than traditional candling methods, to provide sufficient integration for imaging while avoiding potential negative impact on the egg sample.

[0027] Egg holder 300 allows at least one quarter of the eggshell surface to be exposed to photons to ensure sufficient brightness in the raw images. Egg holder 300 is also designed with light leaking prevention for optimum transmittance imaging.

[0028] The measured intensities for the spectral bands are provided to an input / output interface 208 of a computing device 201, which stores the intensities as training data 214 in a memory 210 of computing device 201. Tn accordance with one embodiment, as part of storing the intensities as training data 210, a processor 212 in computing device 201 performs calibration, masking, and normalization on the intensity values.

[0029] In particular, hyperspectral image preprocessing for egg analysis involves four key steps to enhance data quality and ensure consistency across datasets. First, initial calibration needs to be conducted to correct for camera dark current noise and uneven illumination, achieved by collecting dark field and white panel images. The dark field image is collected with the lens cap on the camera. This produces a Dark value for each spectral band of each pixel. The white panel image is collected without an egg in egg holder 300. This produces a White value for each spectral band of each pixel.U011.0416W01-5-

[0030] For each hyperspectral image, a radiance value is determined for each spectral band of each pixel using the input sensor value, Raw, for the respective spectral band of the respective pixel and formula 1 below:[L0031

[0032] where CF is a radiance calibration factor for the spectral band and pixel, which is provided by the camera manufacturer.

[0033] The radiance value is normalized against a radiance value for the white panel image to produce a standardized transmittance value using formula 2 below:

[0035] Where Rsis the radiance value for the spectral band and pixel of the egg sample, and Rwis the radiance value of the standard white panel for the spectral band and pixel.

[0036] Normalizing the radiance standardizes the measurements, allowing consistent analysis across samples.

[0037] The third step is to isolate the egg from the background, for which the Otsu adaptive threshold algorithm is applied, effectively segmenting the egg pixels by distinguishing them from the darker background. In this step, a transmittance value is identified that best separates the transmittance values in an image into two classes. The two classes will then include one class containing pixels for the dark area surrounding the egg and the other class containing pixels for the egg. This step focuses analysis on the egg area only, ensuring the exclusion of background noise.

[0038] Finally, the segmented egg area undergoes normalization to eliminate spectral baseline drift due to physical factors, such as variations in egg variety and eggshell scattering, as shown in Formula 3 below:

[0040] Where X is the transmittance for a spectral band and pixel of the egg portion of the image, X is the mean value of the spectral bands of the pixel, and axis the standard deviation of the spectral bands of the pixel. The spectrums produced through formula 3 have a mean of zero and a standard deviation of one.

[0041] This process is repeated for each egg in a collection of eggs that includes unfertilized eggs, eggs with male embryos and eggs with female embryos. The true content of each egg isU011.0416W01-6-determined by performing a Polymerase Chain Reaction (PCR) test 203 to determine whether the egg is fertilized and the sex of the embryo if the egg is fertilized. The fertilization status and the sex of the embryo, if any, is associated with the spectral band intensities of the egg in training data 214 such that for each egg, it is possible to retrieve the spectral band intensities recorded for the egg and the corresponding fertilization status and sex of the embryo, if any.

[0042] In accordance with one embodiment, the PCR test is performed several days after the spectral data is collected. For example, for brown eggs, the PCR test is performed ten days after the spectral data is collected and for white eggs, the PCR test is performed twelve days after the spectral data is collected. This delay provides more accurate PCR data.

[0043] At step 104, a model trainer 216 starts training a 3-class classifier using training data 214. At step 106, a spectral band selector 224 in model trainer 216 selects a subset 226 of the spectral bands that were used to form training data 214. In step 110, selected subset of spectral bands 226 is used to form a filter 234 by a filter formation unit 232. Filter 234 is then used by a filtering algorithm 236 against training data 214 to form filtered training data 238 at step 112. Filtered training data 238 only includes light intensity values for the spectral bands in select spectral bands 226. Each set of spectral band intensities in filtered training data 238 also includes the designation of unfertilized, fertilized and male, or fertilized and female that PCR testing 203 provided for the egg that produced the spectral band intensities. At step 114, filtered training data 238 is used by classifier trainer 240 to form a 3-class classifier 242 that provides the most accurate 3-class classification for filtered training data 238. Classifier trainer 240 also generates classification metrics 244 that indicate how well 3-class classifier 242 performs in classifying filtered training data 238.

[0044] In step 116, model trainer 216 determines if training is complete. In accordance with one embodiment, this determination is made based on the 3-class classifier metrics 244 for one or more of the 3-class classifiers 242. If model trainer 216 determines that training is not complete, the process returns to step 106 and spectral band selector 224 selects a new subset of spectral bands. Steps 110, 112, 114 and 116 are then performed for the new subset of spectral bands.

[0045] When model trainer 216 determines that training is complete at step 116, model trainer 216 outputs a final 3-class classifier 248 and a final filter 246 at step 118. Final 3-classU011.0416W01-7-classifier 248 is the optimum 3-class classifier trained by classifier trainer 240 and final filter 246 is the filter for the selected spectral bands associated with final 3-class classifier 248.

[0046] Final filter 246 uses a smaller number of spectral bands than found in earlier egg classifiers. For example, there are twenty-five spectral bands in final filter 246 of one embodiment. This small number of spectral bands allows the classifier of the present embodiments to operate faster than previous classifiers, which used over two-hundred spectral bands. In accordance with one embodiment, final filter 246 includes at least one spectral band in the visible spectrum and at least one spectral band in the short wave infrared spectrum.

[0047] Final three-class classifier 248 is able to assign an egg to one of three classes: unfertilized; fertilized and male; and fertilized and female, in a single classification step instead of performing two classification steps in series. In accordance with one embodiment, final three-class classifier 248 is a support vector machine.

[0048] Once final three-class classifier 248 has been produced, it can be used in an egg sorting system such as egg sorting system 500 of FIG. 5. FIG. 4 provides a flow diagram of a method of operating egg sorting system 500 of FIG. 5.

[0049] In step 400, a new egg, such as egg 502, is placed in an inspection area 505 consisting of a light source 504 and a light sensor 506 using a conveyor 508. Inspection area 505 may have the same construction as the system shown in FIG. 3. At step 402, light sensor 506 collects spectral values consisting of intensity values for light across the selected spectrum, which are provided to input / output interface 511 of device 501. Device 501 can be a computer or a combination of application-specific hardware. Input / output interface 511 stores the intensity values as spectral data 509 in a memory 510 of device 501. Storing the intensity values includes performing the hyperspectral image preprocessing steps described above.

[0050] At step 404, spectral data 509 is filtered by filtering function 236 using final filter 246 of FIG. 2 to produce filtered spectral data 513. The spectral data in filtered spectral data 513 includes only the spectral bands selected by final filter 246. At step 406, the spectral band intensities of filtered spectral data 513 are applied to final three-class classifier 248, which in one embodiment is a support vector machine. Final three-class classifier 248 identifies one of three classes for egg 502: unfertilized, fertilized and male or fertilized and female at step 408. This classification is then used by an actuator control 512 at step 410 to control selection mechanisms,U011.0416W01-8-such as gates 514 and 516, to direct egg 502 to a desired bin designated to contain eggs of the class identified by final three-class classifier 248. For example, gate 514 can be actuated between two positions 518 and 520. When gate 514 is in position 520, gate 514 guides egg 502 to unfertilized bin 522. When gate 514 is in position 518, gate 514 guides egg 502 toward gate 516. Gate 516 can be actuated between positions 524 and 526. When gate 516 is in position 524, gate 516 guides egg 502 to fertilized-and-female bin 528. When gate 516 is in position 526, gate 516 guides egg 502 to fertilized-and-male bin 530. Thus, when final three-class classifier 248 indicates that egg 502 is unfertilized, actuator control 512 moves gate 514 to position 520. When classifier 248 indicates that egg 502 is fertilized and male, actuator control 512 moves gate 514 to position 518 and gate 516 to position 526. When classifier 248 indicates that egg 502 is fertilized and female, actuator control 512 moves gate 514 to position 518 and gate 516 to position 524.

[0051] Although the eggs are described as being directed into bins above, the eggs may be directed to any desired conveyances, areas or containers.

[0052] Although elements have been shown or described as separate embodiments above, portions of each embodiment may be combined with all or part of other embodiments described above.

[0053] Although the subject matter has been described in language specific to structural features and / or methodological acts, it is to be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or acts described above. Rather, the specific features and acts described above are disclosed as example forms for implementing the claims.

Claims

U011.0416W01-9-WHAT IS CLAIMED IS:

1. A method comprising:sensing light coming from an egg to produce spectral data for the egg;applying at least some of the spectral data to a three-class classifier to identify one class of three classes for the egg, wherein the three classes consist of unfertilized, fertilized and male, and fertilized and female.

2. The method of claim 1 further comprising:using the identified class to control at least one actuator so as to sort the egg.

3. The method of any of claims 1 and 2 further comprising filtering the spectral data to form filtered spectral data, wherein the filtered data has fewer spectral bands than the spectral data and wherein applying at least some of the spectral data to the three-class classifier comprises applying the filtered spectral data to the three-class classifier.

4. The method of claim 3 wherein the filtered spectral data comprises less than twenty six spectral bands.

5. The method of any of claims 1 -3 wherein the spectral data comprises short wave infrared light.

6. The method of any of claims 3-5 wherein the spectral bands of the filtered spectral data are identified during training of the three-class classifier.

7. The method of any of claims 1-6 wherein the three-class classifier is a support vector machine.

8. An egg sorter comprising:an inspection area for inspecting an egg. the inspection area comprising:U011.0416W01-10-a light source that applies light to the egg; anda light sensor that senses light from the egg and generates spectral values from the sensed light;a classifier that uses the spectral values to perform a single classification of the egg into one of three classes consisting of unfertilized; fertilized and male; and fertilized and female; andan actuator controller that controls at least one actuator based on the single classification of the egg so as to move the egg to an area designated for eggs having the class of the egg.

9. The egg sorter of claim 8 wherein the classifier uses spectral values in the visible and shortwave infrared ranges.

10. The egg sorter of any of claims 8 and 9 wherein the generated spectral values comprise a set of spectral bands and wherein the classifier uses only a subset of the set of spectral bands.

11. The egg sorter of claim 10 wherein the subset of the set of spectral bands consists of fewer than twenty-six spectral bands.

12. The egg sorter of any of claims 10 and 11 wherein the subset of the set of spectral bands is selected while training the classifier.

13. The egg sorter of any of claims 8-12 wherein the classifier comprises a support vector machine.

14. The egg sorter of any of claims 8-13 wherein the area designated for eggs having the class of the egg is one of:an area designated for unfertilized eggs;an area designated for fertilized and male eggs; andan area designated for fertilized and female eggs.U011.0416W01-I l¬ls. A method of sorting an egg comprising:applying spectral data obtained from light from the egg to a three-class classifier to assign the egg to one of three classes with a single classification; andusing the class that the egg is assigned to to control an actuator so as to place the egg with other eggs of the same class as the egg.

16. The method of claim 15 wherein the three classes comprise:unfertilized;fertilized and male; andfertilized and female.

17. The method of any of claims 15 and 16 wherein the three-class classifier is a support vector machine.

18. The method of any of claims 15-17 wherein the spectral data comprises fewer than twenty-six spectral bands.

19. The method of any of claims 15-18 wherein the spectral bands of the spectral data is selected during training of the three-class classifier.

20. The method of any of claims 15-19 wherein the spectral data comprises visible light and shortwave infrared light.

Images