Methods and systems for handling multiple eggs
By training a model to process multiple images of poultry eggs and combining them with pre-determined information, the problem of low efficiency in poultry egg quality detection in existing technologies has been solved, achieving efficient and economical poultry egg quality detection and classification.
Patent Information
- Application Number
- CN202580014955.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Priority Date
- 2024-04-15
- Filing Date
- 2025-04-15
- Publication Date
- 2026-11-10
AI Technical Summary
Existing technologies make it difficult to efficiently and economically determine the quality of multiple eggs in a short period of time, especially the classification and packaging of unfertilized eggs.
By processing multiple images of poultry eggs using a trained learning model and combining them with pre-determined egg-specific information, such as egg age and laying date, a convolutional neural network (CNN) is used for quality estimation, achieving efficient and reliable egg quality detection.
It enables efficient and reliable quality testing and classification of large quantities of poultry eggs in a short time, improving the accuracy and economy of testing, and is suitable for the classification and packaging of poultry eggs.
Smart Images

Figure CN123076306A_ABST
Abstract
Description
[0001] The present invention relates to a method and system for processing multiple poultry eggs (e.g., poultry eggs, especially unfertilized eggs).
[0002] Egg detection systems are known to the applicant and have been marketed. One example is the MOBA egg detector (see www.moba.com), which includes a camera, a dedicated lighting device, and software for detecting leaking and soiled eggs at the feed end of an egg sorting machine.
[0003] Furthermore, WO2010074572 discloses an egg classification method (each egg comprises different parts, namely, eggshell, egg liquid, and air cell), the method comprising: performing light detection on each egg to obtain at least one image of the egg, such that the different parts of the egg can be distinguished in the image; and processing the at least one image to classify the eggs. The volume of egg liquid in the egg can be determined by processing the at least one image. According to a further improvement of this known method, the at least one image is processed using reference values to determine the quality of each part of the egg. According to a particularly advantageous embodiment, the quality of the egg is determined by processing the at least one image and determining the quality of each part (e.g., by determining the quality of the eggshell and the quality of the egg liquid, and using relevant reference values).
[0004] This invention aims to provide an improved egg processing method, particularly a method for egg sorting. One object of this invention is to provide a method that can reliably, economically, accurately, and efficiently determine egg quality, especially ensuring the processing of large quantities of eggs within a relatively short processing time.
[0005] According to one aspect of the invention, this is achieved by the feature described in claim 1.
[0006] This disclosure provides a method for processing multiple eggs (particularly unfertilized eggs), comprising: - Each egg is transported along the conveyor path; - Using illumination light to generate multiple images of the egg from different sides; and A digital image processor uses a trained learning model to process multiple images of each egg to determine its quality and outputs the determined quality, for example, for egg classification. Research has found that this method yields excellent and reliable egg quality estimates when processing a large number of eggs. The determined egg quality can then be used for egg classification / sorting and packaging during the packaging process (e.g., into cartons or pallets). Specifically, this process does not require determining the quality of individual parts of the egg (e.g., shell quality and egg white quality). The process only requires inputting relatively unprocessed images of the eggs (e.g., raw image data or partially processed image data, with the image data for each egg preferably containing its complete outline) into the trained learning model to accurately determine the quality of each egg. Furthermore, this method achieves efficient image processing.
[0007] Advantageously, this method utilizes pre-determined egg-specific information and provides this information to a digital image processor. For example, as described below, a trained learning model can be based in part on such pre-determined egg-specific information (i.e., trained using such information). Research has found that utilizing such pre-determined egg-specific information and providing it to a digital image processor (so that the processor can use this information to determine egg quality) can improve egg quality estimation.
[0008] Various types of predetermined egg-specific information can be applied. According to one embodiment, the predetermined egg-specific information includes at least one of the following: egg age information (e.g., in days or hours) and / or laying date information of the eggs to be processed (i.e., eggs transported along a conveyor path). For example, egg age affects egg weight primarily due to changes in the volume of air inside the egg over time. Furthermore, or alternatively, the predetermined egg-specific information may include egg farm information and / or poultry house information, such as the name, location, and / or (other) identification information of the egg farm and / or poultry house where the eggs (to be processed) were laid. Similarly, according to one embodiment, the predetermined egg-specific information may include poultry-specific information regarding the poultry animals that laid the eggs (to be processed). Such poultry-specific information may, for example, include breed information of the eggs, particularly the breed of animal that laid the eggs to be processed. Furthermore (or alternatively), the poultry-specific information may also include information regarding the age and / or weight of the animals that laid the eggs to be processed, such as the average age and / or average weight of the group of animals that laid these eggs.
[0009] Specifically, predetermined egg-specific information refers to information that is available before egg sorting (i.e. before egg quality is determined), that is, information that is available before the step of processing multiple images of each egg using a trained learning model to determine egg quality.
[0010] For example, predetermined egg-specific information (about a particular egg) can be selected from a group that includes at least the following: 1. Prior data, including one or more of the following: 1a. The age of the egg, or the number of days since it was laid (e.g., 5 days); 1b. The age of the chicken (e.g., 30 weeks); 1c. The corresponding chicken breed (e.g., "Hendrix Genetics Babcock Brown"); 1d. Egg color (e.g., brown; this is an indicator of chicken breed / strain); 1e. Type of feed and / or the content of specific nutrients in the feed given to the respective chickens (e.g., calcium content, which may affect eggshell thickness); 1f. Egg production and / or storage conditions, especially temperature and humidity (which affect the evaporation of the egg liquid); this may include information about different (subsequent) storage locations, such as from the henhouse to the packing station; 1g. Egg colony information, such as egg colony identification code; and 1h. Average egg weight of the previous batch in the same group.
[0011] Alternatively, for example, predetermined egg-specific information (about a particular egg) can be selected from a group that includes at least the following: 2. Sample data, particularly data concerning quality sample values obtained prior to classification, including one or more of the following: 2a. Sample average weight, for example, calculated from the weight of the batch containing the eggs, such as by subtracting the known weight of the packaging material used in the batch from the total weight of the pallets (the pallets carrying the batch); 2b. Eggshell thickness and / or eggshell weight of sampled eggs (e.g., destructive measurements by breaking the egg and measuring the eggshell thickness and / or weighing the eggshell). 2c. Eggshell density and / or eggshell porosity of sampled eggs (e.g., destructive measurements by breaking the egg and measuring the eggshell density and / or eggshell porosity); 2d. Egg white content, yolk content, and / or the ratio of egg white content to yolk content of the sampled eggs; 2e. The protein content of the sampled eggs (e.g., characterized by viscosity, Haugh units, pH value, and / or protein percentage); and 2f. Yolk quality of sampled eggs (e.g., characterized by fat content and / or color).
[0012] Alternatively, for example, predetermined egg-specific information about a particular egg may be selected from a group that includes at least the following: 3. Data derived from automated measurements of other egg characteristics, such as data derived from one or more automated measurements, are used to determine: 3a. The shape of a specific egg (e.g., determined by measuring the length of the egg's long axis); 3b. The degree of soiling of a specific egg; 3c. The degree of cracking in a specific egg (e.g., measured acoustically or visually); 3d. The degree of blood spots on a specific egg; and 3e. Detected egg abnormalities, such as specific eggs being spoiled and / or specific eggs having no yolk or being double-yolk eggs (e.g., determined using spectrophotometry).
[0013] Good results can be obtained if the learning model has been trained via machine learning using training data on multiple eggs (used for training the learning model); this training data includes multiple images of each egg and the measured quality of these eggs. Furthermore, preferably, the training data also includes predetermined egg-specific information about the eggs (used for training the learning model). Those skilled in the art will understand that, in addition to multiple images, this predetermined egg-specific information can simply be added as an additional training parameter to train the model.
[0014] According to a preferred embodiment, the trained learning model includes a neural network, preferably a CNN (Convolutional Neural Network). Research has found that this approach enables optimal real-time data processing, providing sufficiently reliable egg quality detection results and allowing for processing using relatively low-power (and economically inexpensive) image processing devices. In particular, research has shown that egg quality can be measured with high precision, specifically with a standard deviation of 1 gram or less.
[0015] Preferably, a large number of images are taken for each egg for this processing, wherein the images are preferably taken from different perspectives (i.e., each image is taken from a different angle relative to the outer surface of the egg), which has been shown to significantly improve the accuracy of quality estimation. For example, the multiple images may include at least four images of the egg, preferably at least 10 or at least 20 images, such as 25 or more, for example 20-30 images. Preferably, this set of images (i.e., multiple images) for each egg covers / displays the entire outer surface of the egg. The multiple images of each egg can be generated by a single camera, but preferably, at least two cameras (spaced apart from each other) are used to image each egg. The trained learning model can use, for example, raw (i.e., substantially unprocessed) image data provided by each camera, or partially processed image data.
[0016] In addition, at least one type of egg-specific information can be used in the processing, or multiple (i.e. at least two) different types of egg-specific information can be used, which are associated with each egg to be processed.
[0017] One aspect of the invention provides a system for processing multiple eggs (particularly unfertilized eggs), for example, a system configured to implement the method according to the invention. The system includes: - A conveyor used to transport each egg along a conveyor path; - An imaging system having at least one camera and at least one light source for generating multiple images of each egg conveyed by a conveyor, wherein the at least one camera and the at least one light source are positioned on opposite sides of the conveying path; - A digital image processor configured to receive multiple images from an imaging system and process the multiple images of each egg using a trained learning model to determine the quality of the egg, wherein the image processor is configured to output the determined egg quality, for example for generating and / or storing the determined quality associated with a predetermined egg identifier for the corresponding egg, and / or for classifying the eggs.
[0018] According to one aspect, the digital image processor is also configured to determine the quality of an egg using predetermined egg-specific information. The digital image processor may include, for example, a (digital) memory for storing the predetermined egg-specific information for processing. Additionally or optionally, the digital image processor may include an information receiver (e.g., a user interface, editor, input device, etc.) for receiving the predetermined egg-specific information for processing (e.g., for storing the information in the aforementioned memory).
[0019] In this way, the above advantages can be achieved.
[0020] Furthermore, the present invention provides a digital image processor comprising means for performing the method according to the invention, processing multiple images of each egg and using a trained learning model to determine the quality of the egg.
[0021] Furthermore, one aspect of the present invention provides a method for training a machine learning model to predict egg quality, comprising: - Provide a machine learning model that is adapted to receive input data and output predicted data, wherein training includes (in any order): -A) The conveyor transports multiple eggs along the conveying path; -B) The imaging system uses illumination light to generate multiple training images of each egg from different sides (conveyed by a conveyor); -C) Measure the (actual) weight of each egg, for example, using a dedicated egg weight measuring device; and -D) Feed multiple training images of each egg and the egg's measured quality into a machine learning model to train the model to output egg quality as prediction data.
[0022] According to one aspect, methods for training machine learning models to predict egg quality also include: - Provide pre-determined egg-specific information (regarding multiple eggs being transported by the conveyor) and feed that information (e.g., along with multiple training images of each egg and measured egg quality) into a machine learning model to train the model to output egg quality as prediction data.
[0023] Specifically, in addition to training images of eggs, the predetermined egg-specific information is also used as additional parameters in the training of the machine learning model (or as multiple parameters if the information contains multiple types of egg-specific information). These additional parameters can then be used to determine egg quality during egg processing (when using a machine learning model), thereby improving the accuracy of egg quality prediction, as described above.
[0024] Preferably, a training image set of at least 10,000 different eggs and their corresponding measurement qualities are fed into a machine learning model for training. This method has proven to provide good and reliable training data results. For example, a training image set of at least 50,000 different eggs and their corresponding measurement qualities can be input into the machine learning model for training, thereby obtaining a learned model capable of reliable and accurate egg quality determination (during subsequent machine operation). Furthermore, it is preferable to also include predetermined egg-specific information of multiple (preferably all) of the aforementioned at least 10,000 different eggs in the training data.
[0025] Other advantageous embodiments of the invention are given in the dependent claims.
[0026] The invention will now be described in more detail with reference to the accompanying drawings. Wherein: Figure 1 A top view schematically illustrates a non-limiting example of an egg processing system according to an embodiment of the present invention; Figure 2 It shows along Figure 1 Cross-sectional view of centerline II-II; Figure 3 An example of image processing in an embodiment of the present invention is illustrated schematically; Figure 4 schematically depicted Figure 3 Details in Q; and Figure 5 A graph showing the processing results of an embodiment of the present invention is provided.
[0027] In this application, similar or corresponding features are indicated by similar or corresponding reference numerals.
[0028] Figure 1-2 An egg handling system is described, including a conveyor 1 (partially shown) configured to transport multiple eggs E1, E2 along a transport path (in the transport direction T), particularly in the form of multiple rows of eggs E1, E2. This embodiment depicts three rows r1, r2, r3 side-by-side; however, the conveyor 1 can also be configured to transport eggs in more or fewer rows.
[0029] As shown in the figure, eggs may have different shell colors during use. For example, the figure shows that each first egg E1 has a white shell, while each second egg E2 has a non-white shell (e.g., brown). Optionally, this system can visually inspect the two types of eggs E1 and E2.
[0030] Preferably, the conveyor 1 is an endless conveyor, such as an endless roller conveyor 1. It can be configured to rotate / turn the eggs during transport, for example, about their respective longitudinal axes. Specifically, the roller conveyor may include egg support assemblies, such as parallel double-conical (preferably rotatable) rollers 1a, which define grooves between themselves for receiving (and rotating) eggs E1, E2. The egg support elements (e.g., rollers) 1a may be mounted on their respective shafts 1b, which may be driven by suitable drive devices (e.g., motors, belts, or chains, etc., not shown) to move the shafts and rollers along the transport direction T.
[0031] Therefore, the conveyor 1 may include or define egg receiving slots 1c, which are partially open on their respective lower sides (i.e., egg support sides) to allow light to pass through the corresponding egg support elements (rollers 1a in this example).
[0032] The system also includes imaging systems 2 and 3, which have at least one light source 2 and at least one camera 3 for taking multiple (N) images of each egg E.
[0033] At least one or each of the light sources 2 may be configured to emit a corresponding illumination beam B (e.g., near-infrared (NIR) light) toward the egg transport path to illuminate eggs E1, E2 during operation. Alternatively, at least one or each of the light sources 2 may be configured to emit white light (e.g., light that can be reflected by the egg) or blue light (blue light is particularly advantageous when imaging the air cell of white-shelled eggs).
[0034] Each light source 2 (one shown in the figure) can be positioned below the vertical height of the egg conveying path (see Figure 1). Figure 2However, this is not necessary. In this embodiment, the light source 2 is arranged to emit an upward beam, so that the eggs E1 and E2 passing through the corresponding row r2 are subsequently illuminated by the beam B (in this embodiment, the beam enters the corresponding slot through the open side of each egg receiving slot).
[0035] Alternatively, the light source 2 can be arranged to emit beams downward or to the side so as to illuminate the eggs E1, E2 passing through the corresponding row r2 with the corresponding beam B.
[0036] The system may include a varying number of light sources 2, each of which can be configured to emit one or more light beams B, for example, to illuminate eggs passing through one or more conveyor rows r1, r2, r3. In a non-limiting embodiment, the light source includes one or more light-emitting diodes (LEDs) for emitting the light beam B.
[0037] The system may include a varying number of light sources 2, each of which can be configured to emit one or more light beams B, for example, to illuminate eggs passing through one or more conveyor rows r1, r2, r3. In a non-limiting embodiment, the light source includes one or more light-emitting diodes (LEDs) for emitting the light beam B.
[0038] Optionally, each light source 2 can be configured to emit or provide a collimated or focused beam B, specifically illuminating only a portion of the outer surface of the eggshell of the passing egg (E1, E2). During operation, the light B from the light source 2 can be scattered inside the egg E, so that the entire outer contour of the egg can receive light (i.e. be illuminated), thereby enabling imaging by the camera of the imaging system.
[0039] The wavelength of the near-infrared light (if present) in the illumination beam B generated by the light source 2 may be at least 700 nm, preferably not more than 1000 nm; for example, the wavelength is in the range of 700-800 nm, preferably less than 750 nm, and particularly about 720 nm. The wavelength of this near-infrared light is most preferably in the range of 710-750 nm, and more preferably in the range of 710-730 nm.
[0040] If white light is emitted, light source 2 can be a broadband white light emitter.
[0041] If emitting blue light, light source 2 can be a broadband or narrowband blue light emitter.
[0042] Furthermore, preferably, the system is configured such that (e.g., the conveyor speed and egg rotation speed are set during operation) each egg E1, E2 rotates about its respective longitudinal axis when irradiated by the beam B.
[0043] In the attached figure ( Figure 2In this embodiment, the light source 2 is depicted as emitting a light beam toward the detector 3 (see below), but this is not a necessary condition (especially when the egg internally diffuses the received light). The light source 2 can operate continuously, but it is preferred to emit the light beam intermittently. Furthermore, in one embodiment, the operation of the light source 2 can be synchronized with the conveyor 1 (e.g., with the conveying speed) so that the light source 2 only generates the light beam B for illumination when an egg passes by, and does not generate the light beam under other circumstances (e.g., to save energy and / or to avoid the detector 3 on the opposite side being directly illuminated by the light source 2).
[0044] Furthermore, the imaging system includes multiple light detectors 3 (particularly cameras) for detecting light emitted from the egg transport path, such as light that passes through eggs E1 and E2 during operation (and is emitted and / or diffused by the eggs), and / or light that does not pass through the eggs but is reflected from the egg surface. During operation, the camera 3 preferably uses illumination light B (e.g., light passing through egg E) to generate multiple images of the egg from different sides of egg E.
[0045] In this embodiment, camera 3 is located at a vertical height above egg conveyor 1. Therefore, contamination of detector 3 (e.g., by dirt or other substances that may be present on the passing eggs) can be prevented or significantly reduced. Alternatively, one or more (e.g., each) cameras 3 may be located at approximately the same height as the passing eggs, and / or, for example, at a vertical height below the height of the passing eggs. In other words, alternatively, one or more cameras 3 may be located at other heights (e.g., at or below the egg conveyor height), and / or at different locations.
[0046] In this embodiment, pairs of cameras 3a and 3b are associated with each conveyor row r1, r2, r3 defined by the conveyor 1, and for example, with one of the light sources 2. Specifically, for each conveyor row, from a top view, at least cameras 3a and 3b are arranged adjacent to each other to capture images of the eggs from their respective different perspectives. Alternatively, for example, cameras 3 can be installed to detect light emitted from multiple rows of eggs r1, r2, r3.
[0047] Good results can be obtained if cameras 3a and 3b are configured to generate grayscale images processed by image processor 8. Each image fed to image processor 8 can be, for example, a bitmap image (e.g., an 8-bit bitmap image) with a resolution of up to 1000×1000 pixels (specifically up to 800×800 pixels). Studies have found that such relatively low-resolution images can provide good and reliable quality prediction results. Furthermore, according to one embodiment, each camera 3a and 3b can generate higher-resolution images, wherein image processor 8 can be configured to resize and / or crop such higher-resolution images to make them relatively low-resolution images (up to 1000×1000 pixels) for subsequent processing (via the trained learning model 1001 discussed below).
[0048] Each camera 3a, 3b can be configured to generate a detection signal, specifically a digital image of the egg. In a preferred embodiment, each image of each egg (taken by cameras 3a, 3b) includes the entire illuminated outline of the egg (see...). Figure 3 ).
[0049] Optionally, the system can be configured to maintain substantially the same egg illumination conditions / parameters, for example, when processing eggs E1 and E2 with different shell colors, but this is not necessary. Therefore, optionally, the light source can produce the same illumination beam B (with the same or constant beam intensity and the same spectrum) for illuminating eggs E1 and E2. Furthermore, preferably, the corresponding detector 3 is not adjusted during operation (regardless of whether the detector is detecting light from the first or second egg) to detect light from multiple eggs E1 and E2 (located in the same passing rows r1, r2, r3).
[0050] Furthermore, the system includes a processing unit 8 (shown schematically) configured to process the light detection results of the infrared photodetector 3. The processing unit 8 can be configured in various ways and may include, for example, processor software, processor hardware, a computer, a data processing device, a memory 8a for storing the data to be processed, etc. Additionally, the processing unit 8 may include various corresponding communication devices or be connected to these devices to allow communication with the photodetector 3 and receiving its detection results; alternatively, the processing unit 8 and the detector 3 may be integrated with each other. Furthermore, the processing unit 8 may include, for example, a user interface 8b to allow an operator to interact with the central processing unit, for example, for outputting data processed by the processor, and for example, for inputting data (particularly predetermined egg-specific information) to the processing unit 8.
[0051] Specifically, the processing device is a digital image processor 8, configured to receive multiple (N) images from imaging systems 2 and 3, and process the multiple images of each egg E using a trained learning model 1001 to estimate the quality of the egg. Preferably, imaging systems 2 and 3 are configured to take at least 4 images (N=4) of each passing egg, preferably at least 10 or 20 images (e.g., 25 or more images), for processing by the image processor 8 (using the trained learning model 1001).
[0052] Furthermore, the image processor 8 is configured to output the determined egg quality ME, for example, to generate and / or store the determined quality ME associated with a predetermined egg identifier (e.g., a unique code or number) of the corresponding egg, and / or to classify the eggs. This output may be stored, for example, in a memory 8a and / or a server, and / or provided to an operator via a user interface 8b or similar means, as will be clear to those skilled in the art.
[0053] According to a preferred embodiment, the processing device (processor) 8 is configured to additionally utilize predetermined egg-specific information to determine the quality of the egg. Various examples of such predetermined egg-specific information have been described above: this information may include one or more of the following: information about the egg itself and / or information about the poultry that laid the egg. Specifically, the predetermined egg-specific information is not generated by the imaging systems 2, 3 (i.e., the information is not an image of the egg). The predetermined egg-specific information of the egg (to be processed) may be or include one or more of the following: laying date, laying time, egg age, laying location, laying address, poultry information (e.g., the poultry weight, color, age, and breed of the animal that laid the egg), and / or other types of information about the egg. Other examples of predetermined egg-specific information have been mentioned above.
[0054] The processing device preferably includes an information receiver 8b for receiving predetermined egg-specific information. The information receiver 8b (in...) Figure 2 The information receiver 8b (illustrated by a dashed box) may be a component of the processor 8 and / or a remote information receiver, such as an information receiver accessible by the processor 8 through a suitable communication network (e.g., LAN, WiFi, computer network, Internet, and / or similar network), a reader for reading a data carrier DC containing the information, and / or similar devices known to those skilled in the art. For example, the information receiver 8b may be or include a user interface, a user-accessible editor, and / or similar devices configured to receive and / or input predetermined specific information.
[0055] The processing device 8 preferably includes at least the aforementioned memory 8a for storing predetermined egg-specific information (received by the information receiver 8b). It should be noted that such a memory (in...) Figure 2 The memory (indicated by a dashed box) may be a component of the processor 8 and / or a remote memory, such as memory that the processor 8 can access via a suitable communication network (such as a local area network, WiFi, computer network, Internet and / or similar network), as will be clear to those skilled in the art.
[0056] Predetermined egg-specific information about the multiple eggs to be processed (e.g., a batch of eggs fed to the system during operation, particularly via the information receiver 8b) can be provided, for example, in various ways and formats (e.g., digital data files, text files, spreadsheets) and can be read by the digital image processor 8 (as will be understood by those skilled in the art).
[0057] It should be noted that, during operation, it is preferable to provide predetermined egg-specific information for each egg supplied, but this is not mandatory. Alternatively, for a large number of eggs supplied, such information may not be available; in this case, the digital image processor 8 is preferably configured to estimate the quality of each egg without using such predetermined egg-specific information (i.e., based solely on the image).
[0058] As described above, conveyor 1 is configured to rotate egg E during transport along the conveyor path, particularly as the egg passes through imaging systems 2 and 3. In this way, imaging systems 2 and 3 can generate multiple images of each egg E from different perspectives (i.e., relative to the outer surface of the egg). Preferably, conveyor 1 and imaging systems 2 and 3 are configured to cooperate (e.g., by the imaging systems capturing multiple N images during a predetermined egg rotation provided by conveyor 1) such that the corresponding multiple images of egg E cover the entire outer surface (i.e., all sides) of egg E. Two (or more) cameras 3a and 3b associated with one of the conveyor paths r1, r2, and r3 can, for example, simultaneously capture their respective egg images. Furthermore, the time interval between two consecutive images of each egg E captured by each camera 3a and 3b can be constant (particularly when conveyor 1 provides a constant egg rotation speed).
[0059] Preferably, the learning model 1001 has been trained using machine learning, and the training data includes multiple images of multiple eggs and the determined (actual) mass MM of these eggs. The determined (actual) egg mass MM used for model training can be determined, for example, by a known egg weighing device (e.g., a scale), either manually or using an automated egg weighing system.
[0060] According to a preferred embodiment, the training data also includes predetermined specific information about multiple eggs, such as (but not limited to) egg age and / or laying date and / or egg farm and / or poultry house information and / or poultry-specific information. This will be described in more detail below.
[0061] The trained learning model 1001 preferably includes a neural network, preferably a convolutional neural network (CNNs are known in themselves, see example). https: / / en.wikipedia.org / wiki / Convolutional_neural_network ). Figure 3 and Figure 4 An example of model 1001 is shown, which will be described in more detail below.
[0062] For example, the initial operation of the system may involve a method for training a machine learning model 1001 to predict the quality of poultry eggs.
[0063] The learning process of the learning model 1001 (e.g., CNN) can be a process of determining the values of all learning model parameters within the learning model network, such that the resulting value produced by the input image (of the egg) is highly correlated with the actual weight of the egg (to be measured). According to one aspect, at least one learning parameter includes the aforementioned pre-determined egg-specific information.
[0064] This training method may include at least the following steps: Provide machine learning model 1001, which is suitable for receiving input data and outputting predicted data, wherein the training process includes: -A) Conveyor 1 transports multiple eggs E along conveying paths r1, r2, and r3; -B) Imaging systems 2 and 3 use illumination light B to generate a set (i.e., multiple N images) of each egg E (conveyed by conveyor 1 and rotated, for example) from different sides of the egg. -C) Measure the (actual) mass MM of each egg E; and -D) Feed multiple training images of each egg E and the measured quality MM of the egg E into the machine learning model to train the learning model to output the egg quality ME as prediction data.
[0065] Furthermore, preferably, step D) also includes, in addition to (e.g., in combination with) multiple training images of each egg E and the measured quality MM of egg E, feeding predetermined egg-specific information into the input machine learning model to train the learning model and output egg quality as prediction data. It should be noted that the predetermined egg-specific information and multiple images can be fed into the learning model simultaneously, but this is not mandatory and depends on the learning model used, as will be clear to those skilled in the art.
[0066] Here, it is preferable to process a relatively large number of eggs for training, wherein at least 10,000 training images of different eggs and their corresponding measurement quality are preferably fed into the machine learning model to train the learning model. Furthermore, as previously mentioned, the set of N training images for each egg input into the learning model 1001 preferably includes at least 4 images of that egg, more preferably at least 10 or 20 images (e.g., 25 or more, or in the range of 20 to 30 images).
[0067] After training model 1001 is complete, the corresponding system can execute a method for processing egg E. This method includes: - Transport each egg E along the transport paths r1, r2, r3; - Using illumination light B (e.g., light transmitted through egg E), generate multiple images of each egg E (conveyed by a conveyor and preferably rotated) from different sides; and - The digital image processor 8 uses a trained learning model to process multiple images of each egg E to determine the quality ME of the egg and outputs the determined egg quality ME, for example, for classifying eggs.
[0068] Furthermore, the aforementioned predetermined egg-specific information about eggs E (e.g., all eggs E) can be fed to the system, particularly through information receiver 8b. Subsequently, the corresponding egg processing method (i.e., processor 8) can utilize this predetermined egg-specific information to determine the egg quality ME, in addition to (e.g., in combination with) the generated multiple egg images.
[0069] As described above, during the egg processing operation of the system, multiple images of each egg E cover the entire outer surface of the egg. Preferably, during the generation of N images of egg E, each egg E is rotated. Preferably, the number N of images per egg during the training of the learning model 1001 is the same as the number N of images per egg during the subsequent normal operation of the system using the trained learning model.
[0070] Furthermore, the trained learning model 1001 can use substantially raw image data provided by at least one camera 3. Here, raw image data specifically refers to images generated by the camera 3 that have not undergone substantial image processing before being fed to the machine learning model. Before feeding the (cropped) images to the machine learning model, some image processing, such as cropping, can be applied to extract the portion of the image containing the complete outline of the imaged egg, while removing other portions of the image that do not contain the egg. Additionally, image processing may include resizing or reducing the image resolution before feeding the images to the trained learning model 1001. The raw image data for each (optionally cropped or resized) image is preferably a low-resolution grayscale image, such as an image with a maximum of 1000 × 1000 pixels (e.g., less than 900 × 900 pixels).
[0071] Figure 3 and Figure 4 An example of a learning model architecture is shown, along with the corresponding steps that can be performed by the learning model 1001, particularly a CNN (e.g., by the system's image processor 8).
[0072] Figure 3 Multiple images Px (P1, P2, P3, P4, ..., PN) of a single egg taken from different perspectives by imaging systems 2 and 3 are shown. A total of N images were taken for each egg E. These N images are preferably grayscale bitmap images (as described above), and preferably low-resolution images.
[0073] also, Figure 3 The input of predetermined egg-specific information K about egg E is also shown. As previously mentioned, this information K can contain various types of information, such as the age of the egg (as shown in the figure) and / or many other types of information. It should be understood that the predetermined egg-specific information K can be input into different parts of the learning model 1001 (e.g., the first layer or intermediate layers) (depending on the type of model used, etc.).
[0074] During the operation, each of the N images Px and the corresponding egg-specific information K (if any) is fed into the learning model 1001 (executed by the image processor 8) for processing to estimate the egg quality EM(Px, K) associated with that image Px. After determining the egg quality EM(Px, K) for all N images of the egg E, the image processor 8 can calculate the average egg quality EM (i.e., EM equals the sum of the determined N egg qualities EM(Px) divided by N) and output this quality EM as the estimated egg quality of the egg E.
[0075] Figure 4An example of the learning model 1001 described above is shown, specifically illustrating the process of processing one of multiple egg images Px (to estimate the corresponding egg quality EM(Px, K)). The learning model 1001 preferably includes several fully connected layers 1003, 1004, 1005 (i.e., layers where each input pixel is multiplied by a parameter value), such as neural network 1002 (see [link to example]). Figure 4 These fully connected layers include an input layer 1003, at least one intermediate layer 1004, and an output layer 1005, each layer having multiple nodes. For example, the input layer may have multiple nodes 1003, to which data from the pooling layer 1006 (see below) is input. By way of a non-limiting example, those skilled in the art will understand that the aforementioned predetermined egg-specific information K is also input to the input layer 1003.
[0076] The intermediate layer may contain multiple nodes 1004, each node performing calculations on data input from various nodes in the input layer 1003 using parameters. The multiple nodes 1004 output data to multiple nodes 1005 in the output layer. The nodes 1005 in the output layer include at least one node for outputting the determined (estimated) egg quality EM(Px, K). In one embodiment, only a single output node 1005 exists for outputting the determined egg quality EM(Px, K).
[0077] The pooling layer 1006 of the CNN is part of a set of layers comprising M convolutional layers 1006(1, ..., M) and M associated pooling layers 1007(1, ..., M). Each convolutional layer 1006(1, ..., M) is configured to apply a filter matrix (convolution) to the image received from its preceding layer. Each pooling layer is configured to reduce the number of pixels (for the image received from the preceding convolutional layer) by taking the maximum value within the image region. The first convolutional layer 1006(1) receives the image Px generated by imaging systems 2 and 3.
[0078] Specifically, during operation, each convolutional layer 1006 can convolve its input (pixels) using a filter matrix of a specific dimension. Each value in the matrix is a parameter that can be "tuned" by a learning algorithm (during model training). For example, it is well known that if a grayscale image of size 512×512×1 (1 being the number of channels) is used as input, then using a 3×3 kernel containing 32 filters and biases will produce 1 3 3 32 + 32 = 320 parameters. Without considering bias, the calculation result is 1. 3 3 32 = 288.A.
[0079] According to one implementation, the learning model 1001 contains fewer than 2 million parameters in total (including convolutional and fully connected layers), for example, only 1.6 million parameters. Therefore, even with a relatively low-cost digital image processor 8, egg images can be processed in real time.
[0080] Research has found that this method can efficiently estimate egg quality without (for example) using complex 3D rendering techniques to determine the quality of individual egg components (such as the shell and yolk). Furthermore, because it uses a relatively simple learning algorithm, the algorithm can run on relatively low-cost image processing hardware.
[0081] Figure 5 The results of this system and method are presented, showing a comparison between the estimated mass EM1 (estimated by this system without utilizing additional pre-determined egg-specific information) and the actual measured mass MM for several types of eggs. These different types of eggs include several brown-shelled eggs (B5, B10, B15, B20) and several white-shelled eggs (W5, W10, W15, W20), with the difference between the different eggs being the storage time of the eggs (i.e., egg age). The results show that the estimated mass EM1 matches the actual (measured) mass MM to a surprisingly good degree.
[0082] The study found that the results were even better if additional parameters involving predetermined egg-specific information (as described above) were incorporated into the processing. Figure 5 This is also reflected in the fact that the numeral EM2 indicates the egg quality estimated using the following two additional parameters involving predetermined egg-specific information: (1) the number of days after laying the egg (i.e., the egg age in days), and (2) the breed of chicken (i.e., the breed of chicken that laid the egg).
[0083] Unwilling to be bound by any theoretical constraints, this system and method are designed to handle relatively simple grayscale bitmap egg images. These images show the overall shape of the egg, as well as features and dimensions within the egg that may be related to the air cell, and can be processed in conjunction with eggshell information. The learning model automatically considers these egg-related features during the learning process, and the trained model can then provide good egg quality predictions. Utilizing pre-defined specific egg information can further improve the accuracy of the egg quality prediction results (see...). Figure 5 ).
[0084] It is evident that the present invention is not limited to the exemplary embodiments described above. Various modifications can be made within the scope of the invention as defined by the appended claims.
[0085] In this application, the eggs to be processed specifically refer to non-living eggs, dead eggs, that is, unfertilized (and without embryos) edible eggs. Such eggs can be poultry eggs, such as chicken eggs.
[0086] Furthermore, in a preferred embodiment, the detector that detects (transmitted) near-infrared light maintains a predetermined light detection state when detecting light from eggs with different shell colors, without interfering with the detection results (and subsequent processing results). However, this is not mandatory. Alternatively, for example, at least a portion of the detector can change its state accordingly for detecting emitted light from various (e.g., different) eggs.
[0087] Furthermore, each camera can obviously be located at a different position and can be configured to detect, for example, light transmitted from a single egg or light transmitted from multiple eggs. Alternatively, at least one or each camera can be configured to detect illumination light that does not pass through the egg but only illuminates the outer surface of the egg, such as light reflected by the egg. Similarly, at least one or each camera can be configured to simultaneously detect transmitted illumination light (light that has passed through the egg) and non-transmitted illumination light (i.e., light that has not passed through the egg).
[0088] For example, the system may include multiple cameras for observing eggs from different viewing directions. For instance, at least one camera-type detector may be configured, with each camera arranged to simultaneously capture images of one or more eggs to be detected. In this case, for example, the images captured by the cameras may be segmented or cropped into multiple images associated with different eggs, and these images may be fed as egg images to an image processor.
[0089] Furthermore, each of the light sources can be positioned at different locations, such as below, at, and / or above the vertical height of the egg. Moreover, multiple light sources can be provided to illuminate individual eggs E1 and E2 from the same or different directions (e.g., simultaneously).
[0090] Furthermore, for example, the color of the eggshell can be a color perceptible to the naked eye (human eye), which is understandable to those skilled in the art.
[0091] Furthermore, the method according to the exemplary embodiments described above can be implemented by a digital image processor executing instructions (including program instructions) stored in a non-transitory computer-readable medium. The medium can be used alone or in combination with program instructions, and may also include data files, data structures, etc. The program instructions recorded on the medium may be instructions specifically designed and constructed for the exemplary embodiments, or instructions well-known and available to those skilled in the art of computer software. Examples of non-transitory computer-readable media include: magnetic media, such as hard disks, floppy disks, and magnetic tapes; optical media, such as CD-ROMs, DVDs, and / or Blu-ray discs; magneto-optical media, such as optical discs; and hardware devices specifically configured for storing and executing program instructions, such as read-only memory (ROM), random access memory (RAM), flash memory (e.g., USB flash drives, memory cards, memory sticks, etc.). Examples of program instructions include, for example, machine code generated by a compiler and files containing high-level code that can be executed by a computer using an interpreter. The aforementioned devices can be configured as one or more software modules to perform the operations of the exemplary embodiments described above, and vice versa.
[0092] Software may include computer programs, code segments, instructions, or combinations thereof, used independently or collectively to instruct or configure a digital image processor to operate as intended. Software and data may be stored permanently or temporarily in any type of machine, component, physical or virtual device, computer storage medium, or device, or in propagated signal waves capable of providing instructions or data to or being interpreted by the image processor. Software may also be distributed across network-coupled computer systems for distributed storage and execution. Software and data may be stored in one or more non-transitory computer-readable recording media. Non-transitory computer-readable recording media may include any data storage device that stores data that can subsequently be read by a computer system or processing device.
[0093] While this disclosure contains specific exemplary embodiments, those skilled in the art will understand that various changes to the form and details of these exemplary embodiments may be made without departing from the scope of the claims. The exemplary embodiments described herein are for illustrative purposes only and are not intended to limit the invention.
[0094] For example, it is preferable to use a large number of N images for each egg, where the images can be taken sequentially from two different angles as the egg rotates, but images from a single angle (i.e., taken by a single camera) or images from three or more angles can also be used.
[0095] In addition, the method may include using a standard convolutional neural network 1001, for example, combined with Bayesian optimization for hyperparameter search.
[0096] In addition, illumination light of various wavelengths or wavelength ranges can be used. Optionally, near-infrared (NIR) light can be used as illumination light. Alternatively or additionally, white light (e.g., wavelength range of about 400 to 700 nm) and / or blue light (with a single wavelength or wavelength range in the range of about 380 to 500 nm) can also be used.
[0097] Furthermore, it should be understood that the learning model 1001 may include multiple fully connected layers 1003, 1004, 1005 (see example...) Figure 4 ), and multiple ReLU (Modified Linear Unit) activation layers (which can be part of all fully connected modules and all convolutional modules; here, "module" can be defined as a collection of combined layers, for example: fully connected module: fully connected -> ReLU; convolutional module: convolution -> batch normalization (BatchNorm) -> ReLU -> two-dimensional average pooling (AveragePool2D)).
[0098] It will be clear to those skilled in the art that the aforementioned predetermined egg-specific information specifically refers to information determined before the camera of the imaging system images the corresponding egg E. There are various ways to determine or provide this predetermined egg-specific information, such as: input by operators or staff of the egg production facility (e.g., an egg farm or its poultry houses) through the information receiver 8b; and / or automatic generation by the egg-specific information generator of the corresponding egg facility (e.g., the generator could be a date / time generation system that automatically generates egg-laying date or time information, the generated information of which can be automatically provided to the information receiver 8b for processing these eggs); and / or by other means.
[0099] According to one implementation, egg-specific information can be stored in an information carrier DC (schematically shown). Figure 2 The information carrier may be on or provided by it, and the information carrier may be, for example, an analog or digital information carrier.
[0100] According to one implementation, egg-specific information may be stored and / or provided by an electronically readable information carrier (DC), such as a digital information carrier, a readable code, a QR code, a barcode, a data matrix code, or an RFID tag.
[0101] According to one embodiment, when supplying multiple (e.g., a batch) eggs to be processed, the corresponding information carrier DC can be attached or coupled to a carrier that carries or contains the multiple eggs (the carrier is, for example, a crate, a tray, an egg box, an egg holder, or a combination of such carrier structures).
[0102] For example, according to one embodiment, the information carrier can be read during egg processing to obtain corresponding predetermined egg-specific information, which is then used by a digital image processor to determine the quality of the egg. For this purpose, the system may include an information receiver 8b, which includes a reading device for reading the information carrier. Alternatively or additionally, an operator may assist in reading the information carrier (e.g., using a manually operated reader).
[0103] If the egg-specific information pertains to the weight of an egg batch, the system may include a weighing device for weighing the supplied batch of eggs to be processed (e.g., a pallet weighing device in cases where the batch includes one or more pallets containing egg containers such as egg trays or egg boxes).
Claims
1. A method for processing multiple poultry eggs, particularly unfertilized eggs, comprising: - Each egg (E) is transported along the conveyor path; - Using illumination light (B), generate multiple images of the egg from different sides of each egg (E); as well as - A digital image processor (8) processes multiple images of each egg (E) using a trained learning model to determine the quality of the egg and outputs the determined quality of the egg, for example, for classifying the egg. The method utilizes predetermined egg-specific information, which is provided to the digital image processor (8).
2. The method of claim 1, wherein the learning model has been trained by machine learning using training data involving multiple eggs, the training data including multiple images of each of the multiple eggs and determined egg quality, the training data also including predetermined egg-specific information about the eggs.
3. The method according to any one of the preceding claims, wherein the predetermined egg-specific information includes at least one of egg age information and egg-laying date information.
4. The method according to any one of the preceding claims, wherein the predetermined egg-specific information includes egg farm information and / or poultry house information.
5. The method according to any one of the preceding claims, wherein the predetermined egg-specific information includes poultry-specific information.
6. The method according to any one of the preceding claims, wherein the trained learning model comprises a neural network, preferably a CNN.
7. The method according to any one of the preceding claims, wherein multiple images of each egg (E) cover the entire outer surface of the egg.
8. The method according to any one of the preceding claims, wherein the egg is rotated during the generation of multiple images of each egg (E).
9. The method according to any one of the preceding claims, wherein the plurality of images comprises at least four images of the egg, preferably at least ten or at least twenty images, such as 25 or more images, for example in the range of 20 to 30 images.
10. The method according to any one of the preceding claims, wherein each of the images is a grayscale image.
11. The method according to any one of the preceding claims, wherein each of the images is a bitmap image, such as an 8-bit bitmap image, for example having a resolution of up to 1000 × 1000 pixels.
12. The method according to any one of the preceding claims, wherein multiple images of each egg are generated by at least one camera (3), and the trained learning model uses substantially raw image data provided by the at least one camera (3).
13. The method according to any one of the preceding claims, wherein the egg is imaged using infrared light (B) (preferably near-infrared light), wherein the wavelength of the light (B) is preferably less than 800 nm, for example less than 750 nm, for example in the range of 710 to 750 nm, preferably in the range of 710 to 730 nm.
14. The method according to any one of the preceding claims, wherein white light (B) is used to image the egg.
15. The method according to any one of the preceding claims, wherein blue light (B) is used to image the egg.
16. A system for processing multiple eggs, particularly unfertilized eggs, wherein the system is configured to perform the method of any one of the preceding claims, the system comprising: - A conveyor (1) that transports each egg (E) along a conveyor path; - An imaging system having at least one camera (3) and at least one light source (2) for generating multiple images of each egg (E) conveyed by the conveyor (1), wherein the at least one camera (3) and the at least one light source (2) are arranged on opposite sides of the conveying path; - A digital image processor (8) configured to receive multiple images from the imaging system and process the multiple images of each egg (E) using a trained learning model to determine the quality of the egg, wherein the image processor is configured to output the determined egg quality, for example, to generate and / or store the determined quality in association with a predetermined egg identifier for the corresponding egg, and / or to classify the eggs. The digital image processor (8) is configured to additionally utilize predetermined egg-specific information to determine the quality of the egg, wherein the digital image processor (8) preferably includes at least one of a memory (8a) for storing the predetermined egg-specific information and an information receiver (8b) for receiving the predetermined egg-specific information.
17. The system of claim 16, wherein the conveyor is configured to rotate the egg during transport along the conveying path, particularly as the egg passes the imaging systems (2, 3).
18. The system of claim 16 or 17, wherein the imaging system comprises at least two cameras (3a, 3b) arranged to capture images of the egg (E) passing along the transport path from mutually different directions.
19. The system according to any one of claims 16 to 18, wherein the conveyor (1) and the imaging system (2, 3) are configured to cooperate with each other such that the corresponding plurality of images of the egg (E) include the entire outer surface of the egg (E).
20. The system according to any one of claims 16 to 19, wherein the imaging system is configured to capture at least 4, and preferably at least 10 or at least 20, such as 25 or more, images of the passing egg for processing by the image processor (8).
21. The system according to any one of claims 16 to 20, wherein the learning model has been trained by machine learning, and the training data used includes multiple images of multiple eggs and the determined quality of the eggs; wherein the training data also includes predetermined egg-specific information about the multiple eggs, such as egg age and / or laying date and / or egg farm information and / or poultry house information and / or poultry-specific information, wherein the trained learning model preferably includes a neural network, preferably a CNN.
22. A digital image processor comprising means for processing multiple images of each egg (E) using a trained learning model to determine the quality of the egg, for implementing the method of any one of claims 1-15.
23. A method for training a machine learning model to predict the quality of poultry eggs, comprising: A machine learning model is provided, the machine learning model being adapted to receive input data and output prediction data, wherein training includes: -A) Conveyor (1), which is used to transport each egg (E) along the conveyor path; -B) Imaging system (2,3) for generating multiple training images of the egg (E) from different sides of each egg using illumination light (B); -C) Measure the weight of each egg; and -D) Input multiple training images of each egg and the measured egg quality into the machine learning model to train the machine learning model to output the egg quality as prediction data. Preferably, at least 10,000 training images of different eggs and the quality of corresponding measurements are fed into the machine learning model to train the machine learning model. The method further includes: - Provide predetermined egg-specific information and feed the egg-specific information (e.g., along with multiple training images of each egg and measured egg quality) into the machine learning model to train the machine learning model to output the egg quality as prediction data.
24. The method of claim 23, wherein the multiple training images for each egg include at least four images of the egg, preferably including at least 10 or at least 20 images, such as 25 or more images, for example in the range of 20-30 images.
25. The method according to any one of claims 23-24, wherein the trained machine learning model comprises a neural network, preferably a CNN.
Citation Information
Patent Citations
Method and apparatus for classifying eggs
WO2010074572A1
