Egg sorting device, egg sorting method, and egg grading and packaging system equipped with the egg sorting device

Abstract

To provide an egg classification device which classifies eggs according to an appearance state of the eggs after being washed using a learning model created by an intelligent information technology using the teacher data including a reflected image obtained by labeling the appearance state of the eggs after being washed in a predetermined size region.SOLUTION: An egg classification device 1 includes a storage part 2 which stores an egg classification learning model created by an intelligent information technology using the teacher data including a reflected image labeled in a predetermined size region corresponding to an appearance state of the eggs; and an egg classification part 3 for classifying the eggs according to the appearance state of the eggs by inputting the reflected image obtained by imaging the eggs into the egg classification learning model as the input data.SELECTED DRAWING: Figure 1C

Description

[Technical Field]

[0001] The present invention relates to an egg sorting device, an egg sorting method, and an egg grading and packaging system including the egg sorting device. [Background technology]

[0002] For example, in a typical egg processing process, eggs brought in from a chicken coop are washed, dried, aligned, inspected, weighed, and then sorted and packaged according to grade. Eggs are transported on a double-row transport conveyor, but if there are any broken eggs with partially missing shells among them, the liquid eggs that leak out will contaminate the transport conveyor and surrounding eggs, so it is desirable to remove them as soon as possible. Generally, broken eggs are removed by visual inspection upstream of the drying device, and if the conveyor is heavily contaminated, the conveyor must be stopped, the broken eggs that are the cause of the contamination and the surrounding contaminated eggs must be removed, and the conveyor must be cleaned, which is the main cause of reduced operation rates. On the other hand, dirty eggs with large amounts of feces and other substances attached to the eggshell surface are unhygienic, and extremely large eggs and deformed eggs may interfere with the movement of eggs being transported in other rows, causing them to break or crack, so it is desirable to remove them from the conveyor as soon as possible. In other words, (1) broken eggs are easily contaminated with bacteria and should be discarded, and (2) dirty eggs, very large eggs, and deformed eggs are desired for use as processed eggs. Therefore, it is desirable to separate and detect (1) broken eggs and (2) dirty eggs, very large eggs, and deformed eggs.

[0003] Patent Document 1 discloses a non-destructive inspection of brown chicken eggs using transmitted light, which utilizes a calculation unit with a hierarchical neural network structure.

[0004] Patent Document 2 discloses an egg sorting device and an egg sorting system that utilize a learning model. The egg sorting system includes a conveying device, a washer, a broken egg detection device, a dryer, an egg image analyzer, a size measuring device, an impact analysis device, a sterilizer, a movement amount detection device, a weighing device, a distributor, a transmitted light analysis device, and an in-container image analysis device. The broken egg detection device is installed between the washer and the dryer and includes a calculation unit and an analysis unit with a learning model. The eggs are illuminated from below, and images captured of the transmitted light passing through the eggs are input into the learning model to determine whether or not the eggshell is damaged and the extent of the damage. The learning model is trained in advance using training data including information describing the condition of each egg and image data representing the captured image of each egg. The document also describes learning models that use images of transmitted light, reflected light, and fluorescent light. The document also describes determining whether or not the eggshell is damaged and the extent of the damage based on image data based on predetermined rules, and then performing a determination using the learning model. It also describes that the number of eggs without shell damage, the number of eggs with shell damage, or the number of eggs with shell damage falling within a predetermined range can be displayed. The egg image analysis device, located downstream of the dryer, includes an analysis unit having a calculation unit and a learning model. The calculation unit inputs images into the learning model and determines the eggshell color, eggshell color uniformity, presence or absence of eggshell stains, the level of staining, the type of staining, presence or absence of eggshell damage, the level of damage, the eggshell surface texture, the eggshell pattern, or any imprinting on the eggshell. The learning model is trained in advance to determine the eggshell color, color uniformity, presence or absence of eggshell stains, the level of staining, the type of staining, presence or absence of eggshell damage, the level of damage, the presence or absence of eggshell malformations, the type of malformations, the eggshell surface texture, the eggshell pattern, or any imprinting on the eggshell from the image data, using training data including image data representing photographed eggs and information representing the eggshell color, color uniformity, presence or absence of eggshell stains, the level of staining, the type of staining, presence or absence of eggshell damage, the level of damage, the presence or absence of eggshell malformations, the type of malformations, the eggshell surface texture, the eggshell pattern, or any imprinting on the eggshell. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Patent No. 3333472 [Patent Document 2] Japanese Patent Application Publication No. 2020-148621 Summary of the Invention [Problem to be solved by the invention]

[0006] As described above, in egg grading and packaging, there is a desire to remove as many broken and dirty eggs as possible after cleaning to remove as much dirt as possible and before sending the eggs to processing steps downstream of the cleaning. Patent Documents 1 and 2 disclose a configuration in which captured transmission image data is input into a learning model for judgment. However, to acquire images after washing, a lighting device is placed under the transported eggs, and an imaging device is placed opposite the eggs, sandwiching them. In this case, the lighting device may become contaminated by the egg washing water, liquid eggs from broken eggs, or falling eggshells. On the other hand, if the device is placed upside down, the imaging device will become contaminated. Transmission image data is suitable for judging the internal state through spectral analysis, but it is difficult to analyze stains on the outer shells of wet eggs after washing or broken eggs.

[0007] Patent Document 2 also states that a learning model based on reflected images may also be used. To acquire reflected images, it is envisioned that the eggs are illuminated from diagonally above, and the reflected images are captured by an imaging device from above the eggs. In this case, contamination of the lighting device is avoided, but identifying cracked or dirty eggs after washing is not easy, even with machine learning. Patent Document 2 does not describe any experimental examples, and there is no objective basis for recognizing that the content described in Patent Document 2 can detect cracked or dirty eggs after washing from reflected images.

[0008] Contaminated eggs can be classified by size, for example, into large, medium, and small contaminated eggs, and there is a desire to remove the relatively small number of large contaminated eggs as far upstream as possible, and remove the large number of small contaminated eggs as far downstream as possible. However, if all eggs are removed at the same time regardless of size, the removal device will become larger, which will increase the burden on the operator. Furthermore, there is a problem with broken eggs, which must be removed after washing to avoid contamination to other items.

[0009] In view of the above-mentioned circumstances and problems, an object of the present invention is to provide an egg sorting device and an egg sorting method that use a learning model created by intelligent information technology using training data including reflection images in which the appearance of eggs after washing is labeled in regions of a predetermined size. Another object of the present invention is to provide a device that can detect transported eggs using a learning model. Another object of the present invention is to provide an egg grading and packaging system including the above-mentioned egg sorting device. [Means for solving the problem]

[0010] The egg sorting device of the present invention comprises: a storage unit that stores an egg classification learning model created by intelligent information technology using training data including reflection images labeled with regions of a predetermined size according to the appearance of the eggs; The apparatus further includes an egg classification unit that classifies eggs according to their appearance by inputting a reflection image of the eggs as input data into the egg classification learning model. The egg sorting apparatus comprises: The apparatus may include an acquisition unit that acquires a reflected image from an imaging unit that captures a reflected image obtained by irradiating an egg with light directly or indirectly from a light source.

[0011] The egg sorting apparatus comprises: A conveying unit that conveys the egg while rotating it in a direction (short axis direction) perpendicular to the long axis of the egg (the length of the line connecting the blunt end and the sharp end); A light source (a lighting device, a reflecting unit having a reflective surface) that directly or indirectly irradiates the eggs with light; an imaging unit that captures a reflected image obtained by irradiating the eggs being transported by the transport unit with light directly or indirectly emitted from the light source; may have The image data captured by the imaging unit may be used as input data (verification data) and / or the teacher data (or training data) of the egg classification learning model. The egg classification learning model is a software program. The eggs transported by the transport unit (eggs placed on the egg placement spots) are rotating while being transported along the transport direction. The reflected image data may be a still image taken at any timing in that state, or an image cut out from a video.

[0012] The egg sorting apparatus comprises: The egg sorting unit may include an output unit that outputs the classification results obtained by the egg sorting unit and / or various data on defective eggs. The output unit may include a display device that displays various data, a printer that prints various data, a communication device that transmits various data, and a storage device that stores various data on a storage medium. The output unit may mark the defective egg information in the reflected image and display it on the display unit, or may transmit the information to an external device (including a mobile terminal or a server). The output unit may display data on the types and numbers of normal eggs and the types and numbers of defective eggs as classification results on the display unit, or may send the data to an external device (including a mobile terminal or server).

[0013] The egg classification training model may consist of a single training model or multiple training models. A single egg classification learning model is a first defective egg classifier that classifies first defective eggs that are different from normal eggs and should be discarded from other eggs; a second defective egg classifier that classifies a second defective egg different from the normal egg and the first defective egg from other eggs; a third defective egg classifier that classifies a third defective egg different from the normal egg and the first and second defective eggs from other eggs; and and a normal egg classifier that classifies eggs by size. In the present invention, the term "classifier" refers to a unit of classification function, and is not used as a term that directly refers to hardware such as equipment unless otherwise specified. The egg classification learning model, which is composed of a plurality of learning models, a first defective egg classification learning model that classifies first defective eggs that are different from normal eggs and should be discarded from other eggs; a second defective egg classification learning model that classifies a second defective egg different from normal eggs and the first defective egg from other eggs; may have The egg classification learning model The apparatus may further include a third defective egg classification learning model for classifying a third defective egg, which is different from normal eggs and the first and second defective eggs, from the other eggs. The egg classification learning model It may further include a normal egg classification learning model that classifies eggs by size (SS, S, MS, M, L, LL, LLL). The normal egg classification learning model may be composed of a single learning model or multiple learning models. The single normal egg classification learning model is An SS egg classifier that classifies SS size eggs from other size eggs; An S-egg classifier that classifies S-sized eggs from other sized eggs, an MS egg classifier that classifies MS size eggs from other size eggs; An M-egg classifier that classifies M-sized eggs from other sized eggs; An L egg classifier that classifies L eggs from other sizes of eggs; an LL egg classifier that classifies LL size eggs from other size eggs; and and an LLL egg classifier that classifies LLL size eggs from eggs of other sizes. The normal egg classification learning model, which is composed of a plurality of learning models, An SS egg classification learning model that classifies SS size eggs from other size eggs. An S-egg classification learning model that classifies S-sized eggs from other sized eggs, An MS egg classification learning model that classifies MS size eggs from other size eggs; An M-egg classification learning model that classifies M-sized eggs from eggs of other sizes, An L-egg classification learning model that classifies L-sized eggs from eggs of other sizes, an LL egg classification learning model that classifies LL size eggs from other size eggs; and / or and an LLL egg classification learning model that classifies LLL size eggs from eggs of other sizes.

[0014] Another egg grading and packaging system of the present invention includes: a conveying unit that conveys eggs; an egg washing section for washing eggs; a first inspection unit (corresponding to the egg sorting device) that sorts the eggs processed in the egg washing unit; a first exclusion unit that excludes (excludes from the conveying unit) first defective eggs that have been classified in the first inspection unit as defective eggs that should be discarded and are different from normal eggs, so that the first defective eggs can be distinguished from other eggs; a second exclusion unit that excludes (excludes from the conveying unit) second defective eggs that have been classified as defective eggs different from normal eggs and the first defective eggs in the first inspection unit so that they can be distinguished from other eggs; a drying section for drying eggs, the drying section being disposed downstream of the egg sorting device and the first rejection section; It has.

[0015] The egg grading and packaging system includes: A direction alignment unit for aligning the direction of the blunt end or sharp end may be provided downstream of the drying unit. The egg grading and packaging system includes: a second inspection unit that detects a third defective egg different from the first and second defective eggs; a third exclusion unit that excludes (excludes from the conveying unit) third defective eggs detected by the second inspection unit so that they can be distinguished from normal eggs; may have The second inspection unit The test may be performed using an egg classification learning model that classifies normal eggs and third defective eggs that are different from the first and second defective eggs.

[0016] The egg grading and packaging system includes: a weighing unit that is arranged downstream of the drying unit or downstream of the second inspection unit and that measures the weight of the eggs; a sorting and packaging unit (such as a device for packing eggs into egg cartons and packaging them, or a device for placing them on trays) that sorts eggs according to the weight of the eggs measured by the weighing unit and places them in containers (packs or trays); may have The egg grading and packaging system includes: The apparatus may also have a size sorting unit (corresponding to the egg sorting device) that sorts eggs according to their external size. The sorting and packaging unit may sort the eggs according to the weight of the eggs measured by the weighing unit and / or the size of the eggs classified by the size classifying unit, and place the eggs in containers.

[0017] The eggs used in the training data may be washed eggs and / or washed and dried eggs. In the present invention, the same level of classification accuracy can be obtained even when training data is created using dried eggs. The "eggs" may be white, pink, or brown. Examples of "dirty eggs" include yolk stains, feces and urine stains, etc. "Appearance" refers to the appearance of normal eggs with different outer diameter sizes (SS, S, MS, M, L, LL, etc.), as well as the appearance of defective eggs (eggs other than normal eggs) such as cracked eggs, dirty eggs, extremely large eggs, and deformed eggs. Examples of the first defective eggs include broken eggs (eggs with contents leaking out from the cracked area), cracked eggs (eggs with contents not leaking out from the cracked area), sunken eggs, etc. Cracked eggs may develop cracks that may grow larger or develop into broken eggs during the conveyance process, and are therefore preferably removed. Examples of the second defective eggs include dirty eggs, extremely large eggs, and deformed eggs. Examples of the third defective eggs include dirty eggs with a smaller stain area than the dirty area of ​​the second defective eggs, and small cracked eggs with a smaller crack than the crack area of ​​the first defective eggs. Small cracked eggs are less likely to develop into cracked eggs than the above-mentioned cracked eggs, but they may still develop into cracked eggs during transportation to retail stores, so they are preferably rejected. Normal eggs come in sizes such as SS, S, MS, M, L, and LL. Eggs larger than LL are classified as extra-large eggs. Malformed eggs are not typically egg-shaped (those with a blunt end and a sharp end), but rather are gourd-shaped, teardrop-shaped (those with extremely acute ends), wrinkled, or irregularly shaped. A "reflection image" is, for example, a direct reflection image obtained by direct illumination with a light source, an indirect reflection image obtained by indirect illumination with a light source reflected by a reflecting part, a reflection image obtained by both of these, or a reflection image obtained through a polarizing film placed between the imaging device and / or the light source and the egg.

[0018] The "labeling" of the training data may be performed according to the following rules. (1) For first defective eggs, cracked areas, cracked areas, and sunken areas of the eggshell are labeled with a predetermined mark such as a rectangle, circle, and / or polygon, which is a first area (first defective egg label). The first area may be set, for example, as an area ratio of 10% to 30%, with the flat area of ​​the egg recognized from the reflected image of the egg being 100%. The mark may be one or more marks along the edge or shape of the cracked area. For example, when using a rectangular mark, a square or rectangular mark may be used to match the shape of the cracked area, or an elongated rectangle may be used to match the shape of the cracked area. Multiple square or circular marks may be used along the edge or shape of the cracked area. Furthermore, the "marks" may be set in different colors and line thicknesses, or may be the same color and line thickness. Hereinafter, the same interpretation applies to "marks" unless otherwise specified. (2) For the second defective egg, the dirty egg is labeled with a predetermined mark such as a rectangle, circle, and / or polygon having a second area larger than the first area (large dirty egg label). The second area may be set to an area ratio of, for example, 20% to 90%, with the planar area of ​​the egg recognized from the reflected image of the egg being 100%. (3) The extra-large eggs of the second defective eggs are labeled with a predetermined mark such as a rectangle, circle, and / or polygon that is different from the first defective egg label and the large dirty egg label (extra-large egg label). (4) The deformed eggs of the second defective eggs are labeled with a predetermined mark such as a rectangle, circle, and / or polygon that is different from the above-mentioned first defective egg label, large dirty egg label, and extremely large egg label (deformed egg label). (5) A dirty egg having a small dirty area of ​​the third defective egg is labeled with a predetermined mark such as a rectangle, circle, and / or polygon having a third area smaller than the first area (small dirty egg label). The third area may be set to an area ratio of, for example, less than 10%, with the planar area of ​​the egg recognized from the reflected image of the egg being 100%. (6) Label normal eggs with different marks for each size (SS, S, MS, M, L, LL, LLL) (SS, S, MS, M, L, LL, LLL normal egg labels). The size of the rectangular mark may be set differently for each size.

[0019] Labeling may be performed by a user operating a user interface in the teacher data creation device, or may be performed in the egg classification learning model creation device. Labeling may also be performed automatically by the teacher data creation device or the egg classification learning model creation device (hereinafter referred to as the "automatic labeling generation device") without user operation. The automatic labeling generation device may have an automatic marking unit that performs image analysis on the reflected image, determines cracked areas, fissures, and depressions based on differences in color or shape from the eggshell, and labels these areas with predetermined marks such as rectangles, circles, and / or polygons that are the first area. The above-mentioned automatic labeling generation device may have an automatic marking unit that performs image analysis on the reflected image, and if the obtained dirty area is equal to or larger than a first threshold, labels the dirty area with a predetermined mark such as a rectangle, circle, and / or polygon having a second area larger than the first area. The automatic labeling generation device may have an automatic marking unit that performs image analysis on the reflected image and marks the eggs having an area equal to or greater than a threshold as being the largest eggs. The above-mentioned automatic labeling generation device may have an automatic marking unit that performs image analysis on the reflected image, calculates the degree of matching between the obtained egg shape (outline) and a preset egg shape (outline), and marks eggs whose matching value is below a threshold as malformed. The above-mentioned automatic labeling generation device may have an automatic marking unit that performs image analysis on the reflected image, and if the obtained dirty area is less than a second threshold value that is smaller than the first threshold value, labels the dirty area with a predetermined mark such as a rectangle, circle, and / or polygon having a third area that is smaller than the first area. The automatic labeling generation device may have an automatic marking unit that performs image analysis on the reflected image, calculates the degree of matching between the obtained egg shape (outline) and the egg shapes (outlines) of each pre-set size (SS, S, MS, M, L, LL, LLL), and marks and labels with the size that has the highest matching value.

[0020] Examples of "intelligent information processing technology" include machine learning, deep learning, reinforcement learning, and deep reinforcement learning. The algorithms for machine learning, deep learning, reinforcement learning, and deep reinforcement learning are not particularly limited, and conventional algorithms may be used. For supervised learning, various algorithms may be adopted, such as linear regression, generalized linear model, support vector regression, Gaussian process regression, ensemble method, decision tree, neural network, support vector machine, discriminant analysis, naive Bayes, and nearest neighbor method.

[0021] The method for generating an egg classification learning model includes: a training data creation step of creating training data including reflection images labeled with regions of a predetermined size according to the appearance of the eggs; The method includes a model generation step of generating an egg classification learning model using the egg reflection image data and the training data by intelligent information processing technology.

[0022] Another invention provides an egg sorting method, which is executed by an information processing device, The method includes an egg classification step of classifying eggs according to their appearance by inputting a reflection image of an egg into an egg classification learning model created by intelligent information technology using training data including reflection images labeled in regions of a predetermined size according to the appearance of the egg. The egg classification step includes: a first sub-step of classifying first defective eggs that are different from normal eggs and should be discarded using a first defective egg classification learning model that classifies first defective eggs from other eggs; a second sub-step of classifying a second defective egg different from the normal egg and the first defective egg using a second defective egg classification learning model that classifies the second defective egg from other eggs; a third sub-step of classifying a third defective egg different from the normal eggs and the first and second defective eggs using a third defective egg classification learning model that classifies the third defective egg from the other eggs; and / or The method may include a normal egg classification step of classifying eggs using a normal egg classification learning model that classifies eggs by size (SS, S, MS, M, L, LL, LLL). The normal egg classification step is The SS size eggs were classified using a SS egg classification learning model that classifies them from eggs of other sizes. The small size eggs are classified using an S-egg classification learning model that classifies small size eggs from other size eggs. Classify MS size eggs using an MS egg classification learning model that classifies MS size eggs from other size eggs. Classify M-sized eggs using an M-egg classification learning model that classifies M-sized eggs from other sizes of eggs. Classify large eggs using an L egg classification learning model that classifies large eggs from other sizes of eggs. Classifying LL size eggs using an LL egg classification learning model that classifies LL size eggs from other size eggs, and / or The method may include classifying LLL size eggs using an LLL egg classification learning model that classifies LLL size eggs from eggs of other sizes.

[0023] The egg detection device a storage unit that stores an egg detection learning model created by intelligent information technology using training data including reflection images that label the egg placement spots (elliptical spots on which eggs are placed, formed by adjacent roller conveyors) formed by a conveying unit (for example, multiple roller conveyors arranged in parallel) with or without eggs placed thereon; The device may also have an egg counting unit that detects eggs being transported by the transport unit and counts the number of eggs by inputting reflected images of eggs being transported by the transport unit into the egg detection learning model as input data. The egg detection device includes: The apparatus may include an acquisition unit that acquires a reflected image from an imaging unit that captures a reflected image obtained by irradiating an egg with light directly or indirectly from a light source. The egg detection device includes: A conveying unit that conveys the egg while rotating it in a direction (short axis direction) perpendicular to the long axis of the egg (the length of the line connecting the blunt end and the sharp end); A light source (a lighting device, a reflecting unit having a reflective surface) that directly or indirectly irradiates the eggs with light; an imaging unit that captures a reflected image obtained by irradiating the eggs being transported by the transport unit with light directly or indirectly emitted from the light source; may have The image data captured by the imaging unit may be used as input data (verification data) and / or the teacher data (or training data) of the egg detection learning model. The egg detection learning model is a software program.

[0024] The egg detection device does not require any other device (sensor, image processing device) to detect eggs being transported in the transport section, and can detect them using an egg detection learning model to determine the number of eggs being transported. The egg detection learning model may have the functionality of the egg classification learning model, and the egg classification learning model may have the functionality of the egg detection learning model. The egg detection device may have the function of the egg sorting unit, and the egg sorting device may have the function of the egg counting unit.

[0025] The "labeling" of the training data for the egg detection learning model may be performed according to the following rules. (1) Egg placement spots in a state where no egg is placed or where an egg is placed are labeled with a predetermined mark such as a rectangle, circle, and / or polygon (egg placement spot label). The eggs placed on the egg placement spot are in a state of rotation while being conveyed along the conveying direction. The reflected image data is an image cut out from a still image or a video at any timing in that state.

[0026] Labeling may be performed by a user operating a user interface in the teacher data creation device, or by a learning model creation device. Labeling may also be performed automatically by the teacher data creation device or the learning model creation device (hereinafter referred to as an "automatic labeling generation device"), without user operation. The automatic labeling generation device may have an automatic marking unit that performs image analysis on the reflected image to determine egg placement spots in states where an egg is not placed or where an egg is placed, and label these areas with predetermined marks such as rectangles, circles, and / or polygons.

[0027] A computer-readable recording medium having computer instructions stored thereon, The computer-readable recording medium stores an egg classification learning model created by intelligent information technology using training data including reflection images labeled with regions of a predetermined size according to the appearance of the eggs.

[0028] The display device is not particularly limited, and examples thereof include a liquid crystal monitor, an organic EL monitor, a CRT monitor, a smartphone, a tablet, and a monitor of a general-purpose personal computer.

[0029] The egg sorting device, the first inspection unit and / or the second inspection unit, and the egg detection device may be configured as an information processing device (e.g., a computer, a server) having a memory, a processor, a software program, a dedicated circuit, firmware, etc. The information processing device may be either on-premise or cloud-based, or a combination of both. [Brief explanation of the drawings]

[0030] [Figure 1A] FIG. 2 is a diagram showing an example of a cross-sectional side view of the egg sorting apparatus in the conveying direction. [Figure 1B] FIG. 2 is a diagram showing an example of a front cross-sectional view of the egg sorting apparatus in the conveying direction. [Figure 1C] FIG. 2 is a functional block diagram showing an example of functions of the egg sorting device. [Figure 2A] FIG. 2 is a functional block diagram showing an example of functions of the egg sorting device. [Figure 2B] FIG. 2 is a functional block diagram showing an example of functions of the egg sorting device. [Figure 2C] FIG. 2 is a functional block diagram showing an example of functions of the egg sorting device. [Figure 3] FIG. 1 is a diagram illustrating an example of an egg grading and packaging system. [Figure 4A] FIG. 10 is a diagram showing an example of training data for broken eggs. [Figure 4B] FIG. 10 is a diagram illustrating an example of teacher data for large dirty eggs. [Figure 5] FIG. 10 is a diagram illustrating an example of a relationship between rectangular marks. [Figure 6] FIG. 10 is a diagram showing an example of a detection result of a broken egg. [Figure 7] FIG. 10 is a diagram showing an example of the detection result of a dirty egg. [Figure 8] FIG. 10 is a diagram showing an example of a detection result of a broken egg in a comparative example. [Figure 9]FIG. 9 is a diagram showing an example of a broken egg detected by a learning model from the image used in the comparative example of FIG. 8. [Figure 10] 1 shows an example of a screen of a display device. DETAILED DESCRIPTION OF THE INVENTION

[0031] (egg sorting device) Fig. 1A is a side cross-sectional view in the conveying direction of egg sorting device 1. As shown in Fig. 1, it is composed of a rotary conveying part (roller conveyor) 10 that conveys eggs, and a cover member 11 that covers the top. 1B is a front cross-sectional view of the egg sorting apparatus 1 in the conveying direction. Imaging cameras are provided in the upper center of the cover member 11, and four imaging cameras, first imaging camera 121 to fourth imaging camera 124, are arranged perpendicular to the egg conveying direction. The number of imaging cameras does not have to be four, and they do not have to be arranged in the upper center of the apparatus.

[0032] First and second lights 131 and 132, which are light sources, are provided in the middle of the height direction of the cover member 11 so as to emit light upward, and both are LED lights. The number of light sources does not have to be two, and they do not have to be LED lights or ring lights. A first diffuse reflector 141 and a second diffuse reflector 142 corresponding to the LED lights 131 and 132, respectively, are provided above the cover member 11, and light emitted from the LED lights 131 and 132 is diffusely reflected toward the eggs on the rotary conveyor unit 10. The diffusely reflected light is reflected at all points in the device before heading toward the eggs, allowing the eggs to be uniformly irradiated with light.

[0033] In this embodiment, eggs are photographed multiple times by the camera as they are conveyed by a roller conveyor that is rotating and being indirectly irradiated with white light. In the case of direct or indirect light, a polarizing film may be placed between the camera and the eggs, and between the eggs and the light source. In another embodiment, the light source is not limited to white light, and may be warm white (orange) light, blue light, or the like.

[0034] The function of the egg sorting device 1 will be described with reference to FIG. 1C. The storage unit 2 stores an egg classification learning model 20 created by intelligent information technology using training data including reflection images labeled in regions of a predetermined size according to the appearance of the eggs. The egg classification learning model 20 has classification functions of a first defective egg classifier 21 and a second defective egg classifier 22. The egg sorting unit 3 sorts the eggs according to their appearance by inputting the reflected images of the eggs as input data into the egg sorting learning model 20. The egg sorting unit 3 reads out the egg sorting learning model 20 stored in the memory unit 2, and inputs the reflected images taken by the first imaging camera 121 to the fourth imaging camera 124 into the egg sorting learning model 20 to sort the eggs. The egg sorting unit 3 calculates the future movement position from the position coordinates and transport speed of the classified defective eggs. The egg sorting unit 3 determines the position coordinates from the reflected images.

[0035] The display device 31 displays defective egg information by marking defective areas in the reflected image (rectangular frame marks).The display device 31 also displays data on the types and numbers of normal eggs and the types and numbers of defective eggs as classification results. The communication unit 32 may send data on the types and numbers of normal eggs and the types and numbers of defective eggs as classification results to the control device of the egg sorting and packaging system. Communications unit 32 sends data on the position coordinates of various defective eggs and their future movement positions to the control units of first rejection unit 54 and second rejection unit 55, which reject first and second defective eggs. First rejection unit 54 may remove and reject the first defective eggs from the conveying unit by dropping them when they reach the drop spot on the conveyor, or by absorbing and transporting them with an absorption means. Second rejection unit 55 may remove and reject the second defective eggs from the conveying unit by dropping them when they reach the drop spot on the conveyor, or by absorbing and transporting them with an absorption means.

[0036] The egg classification learning model includes a first defective egg classification learning model 21 that classifies first defective eggs that are different from normal eggs and should be discarded from other eggs, and a second defective egg classification learning model 22 that classifies second defective eggs that are different from normal eggs and the first defective eggs from other eggs. In this embodiment, each classification learning model is configured as a multi-layer neural network, and may be configured as a deep neural network or a convolutional neural network. In this embodiment, a first bad egg classifier 21 classifies broken eggs from other eggs. A second bad egg classifier 22 classifies dirty eggs with large soiled areas from other eggs. In another embodiment, when two types of learning models, a learning model corresponding to the function of the first defective egg classifier 21 and a learning model corresponding to the function of the second defective egg classifier 22, are stored in the memory unit 2, the classification process may be executed in parallel or one of them may be executed first.

[0037] As described above, when an egg is classified (determined) as a first defective egg (broken egg) by the first inspection unit (egg sorting device 1) of this embodiment, it is dropped at a predetermined position on the conveyor (first rejection unit 54), removed from the conveyor, and discarded. When an egg is determined to be a second defective egg (highly soiled egg, very large egg, deformed egg), it is dropped onto a cushioned shelf or the like at another predetermined position on the conveyor (second rejection unit 55), and removed from the conveyor without being broken. Other eggs are not rejected from the conveyor, but are sent to predetermined inspection and weighing processes located downstream.

[0038] In another embodiment, the egg classification learning model may be trained to include the functionality of a cracked egg classifier to classify cracked eggs and / or a sunken egg classifier to classify sunken eggs in addition to broken eggs. In another embodiment, the egg classification learning model may be trained to include the functions of an extremely large egg classifier that classifies extremely large eggs and / or an abnormal egg classifier that classifies abnormal eggs, in addition to large dirty eggs.

[0039] (Egg grading and packaging system) An egg grading and packaging system 5 is shown in Figure 3. Egg grading and packaging system 5 of this embodiment has a conveying section 51, an egg washing section 52, a first inspection section 53 (corresponding to egg sorting section 1), a first rejection section 54, a second rejection section 55, a drying section 56, a direction alignment section 61, a second inspection section 71, a third rejection section (not shown), a weighing section 81, and a sorting and packaging section 91.

[0040] Conveying section 51 connects each device from upstream to downstream and transports eggs. Conveying section 51 is composed of a roller conveyor, a belt conveyor, etc. The egg washing section 52 washes the eggs. First inspection section 53 classifies the eggs processed in egg washing section 52. It has the functions of egg classifying device 1 described above. The first exclusion section 54 excludes first defective eggs (broken eggs, etc.) that have been classified by the first inspection section 53 as defective eggs that should be discarded and are different from normal eggs, so that they can be distinguished from other eggs (by excluding them from the transport section 51). The second rejection unit 55 rejects second defective eggs (such as heavily soiled eggs) classified as defective eggs different from normal eggs and first defective eggs by the first inspection unit 53 so that they can be distinguished from the other eggs. In this embodiment, the drying section 56 is disposed downstream of at least the first and second rejection sections 54 and 55, and dries the eggs.

[0041] The direction alignment unit 61 is disposed downstream of the drying unit 56 and aligns the direction of the blunt or sharp end. The second inspection unit 71 detects third defective eggs (slightly dirty eggs) that are different from the first and second defective eggs. The second inspection unit 71 may be an image analysis device that performs image analysis to detect dirty eggs. As an example, the dirty egg detection device disclosed in Japanese Patent Application Laid-Open No. 2019-203845 may be used. The third exclusion unit (not shown) excludes third defective eggs detected by the second inspection unit 71 so that they can be distinguished from normal eggs. The third exclusion unit is included as part of the configuration of the sorting and packaging unit 91, and transfers the third defective eggs (slightly dirty eggs) to a tray using suction means or gripping means, etc., and excludes them. Additionally, second inspection unit 71 or a separate component may be provided with a crack detection device that detects cracked eggs from the sound made when the eggs are struck. Cracked eggs detected by the crack detection device are transferred to a tray and removed by the third removal unit.

[0042] Weighing section 81 is disposed downstream of second inspection section 71 and measures the weight of the eggs. The sorting and packaging unit 91 sorts the eggs according to the weight of the eggs measured by the weighing unit 81 and stores them in containers corresponding to the sorted eggs. In this embodiment, the sorting and packaging unit 91 is configured as a device that packs and packages the eggs in egg cartons.

[0043] In another embodiment, the second inspection unit 71 shown in Fig. 2B is configured as an egg sorting device that sorts eggs using an egg sorting learning model. The egg sorting learning model has a third defective egg sorting learning model 23 that sorts third defective eggs (slightly soiled eggs) that are different from normal eggs and the first and second defective eggs from the other eggs. The egg sorting unit 3 reads out the third defective egg classification learning model 23 stored in the memory unit 2, inputs each of the reflected images captured by the first imaging camera 121 to the fourth imaging camera 124 into the third defective egg classification learning model 23, and classifies the third defective eggs from the other eggs. The egg sorting unit 3 calculates the future movement position of the classified defective egg from the position coordinates and conveyance speed. The egg sorting unit 3 determines the position coordinates from the reflected images. The communication unit 32 sends data on the position coordinates of the defective egg and its future movement position to the control unit of the third exclusion unit. The second inspection unit 71 may determine a defective egg as a third defective egg if the egg is detected as defective in both the results from the third defective egg classification learning model 23 and the results from the image analysis device, or if the egg is detected as defective in either one of the results. The determined egg is then rejected by the third rejection unit.

[0044] In another embodiment, the egg grading and packaging system may include a size sorting unit that sorts eggs according to their external size. This may be used instead of or in addition to the weighing unit. The size classification unit 85 shown in FIG. 2B is composed of an egg classification device that classifies eggs using an egg classification learning model. The egg classification learning model has a normal egg classification learning model 24 that classifies eggs for each egg size (SS, S, MS, M, L, LL, LLL). The egg classification unit 3 reads the normal egg classification learning model 24 stored in the storage unit 2, inputs each reflection image captured by the first imaging camera 121 to the fourth imaging camera 124 into the normal egg classification learning model 24, and classifies the eggs for each egg size (SS, S, MS, M, L, LL, LLL). The egg classification unit 3 calculates the future movement position from the position coordinates and conveyance speed of the classified defective eggs. The egg classification unit 3 obtains the position coordinates from the reflection image. The communication unit 32 sends the data of the position coordinates and future movement position of each egg size to the sorting and packaging unit 91. The sorting and packaging unit 91 may sort the eggs according to the weight of the eggs measured by the weighing unit 81 and / or the egg size classified by the size classification unit 85, and store them in a container.

[0045] (Generation of teacher data) In this embodiment, the teacher data of the simultaneous multi-classification egg classification learning model that can classify three types of eggs and others will be described, but it can also be used for the teacher data of the learning model that classifies one type of egg and other eggs. FIG. 4A shows an example of the teacher data of broken eggs, and FIG. 4B shows an example of the teacher data of heavily soiled eggs. The rectangular marks are arranged so as to surround the broken area or the heavily soiled area with the frame of the rectangular mark. If the cracked area is large, two or more rectangular marks are arranged. FIG. 5 shows an example of the relationship of the marks for creating the teacher data. In this embodiment, rectangular marks are used, and the size relationship of the rectangular marks follows the following rules according to the types of eggs to be classified. (1) Rectangular mark for slightly soiled eggs < Rectangular mark for broken eggs < Rectangular mark for heavily soiled eggs: The rectangle is a square. (2) Rectangular mark for SS eggs < Rectangular mark for S eggs < Rectangular mark for MS eggs < Rectangular mark for M eggs < Rectangular mark for L eggs < Rectangular mark for LL eggs (part of the marks is omitted): The rectangle is a rectangle. (3) Using an egg-shaped mark, the matching amount is calculated and the egg size is classified. (4) Rectangular mark for extra large eggs > Rectangular mark for LLL eggs > Rectangular mark for LL eggs As an example, a rectangular mark for an LLL egg may be 70mm x 50mm, and a rectangular mark for an extra large egg may be 774mm x 54mm.

[0046] When generating a learning model that classifies two or more types, it can be trained sequentially using one type of training data, or multiple types of training data can be used simultaneously.When generating a learning model that classifies one type and the others, it is trained using that one type of training data. In this embodiment, teacher data in which a rectangular mark for broken eggs is attached to broken eggs is input into the simultaneous multi-class egg learning model to intensively learn broken eggs. Next, teacher data in which a rectangular mark for large dirty eggs is attached to large dirty eggs is input to intensively learn large dirty eggs. Next, teacher data in which a rectangular mark for large eggs is attached to extremely large eggs is input to intensively learn extremely large eggs. These orders can be reversed, and the order is not particularly limited. In another embodiment, learning can be performed with multiple types of teacher data mixed together. Learning is possible by having different marks for each teacher data.

[0047] (Example) The egg sorting device 1 of this embodiment sorted eggs into broken eggs, dirty eggs, very large eggs, other eggs, and empty areas of the transfer conveyor (where there are no eggs in the egg placement spots). The functional blocks of the egg sorting device 1 of this embodiment are shown in Figure 2C. A trained egg classification learning model 20 is stored in the memory unit 2. The egg classification learning model 20 has various classification functions, including a broken egg classifier 21a, a dirty egg classifier 22a, a very large egg classifier 22b, and an egg placement spot classifier 28a. An egg classification learning model capable of simultaneous multiple classification judgment was generated by learning using the training data for the broken eggs in Figure 4A, the training data for the dirty eggs in Figure 4B, the training data for the extremely large eggs (not shown), and the training data for the egg placement spot (no eggs) (not shown). Images were taken with the cameras (121 to 124) of the egg classification device 1 and input into the egg classification learning model 20, and the classification (judgment) results output from the model are shown in Table 1. A rectangular mark is superimposed on the cracked area of ​​an egg classified (determined) as broken, and information indicating the type of defective egg, such as "crack," is displayed on the display device 31 (see FIG. 6). A rectangular mark is superimposed on the dirty area of ​​an egg classified (determined) as a very dirty egg, and information indicating the type of defective egg, such as "dirty," is displayed on the display device 31 (see FIG. 7). In this embodiment, the size of each rectangular mark is set to the same shape and size as the training data. In this example, the accuracy rate was high at 90% or more for both cracked eggs and heavily dirty eggs, and there were no misidentifications of cracked eggs and heavily dirty eggs.

[0048] [Table 1]

[0049] (Comparative Example) As a comparative example, rule-based image processing was performed. The rule-based image processing involved the following steps: (1) To prevent the influence of dirt on the rollers that transport the eggs, an egg image was extracted that had been masked using a threshold value specified by the eggshell color. (2) If the brightness specified by the eggshell color in the extracted egg image was lower than the threshold value and the color extraction was within a specified range, the egg was determined to be broken. (3) If the brightness specified by the eggshell color in the extracted egg image was lower than the threshold value or the saturation specified by the eggshell color was high, the egg was determined to be dirty. The detection results using rule-based image processing are shown in Table 2. The accuracy rate for cracked egg detection was low, with many cracked eggs being overlooked, while the accuracy rate for dirty egg detection was relatively good, but there were cases where eggs were mistakenly identified as cracked.

[0050] [Table 2]

[0051] In comparison with the comparative example (rule-based image processing), the example (learning model) showed a marked improvement in the accuracy rate for cracked eggs, and also showed a good accuracy rate for heavily soiled eggs. Figure 8 shows the results of detecting broken eggs using the method of the comparative example. Eggs detected as broken are marked with "crack." Figure 8 shows that half of the eggs were not determined to be broken. On the other hand, Figure 9 shows the results of detecting broken eggs using the learning model using the same image. Eggs detected as broken are marked with "crack" and a rectangular frame (mark). Multiple "crack" and rectangular frames (marks) are displayed depending on the size of the broken egg. Figure 9 shows that all of the broken eggs were determined to be broken. 10 shows an example of the screen of the display device of the egg sorting device 1. Images captured by the cameras 121, 122, 123, and 124 are shown. The state of the eggs captured by each camera is shown on the left side of the screen. It can be seen that a "broken egg" has been detected in the image from camera 1 (bottom left). A rectangular mark is displayed overlaid to indicate the cracked area. Cameras 121 and 123 are paired, with camera 121 capturing images from one direction and camera 123 capturing images from the opposite direction. As a result, camera 121 detected a broken egg, while camera 123 was unable to detect it. However, by inputting both images into the learning model, it was confirmed that it was possible to detect an egg even in a position hidden from the camera's capture area. Cameras 122 and 124 are also paired in the same way. The camera settings and the number of eggs counted are shown on the right side of the screen. The number of normal eggs, dirty eggs, broken eggs, large eggs (extra large eggs), and eggs transported by the transport section (number processed) processed by the egg sorting device 1 are counted and displayed in real time. An egg placement spot with no eggs placed thereon is also displayed with a rectangular mark. The egg sorting device 1 also has the function of the egg counting unit described above. The egg counting unit calculates the number of eggs transported (= number processed) by subtracting the number of egg placement spots with no eggs detected (determined) by the learning model from the speed of the transport unit and the set number of eggs that can be transported per unit speed.

[0052] In the comparative example, the inspection range was narrowed due to masking, and there were cases where the area was outside the range of color extraction, which is presumably why many eggs could not be recognized as broken.On the other hand, in the case of the learning model, it is presumed that recognition was improved by learning the characteristics of cracked areas with angular shapes.

[0053] The above examples and comparative examples show examples of classification results for broken eggs and large dirty eggs. Training data and learning models can also be created for cracked eggs, sunken eggs, slightly dirty eggs, deformed eggs, and very large eggs. Training data according to egg size can also be created to create learning models. [Explanation of symbols]

[0054] 1 Egg sorting device 10 Rotating conveyor 11 Cover member 121 First Imaging Camera 122 Secondary Imaging Camera 123 Third Imaging Camera 124 Fourth Imaging Camera 131 First Lighting 132 Second lighting 141 First diffuse reflector 142 Second diffuse reflector 5. Egg grading and packaging system 51 Conveyor 52 Egg washing department 53 First Inspection Department 54 First Exclusion Department 55 Second exclusion section 56 Drying section 61 Alignment Section 71 Second Inspection Department (Egg Sorting Device) 81 Measuring section 85 Size sorting unit (egg sorting device) 91 Sorting and Packaging Department

Claims

1. a storage unit that stores an egg classification learning model created by intelligent information technology using training data including reflection images distinguished by marks in predetermined size areas according to the appearance state of the eggs; an egg classification unit that classifies eggs according to their appearance by inputting a reflection image of the eggs as input data into the egg classification learning model, The first defective eggs are one or more selected from broken eggs, cracked eggs, and sunken eggs, The second bad egg is at least a dirty egg, The third defective egg is a dirty egg having at least a dirty area smaller in size than the dirty area of ​​the dirty egg of the second defective egg, The egg classification learning model a first defective egg classifier that classifies the first defective eggs, which are different from normal eggs and should be discarded, from other eggs; a second defective egg classifier that classifies the second defective egg, which is different from normal eggs and the first defective egg, from other eggs; and a third defective egg classifier that classifies the third defective egg, which is different from the first defective egg and the second defective egg, from other eggs; The first defective eggs are distinguished by a predetermined mark that is a first area, which is a cracked region where the eggshell is broken, a cracked region where there is a crack, and / or a sunken region where there is a depression, and the first area is set to an area ratio of 10% to 30% assuming that a flat area of ​​the egg recognized from a reflected image of the egg is 100%; The dirty eggs of the second defective eggs are distinguished by a preset mark in which the dirty area is a second area larger than the first area, and the second area is set at an area ratio of 20% to 90% assuming that the planar area of ​​the egg recognized from the reflected image of the egg is 100%; the dirty eggs of the third defective eggs are distinguished by a preset mark having a third area smaller than the first area, and the third area is set at an area ratio of less than 10% assuming that the planar area of ​​the egg recognized from the reflected image of the egg is 100%.

2. The egg classification learning model 10. The egg sorting apparatus of claim 1, further comprising a normal egg sorter for sorting eggs by size.

3. The egg classification learning model 3. The egg sorting device according to claim 1, further comprising a classifier trained using training data including reflected images labeled with a state where an egg is not placed on the egg placement spot or a state where an egg is placed on the egg placement spot.

4. An egg grading and packaging system for grading and packaging eggs, comprising: a conveying unit that conveys eggs; an egg washing section for washing eggs; The egg sorting apparatus according to claim 1, which sorts the eggs processed in the egg washing section; a first rejection unit that rejects first defective eggs that have been classified by the egg sorting device as defective eggs that should be discarded and are different from normal eggs, so that the first defective eggs can be distinguished from other eggs; a second rejection unit that rejects second defective eggs classified as defective eggs different from normal eggs and first defective eggs in the egg sorting device so that the second defective eggs can be distinguished from other eggs; a drying section for drying eggs, the drying section being disposed downstream of the egg sorting device and the first rejection section; having Egg grading and packaging system.

5. an egg classification step of classifying eggs according to their appearance by inputting a reflection image of an egg into an egg classification learning model created by intelligent information technology using teacher data including reflection images distinguished by marks in predetermined size regions according to the appearance of the eggs; The first defective eggs are one or more selected from broken eggs, cracked eggs, and sunken eggs, The second bad egg is at least a dirty egg, The third defective egg is a dirty egg having at least a dirty area smaller in size than the dirty area of ​​the dirty egg of the second defective egg, The egg classification step includes: a first sub-step of classifying the first defective eggs to be discarded, which are different from normal eggs, from other eggs using a first defective egg classification learning model; a second substep of classifying the second defective egg, which is different from normal eggs and the first defective egg, using a second defective egg classification learning model to classify the second defective egg from other eggs; and a third sub-step of classifying the third defective eggs, which are different from normal eggs, the first defective eggs, and the second defective eggs, from other eggs using a third defective egg classification learning model; The first defective eggs are distinguished by a predetermined mark that is a first area, which is a cracked region where the eggshell is broken, a cracked region where there is a crack, and / or a sunken region where there is a depression, and the first area is set to an area ratio of 10% to 30% assuming that a flat area of ​​the egg recognized from a reflected image of the egg is 100%; The dirty eggs of the second defective eggs are distinguished by a preset mark in which the dirty area is a second area larger than the first area, and the second area is set at an area ratio of 20% to 90% assuming that the planar area of ​​the egg recognized from the reflected image of the egg is 100%; The dirty eggs of the third defective eggs are distinguished by a preset mark having a third area smaller than the first area, and the third area is set at an area ratio of less than 10% with the planar area of ​​the egg recognized from the reflected image of the egg being 100%. Egg classification method.

6. The egg classification step includes:

6. The egg classification method according to claim 5, further comprising a step of classifying normal eggs using a normal egg classification learning model that classifies eggs by size.

7. The normal egg classification step includes: Classifying SS size eggs using an SS egg classification learning model that classifies SS size eggs from eggs of other sizes; Classifying small eggs using an S egg classification learning model that classifies small eggs from other sizes of eggs; Classifying MS size eggs using an MS egg classification learning model that classifies MS size eggs from other size eggs; Classifying medium-sized eggs using an M-egg classification learning model that classifies medium-sized eggs from other sizes of eggs; Classifying large eggs using an L egg classification learning model that classifies large eggs from other sizes of eggs; Classifying LL size eggs using a LL egg classification learning model that classifies LL size eggs from other size eggs, and / or classifying LLL size eggs using an LLL egg classification learning model that classifies LLL size eggs from eggs of other sizes; 7. The method of claim 6.

8. The egg classification step includes:

6. The egg classification method according to claim 5, further comprising a step of classifying the eggs using a classifier trained using training data including reflection images that distinguish between a state in which an egg is placed on the egg placement spot and a state in which an egg is not placed on the egg placement spot.

9. 2. An automatic labeling generation device for use in the egg sorting device of claim 1, a first automatic marking unit that performs image analysis on the reflected image, determines cracked areas, fissures, and / or depressions based on differences in color or shape from the eggshell, and distinguishes these areas with a predetermined mark having a first area; a second automatic marking unit that performs image analysis on the reflected image, and if the obtained dirty area is equal to or larger than a first threshold, distinguishes the dirty area with a predetermined mark having a second area larger than the first area; and a third automatic marking unit that performs image analysis on the reflected image, and if the obtained dirty area is less than a second threshold value that is smaller than the first threshold value, distinguishes the dirty area with a preset mark having a third area that is smaller than the first area. Automatic labeling generation device.

10. an automatic large egg marking unit that performs image analysis on the reflected image and marks eggs whose area is equal to or greater than a threshold as large eggs; an automatic abnormality marking unit that performs image analysis on the reflected image, calculates the degree of matching between the obtained egg shape and a preset egg shape, and marks eggs whose matching value is equal to or less than a threshold as abnormal; and / or 10. The automatic labeling generation device according to claim 9, further comprising an automatic size marking unit that performs image analysis of the reflected image, calculates the degree of matching between the obtained egg shape and the egg shapes of each preset size, and marks and distinguishes the size with the highest matching value.

Images