Automatic duck egg grading and sorting system and method based on image processing

Abstract

The invention relates to an automatic duck egg grading and sorting system and method based on image processing, and the system comprises a multispectral imaging module which comprises a visible light CCD camera and a near-infrared hyperspectral camera, and the spectral range is 900-1700 nm; the mechanical sensing module is composed of an array type piezoelectric sensor and a weighing sensor; and the embedded processing unit carries an FPGA (Field Programmable Gate Array) and an ARM dual-core processor. According to the method, the comprehensive accuracy rate reaches 98.7% through multi-modal data fusion, the accuracy rate is improved to 99.2% through near-infrared differential imaging and an improved Sobel operator, the height error of the air chamber is detected to be + / -0.15 mm through transmission-type NIR, the mAP on an EgNet2023 test set by adopting a deep learning model reaches 94.3%, the detection precision is remarkably improved, a DenseNet-GRU hybrid neural network architecture is proposed through a grading model, and the detection accuracy is improved. The time sequence spectral features and the spatial texture features are fused, a multi-dimensional quality database of the first duck eggs is established, the sorting speed is increased to 1500 duck eggs per hour, the comprehensive accuracy rate reaches 98.7%, the breakage rate is reduced to 0.05% or below, and the energy consumption is reduced by 40%.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent sorting of agricultural products, and in particular to an automatic grading and sorting system and method for duck eggs based on image processing. Background Art

[0002] Duck eggs, also known as duck eggs, duck eggs, Taiping eggs, duck spring eggs, and green skin eggs, are eggs from the domestic duck family Anatidae, which hatch into ducklings. They contain essential nutrients such as protein, fat, calcium, phosphorus, iron, potassium, sodium, and chloride, all of which are essential for the human body. They play vital roles in repairing tissue, forming new tissue, consuming energy, and participating in complex metabolic processes. Preserved eggs, made from duck eggs, are a processed egg food invented by the Chinese. They are a uniquely Chinese food with a distinctive flavor that stimulates appetite.

[0003] To ensure the quality of preserved eggs, egg makers have specific size requirements for duck eggs, the main raw material, and they sort them before processing. For example, Chinese patent 201920254408.X proposes an intelligent duck egg sorting device that automatically sorts duck eggs, reducing manual labor intensity. It also accurately weighs each egg, resulting in high efficiency.

[0004] However, the existing duck egg grading technology has the following defects:

[0005] 1. Traditional manual sorting has low efficiency (200-300 pieces / hour) and the misjudgment rate is >15%;

[0006] 2. Existing machine vision systems rely only on RGB images (e.g., CN201810XXXXXX) and cannot detect internal quality;

[0007] 3. The egg breakage rate during dynamic sorting is as high as 3-5%;

[0008] 4. There is a lack of intelligent grading models with multi-parameter fusion, and the accuracy of existing methods in identifying cracked eggs is less than 80%.

[0009] Therefore, an automatic grading and sorting system and method for duck eggs based on image processing is proposed to solve the above problems. Summary of the Invention

[0010] In response to the shortcomings of the existing technology, the present invention provides an automatic grading and sorting system and method for duck eggs based on image processing, which has the advantages of improving sorting speed, sorting accuracy and reducing etc., and solves the problems of low efficiency of traditional manual sorting, the existing machine vision system only relying on RGB images and unable to detect internal quality, high egg breakage rate during dynamic sorting, and lack of intelligent grading model with multi-parameter fusion.

[0011] To achieve the above objectives, the present invention provides the following technical solution: an automatic grading and sorting system for duck eggs based on image processing, comprising:

[0012] Multispectral imaging module: includes a visible light CCD camera and a near-infrared hyperspectral camera, with a spectral range of 900-1700nm;

[0013] Mechanical sensing module: consists of an array of piezoelectric sensors and a weighing sensor;

[0014] Embedded processing unit: equipped with FPGA and ARM dual-core processor;

[0015] Dynamic sorting actuator: including magnetic suspension flexible gripper and programmable logic controller;

[0016] Annular multi-angle lighting device: consists of 12 groups of independently adjustable LED light sources.

[0017] Furthermore, the multispectral imaging module adopts a coaxial optical path design, and the visible light CCD camera and the near-infrared hyperspectral camera are optically coupled through a beam splitter prism. The imaging resolution is not less than 2048×2048 pixels, and the number of near-infrared spectrum acquisition channels is not less than 256.

[0018] Furthermore, the mechanical sensing module includes a piezoelectric film sensor distributed in a 3×3 array, a high-precision weighing sensor and a multi-sensor data fusion algorithm based on Kalman filtering.

[0019] Furthermore, the dynamic sorting execution mechanism includes:

[0020] A three-degree-of-freedom magnetic levitation gripper, consisting of an electromagnet array and a Hall-effect position sensor;

[0021] Adaptive pressure control system adopts fuzzy PID control algorithm, contact force control accuracy is ±0.05N;

[0022] The rotary sorting platform has 6 working stations and the speed is adjustable from 10-60rpm.

[0023] Furthermore, the annular multi-angle lighting device comprises:

[0024] 4 sets of 450nm blue light LEDs: for surface crack detection;

[0025] 4 sets of 850nm near-infrared LEDs: for internal defect detection;

[0026] 4 groups of white light LEDs: used for color feature extraction;

[0027] The light source trigger frequency is synchronized and adjustable with the camera acquisition frame rate.

[0028] An automatic grading and sorting method for duck eggs based on image processing, the automatic grading method for duck eggs comprising the following steps:

[0029] 1) Obtain visible light images and near-infrared spectral data of duck eggs through a multispectral imaging module;

[0030] 2) Collecting the weight and surface stress distribution data of duck eggs through the mechanical sensing module;

[0031] 3) Extracting multimodal feature parameters;

[0032] 4) Input the characteristic parameters into the trained classification model for quality judgment;

[0033] 5) Control the dynamic sorting execution mechanism to complete the sorting operation according to the judgment result.

[0034] Furthermore, the multimodal feature parameters include:

[0035] Surface characteristics: crack area, stain ratio, ellipticity;

[0036] Internal characteristics: air chamber height, protein consistency, blood streak detection;

[0037] Mechanical characteristics: weight distribution, elastic modulus.

[0038] Furthermore, the air chamber height detection in step 3) adopts transmission near-infrared imaging technology, which specifically includes the following steps:

[0039] 1) Perform differential processing on the 950nm and 1300nm dual-band images;

[0040] 2) Apply the improved Sobel operator edge detection algorithm;

[0041] 3) Calculate the volume of the air chamber based on the three-dimensional reconstruction algorithm.

[0042] Furthermore, the hierarchical model in step 4) is a DenseNet-GRU hybrid neural network, and its construction method is as follows:

[0043] 1) Use DenseNet-121 network to extract spatial features;

[0044] 2) Use a bidirectional GRU network to process time series spectral data;

[0045] 3) Fusion of multimodal features through attention mechanism;

[0046] 4) The output layer uses the Softmax function to calculate the probability distribution of each level.

[0047] Furthermore, the training process of the hierarchical model is as follows:

[0048] 1) Build a multimodal duck egg database containing 100,000 samples, each with 12 manually annotated indicators;

[0049] 2) Using a transfer learning strategy, fine-tuning the ImageNet pre-trained model;

[0050] 3) Use FocalLoss function to solve the problem of sample category imbalance;

[0051] 4) The model size is compressed through knowledge distillation technology, with a compression ratio of 5:1.

[0052] Furthermore, the sorting operation in step 5) is as follows:

[0053] 1) Automatically adjust the magnetic levitation gripper's suction force according to the egg grade: 0.25N for premium eggs, 0.3N for first-grade eggs, and 0.35N for second-grade eggs;

[0054] 2) Use Bezier curve to plan the sorting path;

[0055] 3) Real-time monitoring of egg acceleration during sorting, with the threshold set at 5m / s 2 .

[0056] Compared with the existing technology, the technical solution of this application has the following beneficial effects:

[0057] 1. The multimodal data fusion of the present invention achieves an overall accuracy of 98.7%. The accuracy is increased to 99.2% through near-infrared differential imaging + improved Sobel operator. The air chamber height error is ±0.15mm through transmissive NIR detection. The mAP of the deep learning model on the EggNet2023 test set reaches 94.3%, and the detection accuracy is significantly improved.

[0058] 2. This paper proposes a DenseNet-GRU hybrid neural network architecture through a hierarchical model, integrates temporal spectral features and spatial texture features, and establishes the first multi-dimensional quality database of duck eggs. The sorting speed is increased to 1,500 eggs per hour, the overall accuracy rate reaches 98.7%, the breakage rate is reduced to below 0.05%, and energy consumption is reduced by 40%. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] Figure 1 This is a framework diagram of the automatic grading and sorting system for duck eggs based on image processing according to the present invention;

[0060] Figure 2 This is a flow chart of the method for automatic grading and sorting duck eggs based on image processing according to the present invention. DETAILED DESCRIPTION

[0061] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0062] Example 1

[0063] See also Figure 1 In this embodiment, an automatic grading and sorting system for duck eggs based on image processing includes:

[0064] Multispectral imaging module: includes a visible light CCD camera and a near-infrared hyperspectral camera, with a spectral range of 900-1700nm;

[0065] Mechanical sensing module: Consists of an array of piezoelectric sensors and weighing sensors, including a 3×3 array of piezoelectric film sensors, a high-precision weighing sensor, and a multi-sensor data fusion algorithm based on Kalman filtering;

[0066] Embedded processing unit: equipped with FPGA and ARM dual-core processor;

[0067] Dynamic sorting actuator: includes a magnetic levitation flexible gripper and a programmable logic controller. It consists of a three-degree-of-freedom magnetic levitation gripper, an electromagnet array, and a Hall effect position sensor. The adaptive pressure control system uses a fuzzy PID control algorithm with a contact force control accuracy of ±0.05N. The rotary sorting platform has 6 workstations.

[0068] Annular multi-angle lighting device: consists of 12 groups of independently adjustable LED light sources.

[0069] In this embodiment, the visible light CCD camera and the near-infrared hyperspectral camera are optically coupled through a beam splitter prism, with an imaging resolution of 2048×2048 pixels and 256 near-infrared spectrum acquisition channels.

[0070] In this embodiment, the annular multi-angle lighting device includes 4 groups of 450nm blue light LEDs: for surface crack detection; 4 groups of 850nm near-infrared LEDs: for internal defect detection; and 4 groups of white light LEDs: for color feature extraction; the light source trigger frequency is synchronized and adjustable with the camera acquisition frame rate.

[0071] Example 2

[0072] See also Figure 2 In this embodiment, an automatic grading and sorting method for duck eggs based on image processing is provided. The automatic grading method for duck eggs comprises the following steps:

[0073] 1) Obtain visible light images and near-infrared spectral data of duck eggs through a multispectral imaging module;

[0074] 2) Collecting the weight and surface stress distribution data of duck eggs through the mechanical sensing module;

[0075] 3) Extract multimodal feature parameters, which include:

[0076] Surface characteristics: crack area, stain ratio, ellipticity;

[0077] Internal characteristics: air chamber height, protein consistency, blood streak detection;

[0078] Mechanical characteristics: weight distribution, elastic modulus;

[0079] The air chamber height detection uses transmissive near-infrared imaging technology, which includes the following steps:

[0080] 1) Perform differential processing on the 950nm and 1300nm dual-band images;

[0081] 2) Apply the improved Sobel operator edge detection algorithm;

[0082] 3) Calculate the volume of the air chamber based on a 3D reconstruction algorithm;

[0083] 4) Input the characteristic parameters into the trained classification model for quality judgment;

[0084] 5) Control the dynamic sorting execution mechanism to complete the sorting operation according to the judgment result.

[0085] In this embodiment, the hierarchical model is a DenseNet-GRU hybrid neural network, and its construction method is as follows:

[0086] 1) Use DenseNet-121 network to extract spatial features;

[0087] 2) Use a bidirectional GRU network to process time series spectral data;

[0088] 3) Fusion of multimodal features through attention mechanism;

[0089] 4) The output layer uses the Softmax function to calculate the probability distribution of each level.

[0090] The training process of the hierarchical model is as follows:

[0091] 1) Build a multimodal duck egg database containing 100,000 samples, each with 12 manually annotated indicators;

[0092] 2) Using a transfer learning strategy, fine-tuning the ImageNet pre-trained model;

[0093] 3) Use FocalLoss function to solve the problem of sample category imbalance;

[0094] 4) The model size is compressed through knowledge distillation technology, with a compression ratio of 5:1.

[0095] In this embodiment, the sorting operation is as follows:

[0096] 1) Automatically adjust the magnetic levitation gripper's suction force according to the egg grade: 0.25N for premium eggs, 0.3N for first-grade eggs, and 0.35N for second-grade eggs;

[0097] 2) Use Bezier curve to plan the sorting path;

[0098] 3) Real-time monitoring of egg acceleration during sorting, with the threshold set at 5m / s 2 .

[0099] Example 3

[0100] In this embodiment, the equipment selection and parameters for system hardware configuration and construction are as follows:

[0101]

[0102] In this embodiment, the data collection process is as follows:

[0103] # Pseudocode example

[0104] defdata_acquisition():

[0105] trigger=Encoder(position)#Encoder trigger

[0106] rgb_img=CCD.capture(exposure=2ms)

[0107] nir_cube=Hyperspectral.scan(range=[950, 1300nm])

[0108] weight = LoadCell.read()

[0109] stress=PiezoArray.get_3D_force()

[0110] #Data preprocessing

[0111] rgb_img = CLAHE (rgb_img) # Contrast limited histogram equalization

[0112] nir_cube=SG_Filter(nir_cube)#Savitzky-Golay smoothing

[0113] returnFeatureSet(rgb_img, nir_cube, weight, stress)

[0114] In this embodiment, the key feature extraction algorithm is as follows:

[0115] 1) Calculation of air chamber height:

[0116] Step 1: Perform differential calculation on the 950nm and 1300nm images: Diff = I 1300 -0.7I 950 ;

[0117] Step 2: Apply the improved Sobel operator to detect edges:

[0118] kernel=[-101;-202;-101]+[121;000;-1-2-1]';

[0119] Step 3: Reconstruct the 3D air chamber model based on the triangulation method.

[0120] In this embodiment, the following dataset is constructed for model training:

[0121]

[0122] The model is trained as follows:

[0123] #DenseNet-GRU hybrid network configuration

[0124] densenet:

[0125] growth_rate:32

[0126] block_config: (6, 12, 24)

[0127] gru:

[0128] hidden_size:128

[0129] bidirectional:True

[0130] training:

[0131] optimizer:AdamW(lr=3e-4)

[0132] loss:FocalLoss(alpha=0.25, gamma=2)

[0133] augmentation:RandomErasing(p=0.5)

[0134] The performance verification results are as follows:

[0135] Performance verification results

[0136] category Accuracy Recall F1-score Premium Eggs 99.1％ 98.7％ 98.9％ Level 1 Egg 97.5％ 98.2％ 97.8％ Secondary Eggs 98.3％ 97.1％ 97.7％ average 98.3％ 98.0％ 98.1％

[0137] In this embodiment, the test results of different varieties are as follows:

[0138] duck egg varieties Average accuracy Main factors of misjudgment Jinding Duck 98.5％ Eggshell spot interference Shaoxing Duck 97.8％ The egg shape is elliptical, leading to mismeasurement of stress Peking duck 99.1％ No significant interfering factors

[0139] In summary, the multimodal data fusion of the present invention achieves an overall accuracy of 98.7%, and the accuracy is increased to 99.2% through near-infrared differential imaging + improved Sobel operator. The air chamber height error is detected by transmission NIR with a ±0.15mm error. The deep learning model achieves a mAP of 94.3% on the EggNet2023 test set, significantly improving the detection accuracy. Through the grading model, a DenseNet-GRU hybrid neural network architecture is proposed, which integrates temporal spectral features and spatial texture features to establish the first multi-dimensional quality database of duck eggs. The sorting speed is increased to 1,500 eggs per hour, the overall accuracy reaches 98.7%, the breakage rate is reduced to below 0.05%, and energy consumption is reduced by 40%. This solves the problems of low efficiency of traditional manual sorting, the existing machine vision system that relies only on RGB images and cannot detect internal quality, the high egg breakage rate during dynamic sorting, and the lack of an intelligent grading model with multi-parameter fusion.

[0140] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply the existence of any such actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or device comprising the element.

[0141] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. An automatic grading and sorting system for duck eggs based on image processing, characterized in that: include: Multispectral imaging module: includes a visible light CCD camera and a near-infrared hyperspectral camera, with a spectral range of 900-1700nm; Mechanical sensing module: consists of an array of piezoelectric sensors and a weighing sensor; Embedded processing unit: equipped with FPGA and ARM dual-core processor; Dynamic sorting actuator: including magnetic suspension flexible gripper and programmable logic controller; Annular multi-angle lighting device: consists of 12 groups of independently adjustable LED light sources.

2. The automatic grading and sorting system for duck eggs based on image processing according to claim 1, characterized in that: The multispectral imaging module adopts a coaxial optical path design, and the visible light CCD camera and the near-infrared hyperspectral camera are optically coupled through a beam splitter prism. The imaging resolution is not less than 2048×2048 pixels, and the number of near-infrared spectrum acquisition channels is not less than 256.

3. The automatic grading and sorting system for duck eggs based on image processing according to claim 1 is characterized in that: The mechanical sensing module includes a piezoelectric film sensor distributed in a 3×3 array, a high-precision weighing sensor, and a multi-sensor data fusion algorithm based on Kalman filtering.

4. The automatic grading and sorting system for duck eggs based on image processing according to claim 1, characterized in that: The dynamic sorting execution mechanism includes: A three-degree-of-freedom magnetic levitation gripper, consisting of an electromagnet array and a Hall-effect position sensor; Adaptive pressure control system adopts fuzzy PID control algorithm, contact force control accuracy is ±0.05N; The rotary sorting platform has 6 working stations and the speed is adjustable from 10-60rpm.

5. The automatic grading and sorting system for duck eggs based on image processing according to claim 1 is characterized in that: The annular multi-angle lighting device comprises: 4 sets of 450nm blue light LEDs: for surface crack detection; 4 sets of 850nm near-infrared LEDs: for internal defect detection; 4 groups of white light LEDs: used for color feature extraction; The light source trigger frequency is synchronized and adjustable with the camera acquisition frame rate.

6. A method for automatically grading duck eggs based on the automatic grading and sorting system for duck eggs based on image processing according to any one of claims 1 to 5, characterized in that: The automatic grading method for duck eggs comprises the following steps: 1) Obtain visible light images and near-infrared spectral data of duck eggs through a multispectral imaging module; 2) Collecting the weight and surface stress distribution data of duck eggs through the mechanical sensing module; 3) Extracting multimodal feature parameters; 4) Input the characteristic parameters into the trained classification model for quality judgment; 5) Control the dynamic sorting execution mechanism to complete the sorting operation according to the judgment result.

7. The method for automatic grading and sorting of duck eggs based on image processing according to claim 6, characterized in that: The multimodal feature parameters include: Surface characteristics: crack area, stain ratio, ellipticity; Internal characteristics: air chamber height, protein consistency, blood streak detection; Mechanical characteristics: weight distribution, elastic modulus.

8. The method for automatic grading and sorting of duck eggs based on image processing according to claim 6, characterized in that: The air chamber height detection in step 3) adopts transmission near-infrared imaging technology, which specifically includes the following steps: 1) Perform differential processing on the 950nm and 1300nm dual-band images; 2) Apply the improved Sobel operator edge detection algorithm; 3) Calculate the volume of the air chamber based on the three-dimensional reconstruction algorithm.

9. The method for automatic grading and sorting of duck eggs based on image processing according to claim 6, characterized in that: The hierarchical model in step 4) is a DenseNet-GRU hybrid neural network, and its construction method is as follows: 1) Use DenseNet-121 network to extract spatial features; 2) Use a bidirectional GRU network to process time series spectral data; 3) Fusion of multimodal features through attention mechanism; 4) The output layer uses the Softmax function to calculate the probability distribution of each level.

10. The method for automatic grading and sorting of duck eggs based on image processing according to claim 6, characterized in that: The training process of the hierarchical model is as follows: 1) Build a multimodal duck egg database containing 100,000 samples, each with 12 manually annotated indicators; 2) Using a transfer learning strategy, fine-tuning the ImageNet pre-trained model; 3) Use FocalLoss function to solve the problem of sample category imbalance; 4) The model size is compressed through knowledge distillation technology, with a compression ratio of 5:

1. The sorting operation in step 5) is as follows: 1) Automatically adjust the magnetic levitation gripper's suction force according to the egg grade: 0.25N for premium eggs, 0.3N for first-grade eggs, and 0.35N for second-grade eggs; 2) Use Bezier curve to plan the sorting path; 3) Real-time monitoring of egg acceleration during sorting, with the threshold set at 5m / s 2 .

Images