A red fruit ginseng sorting device and method based on machine vision

Abstract

The application discloses a red fruit ginseng sorting device and method based on machine vision, wherein the device comprises a feeding hopper, a conveying table, a conveying belt, a camera, a sorting steering engine, a distributing hopper and a control unit; the red fruit ginseng sorting method based on machine vision comprises the following steps: S1, collecting the image of the red fruit ginseng on the conveying roller by the camera and transmitting the image to the MCU of the control unit; S2, identifying and extracting the image contour features of the red fruit ginseng; S3, extracting the color features of the red fruit ginseng image; S4, establishing a machine learning model; and S5, identifying the levels of different fruits by using the machine learning model; according to the characteristic that the red fruit ginseng peel is easy to break, the smooth conveying belt designed by the application can separate the red fruit ginseng and avoid stacking and peel breakage; the machine vision image processing method provided by the application has high fruit recognition accuracy and high sorting accuracy.

Description

Technical Field

[0001] This invention belongs to the field of machine vision technology, specifically relating to a machine vision-based red ginseng sorting device and sorting method. Background Technology

[0002] Machine vision technology was first applied in agriculture in the mid-to-late 1980s. In 1986, G.E. Rehkugler et al. used machine vision to detect surface bruises on apples and graded them using the US apple grading standard, but the grading results had a large error. In 1988, Davenel et al. used machine vision to extract the size and bruise features of apples for grading. In 1990, Han. YJ et al. used a spectrophotometer to measure the reflectance of crops and background soil, separating the crop image from the background soil. In 1991, Tarbell K.A. et al. used chromatic coordinates to overcome the influence of light variations in the study of maize growth characteristics. In 1992, Seginer I et al. used machine vision to monitor the growth of plant leaves, and the results could be used as control signals for irrigation systems. In 1995, Nozer et al. used machine vision to study an automatic fruit grading system, extracting features such as shape, size, and color of fruits and classifying them using a backpropagation neural network. In 1996, Ahmad I.S. et al. extracted color information from maize images and found that RGB values ​​could effectively reflect symptoms of drought and nutrient deficiency in crops. In 2002, Hiroaki and Ishizawa analyzed the correlation between the chemical composition and taste of apples at three different maturity stages and spectral image data. In 2005, Panitnat Yimyam et al. used computer vision technology to detect the shape, size, color, and surface area of ​​mangoes. In 2003, E.J. Van Henten et al. developed a robot system for harvesting cucumbers in greenhouses based on machine vision. In 2010, Nick Sigrimis et al. used a combination of CCD cameras and filters to detect the nutritional and growth status of crops.

[0003] Research on machine vision technology in agricultural production in my country started relatively late compared to other developed countries, beginning in the 1990s. In 1995, Chen Xiaoguang et al. used image processing technology to analyze and determine vegetable seedling growth information, providing necessary information for transplanting and thinning. In 1998, Li Shaokun et al. applied image processing technology to crop information extraction and growth monitoring. In 2005, Wu Congling et al. used computer-aided non-destructive measurement of cucumber leaf crown projection area to predict the actual leaf area of ​​the plant, achieving good accuracy. In 2008, Song Yajie et al. studied a non-destructive crop moisture detection model based on machine vision technology. In 2009, Zhang Yuan et al. used multispectral machine vision technology to detect nitrogen in rapeseed, deriving detection models for different growth stages. In 2012, Peng Hui et al. proposed a disparity segmentation algorithm based on binocular vision to address the difficulty in segmenting overlapping fruits, which can effectively segment overlapping fruits. In 2014, Li Wenbin et al. used machine vision technology to study the location and maturity of Lingwu jujubes, and proposed an algorithm for recognizing the maturity level of Lingwu jujubes based on the combination of hue H and red ratio. The algorithm achieved a discrimination accuracy of 92.60%. Although the application of machine vision technology in agricultural production in my country has developed rapidly in the past decade or so, compared with other developed countries, my country still needs to make continuous progress.

[0004] While there has been some research on machine vision-based agricultural product grading technology both domestically and internationally, existing research suffers from two main problems: ① It only covers the identification and detection of nearly round fruits such as apples, citrus fruits, and tomatoes. For newly developed highland specialty agricultural products such as red ginseng (also known as spider fruit) and other irregularly shaped fruits and vegetables, there are currently no relevant grading standards or detection methods. ② Existing research largely focuses on image processing algorithms and theoretical studies of grading recognition, with very few studies linking recognition and sorting, or developing corresponding sorting systems and equipment. Summary of the Invention

[0005] This invention provides a machine vision-based red ginseng sorting device and sorting method. The sorting device feeds and conveys red ginseng. During the conveying process, a camera captures images of the red ginseng. The sorting method provided by this invention, which is also an image recognition processing method, analyzes and processes the captured images of the red ginseng. After obtaining the analysis and processing results, the MCU processing program in the control unit controls the sorting robotic arm to sort red ginseng of different grades into different sorting hoppers.

[0006] To achieve the above-mentioned technical objectives, the present invention is implemented through the following technical solution:

[0007] A machine vision-based red ginseng sorting device includes: a feeding hopper, a conveyor table, a conveyor belt, a camera, a sorting servo motor, a sorting hopper, and a control unit;

[0008] The feed hopper is located on the left side of the conveyor table;

[0009] The conveyor belt inside the conveyor platform is rotatable.

[0010] A camera is installed between the conveyor and the feed hopper, and above the conveyor;

[0011] A sorting servo motor is installed on one side of the conveyor to complete the sorting action; a corresponding material hopper matching the sorting servo motor is installed on the other side of the conveyor.

[0012] The control unit is connected to the side of the conveyor, and the control unit integrates an MCU. The MCU performs machine vision image processing and analysis on the images captured by the camera, and controls the drive of the sorting servo motor.

[0013] Preferably, the conveyor belt is provided with several fruit slots, and each fruit can fall into a fixed fruit slot;

[0014] Preferably, the camera is placed inside a lightbox under a light shield above the conveyor platform to obtain continuous and stable ambient light, thereby improving the recognition accuracy.

[0015] Another objective of this invention is to provide a machine vision-based method for sorting red ginseng, comprising the following steps:

[0016] S1: The camera captures images of the red ginseng on the conveyor belt and transmits them to the MCU in the control unit;

[0017] S2: Image contour feature recognition and extraction of ginseng:

[0018] An improved minimum graph cut (GrabCut) algorithm is employed: First, a Gaussian mixture model (GMM) with K Gaussian components (generally K=5) is used to model the target and background in the RGB color space. Then, pixels are divided into background pixels and target pixels by bounding boxes, and a Gaussian component in the GMM is assigned to each pixel. The RGB value of pixel n is substituted into each Gaussian component in the target GMM. The parameters of the GMM are learned and optimized for the given image data, and the belonging of each pixel to the target GMM or the background GMM is estimated. Finally, through multiple iterations, the parameters of the GMMs modeling the target and background are improved, and the entire iterative process converges to the optimal image segmentation result.

[0019] S3: Color feature extraction of red ginseng images: Utilizing the differences in color component H of the fruit in the HSV color space, for the H component of the fruit image, the distribution features of the histogram at different angles are used to divide it into significantly different subsets according to different maturity levels, thereby realizing the identification of fruit color;

[0020] S4: Establish a machine learning model: Use K-means for cluster analysis, take the mean of H components as the initial centroid, set K=3, and divide the initial training set into 3 feature sets;

[0021] S5: Identify different fruit grades using machine learning models: Train three SVM classifiers with three feature sets and use a one-to-one strategy to make judgments;

[0022] Preferably, the specific steps of the image contour feature recognition and extraction method are as follows:

[0023] S2.1: Establish a color model: Use the RGB color space, and use a Gaussian mixture model (GMM) with K Gaussian components (generally K=5) to model the target and the background respectively. For each pixel, it comes from either a Gaussian component of the target GMM or a Gaussian component of the background GMM.

[0024] S2.2: Initialization: An initial set is obtained by selecting the target with a box, that is, all pixels outside the box are used as background pixels, and all pixels inside the box are used as pixels that "may be the target";

[0025] S2.3: Iterative minimization: ① Assign Gaussian components in the GMM to each pixel, and substitute the RGB value of pixel n into each Gaussian component in the target GMM. ② Learn and optimize the parameters of the GMM for the given image data. ③ Segment and estimate whether each pixel belongs to the target GMM or the background GMM. ④ Repeat ① to ③ until the iterative process converges.

[0026] S2.4: Smooth the boundaries to form the final outline;

[0027] Preferably, the specific steps of the image color feature extraction method are as follows:

[0028] S3.1: Convert the RGB image captured by the camera into the HSV color space;

[0029] S3.2: Divide the detected images into different levels according to maturity, calculate the mean and variance of the H component at different angles, and thus obtain the distribution characteristics of the H component at different levels.

[0030] S3.3: In the later process of using machine learning models to identify different fruit levels, the distribution characteristics of the H component at different levels are used to train the machine to classify the fruit into the corresponding level based on the characteristics presented by the H component of the image.

[0031] Preferably, the machine learning model building process is as follows:

[0032] S4.1: Cluster analysis using K-means; the process is as follows: ① Use the mean of the H components as the initial cluster centers, with "small intra-cluster differences and large inter-cluster differences" as the optimization objective, and the difference is set as the Euclidean distance from the sample point to the centroid of its cluster; ② Iteratively calculate the distance from each sample point to the centroid, and assign the sample to the centroid that is closest to it, resulting in K clusters; ③ For each cluster, calculate the average distance of all sample points assigned to that cluster as the new centroid, and recalculate; ④ Repeat the above three steps until all clusters no longer change.

[0033] S4.2: To facilitate operation and improve sorting efficiency, the prepared initial training set is divided into 3 clusters using the K-means algorithm, which are 3 feature sets divided by H components;

[0034] S4.3: Train an SVM classifier using a feature set; the process is as follows: ① Map the feature vectors of instances to some points in space as support vectors; ② Find a hyperplane such that points closer to the hyperplane have a larger distance; ③ Use the obtained hyperplane to distinguish the classification of the feature vectors.

[0035] S4.4: Use a one-to-one strategy for judgment; the method is as follows: ① Use three trained binary classifiers to calculate the probability of the feature vector to be classified to belong to the class; ③ Select the class with the highest vote as the category of the feature of the input sample to be classified, so as to achieve the final judgment.

[0036] The beneficial effects of this invention are:

[0037] The device provided by this invention can sort red ginseng. In view of the easily broken skin of red ginseng, the smooth conveyor belt designed by this invention can separate the red ginseng, avoid accumulation, and prevent skin damage. Separating the red ginseng does not affect the camera's image acquisition of the red ginseng. In addition, the machine vision image processing method provided by this invention has high recognition accuracy and high sorting accuracy. Attached Figure Description

[0038] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0039] Figure 1 This is a schematic diagram of the structure of the machine vision-based red ginseng sorting device of the present invention;

[0040] The attached diagram lists the components represented by each number as follows:

[0041] 1-Feed hopper, 2-Conveyor table, 201-Support leg, 3-Conveyor belt, 301-Grain trough, 4-Camera, 401-Light shield, 5-Sorting servo motor, 6-Divider hopper, 7-Control unit. Detailed Implementation

[0042] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0043] Example 1

[0044] A machine vision-based red ginseng sorting device includes: a feeding hopper 1, a conveyor table 2, a conveyor belt 3, a camera 4, a sorting servo motor 5, a sorting hopper 6, and a control unit 7.

[0045] The feed hopper 1 is located on the left side of the conveyor 2;

[0046] The conveyor belt 3 is rotatable inside the conveyor platform 2;

[0047] A camera 4 is installed between the conveyor 2 and the feed hopper 6 and above the conveyor 2;

[0048] A sorting servo motor 5 is installed on one side of the conveyor 2 to complete the sorting action; a sorting hopper 6 matching the sorting servo motor 5 is installed on the other side of the conveyor 2.

[0049] The control unit 7 is connected to the side of the conveyor 2. The control unit 7 integrates an MCU. The MCU performs machine vision image processing and analysis on the images captured by the camera 4, and controls the drive of the sorting servo motor 5.

[0050] Preferably, the conveyor belt 3 is provided with a plurality of fruit slots 301, and each fruit can fall into a fixed fruit slot 301;

[0051] Preferably, the camera 4 is placed inside the light box of the light shield 401 above the conveyor 2 to obtain continuous and stable ambient light, thereby improving the recognition accuracy.

[0052] Example 2

[0053] A machine vision-based method for sorting red berries includes the following steps:

[0054] S1: The camera captures images of the red ginseng on the conveyor belt and transmits them to the MCU in the control unit;

[0055] S2: Image contour feature recognition and extraction of ginseng:

[0056] An improved minimum graph cut (GrabCut) algorithm is employed: First, a Gaussian mixture model (GMM) with K Gaussian components (generally K=5) is used to model the target and background in the RGB color space. Then, pixels are divided into background pixels and target pixels by bounding boxes, and a Gaussian component in the GMM is assigned to each pixel. The RGB value of pixel n is substituted into each Gaussian component in the target GMM. The parameters of the GMM are learned and optimized for the given image data, and the belonging of each pixel to the target GMM or the background GMM is estimated. Finally, through multiple iterations, the parameters of the GMMs modeling the target and background are improved, and the entire iterative process converges to the optimal image segmentation result.

[0057] S3: Color feature extraction of red ginseng images: Utilizing the differences in color component H of the fruit in the HSV color space, for the H component of the fruit image, the distribution features of the histogram at different angles are used to divide it into significantly different subsets according to different maturity levels, thereby realizing the identification of fruit color;

[0058] S4: Establish a machine learning model: Use K-means for cluster analysis, take the mean of H components as the initial centroid, set K=3, and divide the initial training set into 3 feature sets;

[0059] S5: Identify different fruit grades using machine learning models: Train three SVM classifiers with three feature sets and use a one-to-one strategy to make judgments;

[0060] Preferably, the specific steps of the image contour feature recognition and extraction method are as follows:

[0061] S2.1: Establish a color model: Use the RGB color space, and use a Gaussian mixture model (GMM) with K Gaussian components (generally K=5) to model the target and the background respectively. For each pixel, it comes from either a Gaussian component of the target GMM or a Gaussian component of the background GMM.

[0062] S2.2: Initialization: An initial set is obtained by selecting the target with a box, that is, all pixels outside the box are used as background pixels, and all pixels inside the box are used as pixels that "may be the target";

[0063] S2.3: Iterative minimization: ① Assign Gaussian components in the GMM to each pixel, and substitute the RGB value of pixel n into each Gaussian component in the target GMM. ② Learn and optimize the parameters of the GMM for the given image data. ③ Segment and estimate whether each pixel belongs to the target GMM or the background GMM. ④ Repeat ① to ③ until the iterative process converges.

[0064] S2.4: Smooth the boundaries to form the final outline;

[0065] Preferably, the specific steps of the image color feature extraction method are as follows:

[0066] S3.1: Convert the RGB image captured by the camera into the HSV color space;

[0067] S3.2: Divide the detected images into different levels according to maturity, calculate the mean and variance of the H component at different angles, and thus obtain the distribution characteristics of the H component at different levels.

[0068] S3.3: In the later process of using machine learning models to identify different fruit levels, the distribution characteristics of the H component at different levels are used to train the machine to classify the fruit into the corresponding level based on the characteristics presented by the H component of the image.

[0069] Preferably, the machine learning model building process is as follows:

[0070] S4.1: Cluster analysis using K-means; the process is as follows: ① Use the mean of the H components as the initial cluster centroids, with "small intra-cluster differences and large inter-cluster differences" as the optimization objective, and the difference is set as the Euclidean distance from the sample point to the centroid of its cluster; ② Iteratively calculate the distance from each sample point to the centroid, and assign the sample to the centroid closest to it, resulting in K clusters; ③ For each cluster, calculate the average distance of all sample points assigned to that cluster as the new centroid, and perform the calculation again; ④ Repeat the above three steps until all clusters no longer change.

[0071] S4.2: To facilitate operation and improve sorting efficiency, the prepared initial training set is divided into 3 clusters using the K-means algorithm, which are 3 feature sets divided by H components;

[0072] S4.3: Train an SVM classifier using a feature set; the process is as follows: ① Map the feature vectors of instances to some points in space as support vectors; ② Find a hyperplane such that points closer to the hyperplane have a larger distance; ③ Use the obtained hyperplane to distinguish the classification of the feature vectors.

[0073] S4.4: Use a one-to-one strategy for judgment; the method is as follows: ① Use three trained binary classifiers to calculate the probability of the feature vector to be classified to belong to the class; ③ Select the class with the highest vote as the category of the feature of the input sample to be classified, so as to achieve the final judgment.

Claims

1. A machine vision-based hawthorn berry sorting device, characterized in that, include: Feed hopper, conveyor table, conveyor belt, camera, sorting servo motor, sorting hopper, control unit; The feed hopper is located on the left side of the conveyor table; The conveyor belt inside the conveyor platform is rotatable. A camera is installed between the conveyor and the feed hopper, and above the conveyor; A sorting servo motor is installed on one side of the conveyor to complete the sorting action; a corresponding material hopper matching the sorting servo motor is installed on the other side of the conveyor. The control unit is connected to the side of the conveyor, and the control unit integrates an MCU. The MCU performs machine vision image processing and analysis on the images captured by the camera, and controls the drive of the sorting servo motor. The sorting method of the red ginseng sorting device based on machine vision includes the following steps: S1: The camera captures images of the red ginseng on the conveyor belt and transmits them to the MCU in the control unit; S2: Image contour feature recognition and extraction of ginseng: An improved minimum graph cut algorithm is adopted: First, a Gaussian Mixture Model (GMM) with K Gaussian components is used to model the target and background in the RGB color space. Then, pixels are divided into background pixels and target pixels by bounding boxes. A Gaussian component in the GMM is assigned to each pixel. The RGB value of pixel n is substituted into each Gaussian component in the target GMM. The parameters of the GMM are learned and optimized for the given image data to estimate whether each pixel belongs to the target GMM or the background GMM. Finally, through multiple iterations, the parameters of the GMMs modeling the target and background are improved. The entire iterative process converges to the optimal image segmentation result. S3: Color feature extraction of red ginseng images: Utilizing the differences in color component H of the fruit in the HSV color space, for the H component of the fruit image, the distribution features of the histogram at different angles are used to divide it into significantly different subsets according to different maturity levels, thereby realizing the identification of fruit color; S4: Establish a machine learning model: Use K-means for cluster analysis, take the mean of H components as the initial centroid, set K=3, and divide the initial training set into 3 feature sets; S5: Identify different fruit grades using machine learning models: Train three SVM classifiers with three feature sets and use a one-to-one strategy to make judgments; The process of establishing the machine learning model is as follows: S4.1: Cluster analysis using K-means; the process is as follows: ① Use the mean of the H components as the initial cluster centers, with "small intra-cluster differences and large inter-cluster differences" as the optimization objective, and the difference is set as the Euclidean distance from the sample point to the centroid of its cluster; ② Iteratively calculate the distance from each sample point to the centroid, and assign the sample to the centroid that is closest to it, resulting in K clusters; ③ For each cluster, calculate the average distance of all sample points assigned to that cluster as the new centroid, and recalculate; ④ Repeat the above three steps until all clusters no longer change. S4.2: To facilitate operation and improve sorting efficiency, the prepared initial training set is divided into 3 clusters using the K-means algorithm, which are 3 feature sets divided by H components; S4.3: Train an SVM classifier using a feature set; the process is as follows: ① Map the feature vectors of instances to some points in space as support vectors; ② Find a hyperplane such that points closer to the hyperplane have a larger distance; ③ Use the obtained hyperplane to distinguish the classification of the feature vectors. S4.4: Use a one-to-one strategy for judgment; the method is as follows: ① Use three trained binary classifiers to calculate the probability of the feature vector to be classified to belong to the class; ③ Select the class with the highest vote as the category of the feature of the input sample to be classified, so as to achieve the final judgment.

2. The machine vision-based hawthorn sorting device according to claim 1, characterized in that, The conveyor belt is equipped with several fruit slots.

3. The machine vision-based hawthorn sorting device according to claim 1, characterized in that, The camera is placed inside a lightbox under a light shield above the conveyor platform to obtain continuous and stable ambient light, thereby improving the recognition accuracy.

4. The machine vision-based hawthorn sorting device according to claim 1, characterized in that, The specific steps of the image contour feature recognition and extraction method are as follows: S2.1: Establish a color model: Use the RGB color space and model the target and background using a GMM with K Gaussian components. For each pixel, it comes from either a Gaussian component of the target GMM or a Gaussian component of the background GMM. S2.2: Initialization: An initial set is obtained by selecting the target with a box, that is, all pixels outside the box are used as background pixels, and all pixels inside the box are used as pixels that "may be the target"; S2.3: Iterative minimization: ① Assign Gaussian components in the GMM to each pixel, and substitute the RGB value of pixel n into each Gaussian component in the target GMM. ② Learn and optimize the parameters of the GMM for the given image data. ③ Segment and estimate whether each pixel belongs to the target GMM or the background GMM. ④ Repeat ① to ③ until the iterative process converges. S2.4: Smooth the boundaries to form the final outline.

5. The machine vision-based hawthorn sorting device according to claim 1, characterized in that, The specific steps of the image color feature extraction method are as follows: S3.1: Convert the RGB image captured by the camera into the HSV color space; S3.2: Divide the detected images into different levels according to maturity, calculate the mean and variance of the H component at different angles, and thus obtain the distribution characteristics of the H component at different levels. S3.3: In the later process of using machine learning models to identify different fruit levels, the distribution characteristics of the H component at different levels are utilized to train the machine to classify the fruit into the corresponding level based on the characteristics presented by the H component of the image.

Images