Peanut sorting method, device and medium based on shape features and computer vision

Abstract

The present invention relates to a peanut sorting method, device and medium based on shape features and computer vision, wherein the method comprises: obtaining a sample image containing peanuts to be sorted, and the light intensity and exposure time of the sample image; determining a dynamic threshold based on the light intensity and exposure time of the sample image, and binarizing the sample image according to the dynamic threshold to obtain a first image; subjecting the first image to noise reduction filtering and contour extraction in sequence, and extracting a peanut part from the sample image based on the obtained contour as a target image; extracting first-order morphological features of the peanuts to be sorted based on the target image; fusing the first-order morphological features of the peanuts to be sorted to obtain second-order morphological features; inputting the obtained second-order morphological features into a trained sorting model to obtain a sorting result. Compared with the prior art, the present invention has the advantages of being able to accurately sort peanuts under small samples, etc.

Description

Technical Field

[0001] The present invention relates to the field of food raw material sorting, and in particular to a peanut sorting method, device and medium based on shape features and computer vision. Background Art

[0002] Traditional peanut sorting generally uses weight sorting or sorting screens. Both methods are relatively rough. If more detailed sorting is required, manual sorting is required, which is difficult to meet the needs of modern large-scale production.

[0003] In recent years, with the spread of computer vision technology applications, some technicians have tried to apply computer vision technology to color sorters. For example, Chinese patent CN109635845A discloses a material sorting method and system based on principal component analysis and support vector machine. Computer vision technology is used to take pictures of the materials to be sorted. After the material parts are extracted, the main factors are determined by principal component analysis and then sorting is performed. However, the overall method is too rough and can only be applied to the sorting of materials such as sand and gravel. It has no sorting ability for products with high morphological similarity and is highly dependent on sample capacity. It has no reference significance for the sorting of small-scale food.

[0004] In addition, the document "Research on Sichuan Pepper Sorting Technology Based on Shape Features" provides a technical solution for sorting Sichuan pepper by using a color sorter. Similarly, after taking pictures of the materials to be sorted by computer vision technology, the images containing only Sichuan pepper can be identified through threshold segmentation, noise reduction filtering and contour extraction steps. Then, the nine morphological features of Sichuan pepper are extracted: area, perimeter, aspect ratio, circularity, diameter, compactness, rectangularity, diagonal length and slenderness. Based on these nine morphological features, the pepper is sorted by combining principal component analysis with support vector machine. However, this technical solution is difficult to apply to the sorting of peanuts:

[0005] 1. The overall size of Sichuan pepper increases or decreases monotonically along the axial direction, and the mass distribution is uneven. When Sichuan pepper falls, it will present a posture similar to that of a badminton. That is, no matter what posture the Sichuan pepper is in on the conveyor belt, it will present the same posture when it falls. Therefore, its first-order shape characteristics can be directly used for sorting. However, for peanuts, they are overall thicker and shorter than Sichuan peppers, and have a larger mass. Their posture is not controllable during the falling process. In addition, peanuts have single, double or even triple grains. Therefore, directly using the first-order shape characteristics will confuse peanuts of different shapes in different postures, resulting in poor sorting effect.

[0006] 2. There are spots on the surface of peanuts, so the imaging quality is greatly affected by the exposure intensity and exposure time. If a fixed threshold is used for segmentation, it is easy to cause confusion with the background plate, and the concave spots on the surface cannot be correctly identified, resulting in excessive errors in the subsequent shape feature extraction.

[0007] 3. It directly uses principal component analysis and support vector machine to sort all the features, but there is still the problem of requiring huge sample support. Summary of the invention

[0008] The purpose of the present invention is to provide a peanut sorting method, device and medium based on shape features and computer vision, which can realize accurate sorting of peanuts under small samples.

[0009] The purpose of the present invention can be achieved by the following technical solutions:

[0010] A peanut sorting method based on shape features and computer vision, comprising:

[0011] Acquire a sample image containing peanuts to be sorted, as well as the light intensity and exposure time for capturing the sample image;

[0012] Determine a dynamic threshold based on the light intensity and exposure time of the sample image, and perform binarization processing on the sample image according to the dynamic threshold to obtain a first image;

[0013] The first image is subjected to noise reduction filtering and contour extraction in sequence, and the peanut part is extracted from the sample image based on the obtained contour to obtain the target image;

[0014] The first-order morphological features of the peanuts to be sorted are obtained based on the target image extraction;

[0015] The second-order morphological features are obtained by fusing the first-order morphological features of the peanuts to be sorted;

[0016] The obtained second-order morphological features are input into the trained sorting model to obtain the sorting results.

[0017] The dynamic threshold is:

[0018]

[0019] in: T dynamic is the dynamic threshold, T base is the preconfigured baseline threshold. Light Intensity For light intensity, Exposure Time is the exposure time.

[0020] The noise reduction filtering process adopts median filtering, and the contour extraction process is implemented by using the Canny operator.

[0021] The second-order morphological features include a first second-order eigenvalue, a second second-order eigenvalue, and a third second-order eigenvalue;

[0022] The first and second order eigenvalues ​​are:

[0023]

[0024] in: F 1 are the first and second order eigenvalues, SR is the shape ratio, CDAR is the convex defect area ratio, Rl is the rectangularity,

[0025] The second-order eigenvalue is:

[0026]

[0027] in: F 2 is the second-order eigenvalue, P is the circumference, L 1 is the major axis length, L 2 is the minor axis length, S is the area;

[0028] The third second-order eigenvalue is:

[0029]

[0030] in: F 3 is the third second-order eigenvalue, μ color is the average color value, σ color is the color standard deviation, ED is the edge density, and ε is a very small constant.

[0031] The sample image is a grayscale image, the average color value is an average grayscale value, and the color standard deviation is a grayscale standard deviation.

[0032] The first-order morphological features include at least area, perimeter, major axis length, minor axis length, shape ratio, rectangularity, average color value, color standard deviation, edge density, and convex defect area ratio.

[0033] The sorting results include odd-shaped peanuts, moldy peanuts, peanut shells, peanut kernels and transversely multi-grain peanuts.

[0034] The sorting model is a machine learning model.

[0035] A peanut sorting device based on shape features and computer vision comprises a memory, a processor, and a program stored in the memory. When the processor executes the program, the method described above is implemented.

[0036] A storage medium stores a program, which implements the above method when executed.

[0037] Compared with the prior art, the present invention has the following beneficial effects:

[0038] The use of automatic threshold segmentation that takes light intensity and exposure time into consideration can solve the interference of small concave particles on the surface of peanuts during the binary segmentation process, thereby improving the segmentation accuracy of the peanut part in the binary first image. On this basis, the second-order morphological features obtained by fusion of first-order morphological features can explore the posture and morphological information of the depth of peanuts, effectively amplifying the difference between single peanuts and multiple peanuts in a horizontal direction, so that accurate sorting of peanuts can be achieved with small samples. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 It is a schematic flow chart of the main steps of the method of the present invention;

[0040] Figure 2 This is a schematic diagram of the basic components of a color sorter;

[0041] Among them: 1. action valve, 2. camera, 3. material trough, 4. products to be sorted. DETAILED DESCRIPTION

[0042] The present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. This embodiment is implemented based on the technical solution of the present invention, and provides a detailed implementation method and specific operation process, but the protection scope of the present invention is not limited to the following embodiments.

[0043] In order to solve the problem that the concave spots on the surface of peanuts are likely to interfere with the binary segmentation process during imaging, and the problem that peanuts are difficult to distinguish due to their own short and thick shape, uniform mass distribution, and the morphological characteristics of single, double, and multiple peanuts, a peanut sorting method based on shape features and computer vision is provided. Figure 1 As shown, including:

[0044] Step S1: obtaining a sample image containing peanuts to be sorted, as well as the light intensity and exposure time of the sample image;

[0045] The basic structure of the specific color sorter is as follows: Figure 2As shown, similar to the prior art, the color sorter mainly includes an action valve 1, a camera 2 and a plurality of material troughs 3. In the present application, the product 4 to be sorted is peanuts, and the camera 2 captures an image of the product 4 to be sorted, which is the sample image. In the present embodiment, in order to reduce the hardware cost of the whole machine, only one camera is provided, and subsequent distinction is performed through the method designed in the present application. Of course, in some other embodiments, multiple cameras 2 may also be provided. When shooting, the camera 2 will record the corresponding exposure time and light intensity. In the present embodiment, the exposure time and light intensity are directly returned by the controller of the fill light, and are specifically obtained by collecting the timing sequence of the current of the fill light through the current sensor. The specific principle is the improvement of the present application, so it will not be repeated here. Of course, in other embodiments, the corresponding light intensity sensor may also be used for collection, so that the accuracy can still be maintained after the fill light is aged.

[0046] Step S2: determining a dynamic threshold based on the light intensity and exposure time of the sample image, and binarizing the sample image according to the dynamic threshold to obtain a first image;

[0047] Similar to the related prior art, a grayscale image is generally required for segmentation. Therefore, in this embodiment, the sample image needs to be grayscaled first, and a grayscale histogram is used for binary segmentation. In some exploratory prior art, a fixed threshold is usually used for binary segmentation in the sorting of food raw materials, such as pepper and jujube. The fixed threshold binary segmentation is to select a fixed threshold T, set the pixel points less than the threshold T as background points, and set the pixel points greater than the threshold T as sample points. However, for peanuts, there are many concave spots on its surface. Because of the concavity and residual mud, such spots will become dim due to light, and because the main part of the peanut surface is relatively bright, it is impossible to select a too bright background, so the concave spots will interfere with the binary segmentation. For Shandong large peanuts, this problem is relatively not obvious, but for Xiao Jingsheng, this interference becomes very significant.

[0048] Based on this, in this application, in order to adapt to the light source characteristics of different machines to reduce magnetic interference, a dynamic threshold is used for binary segmentation, and the dynamic threshold is specifically:

[0049]

[0050] in: T dynamic is the dynamic threshold, T base is the preconfigured baseline threshold, Light Intensity For light intensity, Exposure Time is the exposure time.

[0051] Thus, compared with a fixed threshold, the present application has better binary segmentation results within a wider range of the color sorter, and can effectively reduce the interference of concave spots on the surface of peanuts.

[0052] In addition, in some embodiments, the sample image is directly selected as a grayscale image, which can reduce the cost, but the versatility of the color sorter will be reduced.

[0053] Step S3: subjecting the first image to noise reduction filtering and contour extraction in sequence, and extracting the peanut part from the sample image based on the obtained contour as the target image;

[0054] In this embodiment, similar to the prior art, median filtering is used for the salt and pepper noise in the image. By traversing the image using a 3*3 window template, the noise is effectively removed while maintaining the clarity of the image edge, providing a good foundation for subsequent feature extraction.

[0055] In this embodiment, on the basis of solving the interference problem of binary segmentation, similar to the prior art, in order to further extract the edge features of the material, the Canny operator is used to extract the contour. The main step of the Canny operator is to calculate the gradient.

[0056] Step S4: extracting the first-order morphological features of the peanuts to be sorted based on the target image;

[0057] In this embodiment, the first-order morphological features include at least area S ,perimeter P , major axis length L 1 , short axis length L 2 , shape ratio SR , rectangularity Rl , average color value μ color , color standard deviation σ color , edge density ED , convex defect area ratio CDAR , as follows:

[0058] (1) Area S

[0059] Similar to the prior art, the area S is the product of the sum of the pixels in the foreground part of the target image and the area of ​​a single pixel;

[0060] (2) Circumference P

[0061] Similar to the prior art, the perimeter PThat is, the product of the sum of the pixels of the contour of the foreground part of the target image and the side length of a single pixel. Of course, in some other embodiments, the product of the sum of the pixels of the contour and the diagonal length of a single pixel may also be used;

[0062] (3) Major axis length L 1

[0063] Similar to the prior art, the major axis length L 1 That is, the length of the long side of the minimum circumscribed rectangle of the outline of the foreground part of the target image;

[0064] (4) Minor axis length L 2

[0065] Similar to the prior art, the minor axis length L 2 That is, the length of the wide side of the minimum circumscribed rectangle of the outline of the foreground part of the target image;

[0066] (5) Shape ratio SR

[0067] The major axis length L 1 and minor axis length L 2 Ratio

[0068] (6) Rectangularity Rl

[0069] Similar to the prior art, the rectangular Rl Specifically, the area S Divide by the area of ​​the minimum bounding rectangle of the foreground contour in the target image;

[0070] (7) Average color value μ color

[0071] Specifically, it is the arithmetic mean of the color values ​​of all pixels in the foreground part;

[0072] (8) Color standard deviation σ color

[0073] Specifically, it is the standard deviation of the color values ​​of all pixels in the foreground part;

[0074] (9) Edge density ED

[0075] Specifically, it is the ratio of the contour perimeter to the area;

[0076] (10) Convex defect area ratio CDAR

[0077]

[0078] in: m is the number of convex hull vertices, ( x i , y i ) is the i The coordinates of the convex hull vertices.

[0079] In addition, if in some embodiments, the sample image is a grayscale image, the average color value is the average grayscale value, and the color standard deviation is the grayscale standard deviation.

[0080] Step S5: The second-order morphological features are obtained by fusing the first-order morphological features of the peanuts to be sorted. In this embodiment, the second-order morphological features include the first second-order eigenvalue, the second second-order eigenvalue and the third second-order eigenvalue, which are as follows:

[0081] The first and second order eigenvalues ​​are:

[0082]

[0083] in: F 1 are the first and second order eigenvalues, SR is the shape ratio, CDAR is the convex defect area ratio, Rl is the rectangularity,

[0084] The second-order eigenvalue is:

[0085]

[0086] in: F 2 is the second-order eigenvalue, P is the circumference, L 1 is the major axis length, L 2 is the minor axis length, S is the area;

[0087] The third second-order eigenvalue is:

[0088]

[0089] in: F 3 is the third second-order eigenvalue, μ color is the average color value, σ color is the color standard deviation, ED is the edge density, and ε is a very small constant.

[0090] In this way, after the previous stage has accurately extracted the image of the peanut part, the second-order morphological features obtained by fusion of the first-order morphological features can be used to explore the deep posture and morphological information of the peanuts, effectively amplifying the difference between a single peanut and multiple peanuts horizontally, so that accurate sorting of peanuts can be achieved with small samples.

[0091] Step S6: input the obtained second-order morphological features into the trained sorting model to obtain the sorting results.

[0092] In this embodiment, the specific sorting results of the sorting model include odd-shaped peanuts, moldy peanuts, peanut shells, peanut kernels and horizontally multi-grain peanuts. The horizontally multi-grain peanuts are collected in the same trough 3, waiting for subsequent manual sorting or re-sorting.

[0093] The sorting model is a machine learning model, such as a neural network model. In this embodiment, a support vector machine (SVM) model is used to classify the new second-order morphological features. The specific process is as follows:

[0094] 1. Input the second-order morphological features to be classified. The second-order morphological features are specifically input in the form of feature vectors.

[0095] 2. Use the RBF kernel function to calculate the similarity between the sample points and the support vector, and classify the samples into one of the following categories: odd-shaped peanuts, moldy peanuts, peanut shells, peanut kernels, and transverse multi-grain peanuts through the optimal hyperplane.

[0096] If the above functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., which can store program codes.

Claims

1. A peanut sorting method based on shape features and computer vision, characterized in that: include: Acquire a sample image containing peanuts to be sorted, as well as the light intensity and exposure time for capturing the sample image; Determine a dynamic threshold based on the light intensity and exposure time of the sample image, and perform binarization processing on the sample image according to the dynamic threshold to obtain a first image; The first image is subjected to noise reduction filtering and contour extraction in sequence, and the peanut part is extracted from the sample image based on the obtained contour to obtain the target image; The first-order morphological features of the peanuts to be sorted are obtained based on the target image extraction; The second-order morphological features are obtained by fusing the first-order morphological features of the peanuts to be sorted; The obtained second-order morphological features are input into the trained sorting model to obtain the sorting results; The dynamic threshold is: in: T dynamic is the dynamic threshold, T base is the preconfigured baseline threshold. Light Intensity For light intensity, Exposure Time is the exposure time; The second-order morphological features include a first second-order eigenvalue, a second second-order eigenvalue, and a third second-order eigenvalue; The first and second order eigenvalues ​​are: in: F 1 is the first and second order eigenvalue, SR is the shape ratio, CDAR is the convex defect area ratio, Rl is the rectangularity, The second-order eigenvalue is: in: F 2 is the second-order eigenvalue, P is the circumference, L 1 is the length of the major axis, L 2 is the length of the minor axis, S is the area; The third second-order eigenvalue is: in: F 3 is the third second-order eigenvalue, μ color is the average color value, σ color is the color standard deviation, ED is the edge density, ε is a very small constant; The first-order morphological features include at least area, perimeter, major axis length, minor axis length, shape ratio, rectangularity, average color value, color standard deviation, edge density, and convex defect area ratio.

2. A peanut sorting method based on shape features and computer vision according to claim 1, characterized in that: The noise reduction filtering process adopts median filtering, and the contour extraction process is implemented by using the Canny operator.

3. The peanut sorting method based on shape features and computer vision according to claim 1, characterized in that: The sample image is a grayscale image, the average color value is an average grayscale value, and the color standard deviation is a grayscale standard deviation.

4. The peanut sorting method based on shape features and computer vision according to claim 1, characterized in that: The sorting results include odd-shaped peanuts, moldy peanuts, peanut shells, peanut kernels and transversely multi-grain peanuts.

5. The peanut sorting method based on shape features and computer vision according to claim 1, characterized in that: The sorting model is a machine learning model.

6. A peanut sorting device based on shape features and computer vision, comprising a memory, a processor, and a program stored in the memory, characterized in that: When the processor executes the program, the method according to any one of claims 1 to 5 is implemented.

7. A storage medium having a program stored thereon, characterized in that: When the program is executed, the method according to any one of claims 1 to 5 is implemented.

Images