Machine vision system for automatic grading of dried orange peel and sorting control method of machine vision system

Abstract

The invention discloses a machine vision system for automatic grading of dried orange peel and a sorting control method thereof, and relates to the technical field of machine vision intelligent grading, and the machine vision system comprises an image collection module which is used for collecting dried orange peel image data, recognizing dried orange peel individual feature pictures in the dried orange peel image data by using a CNN model, and sending the images to a sorting module; the image acquisition module is used for establishing a rectangular plane coordinate system based on the pericarpium citri reticulatae image data, marking a pericarpium citri reticulatae individual feature picture in the pericarpium citri reticulatae image data, acquiring a center coordinate of the pericarpium citri reticulatae individual feature picture, and acquiring a color forming parameter of a pericarpium citri reticulatae individual based on the pericarpium citri reticulatae individual feature picture; the control module collects a historical training data set in advance, and trains a machine learning model for identifying the color grade of the pericarpium citri reticulatae individual based on the historical training data set; automatic grading and sorting of the dried orange peel are achieved, and the efficiency and effect of automatic grading and sorting of the dried orange peel are greatly improved.

Description

Technical Field

[0001] The invention relates to the technical field of machine vision intelligent grading, and in particular to a machine vision system for automatic grading of tangerine peel and a sorting control method thereof. Background Art

[0002] Machine vision is a rapidly developing branch of artificial intelligence. Simply put, machine vision is to use machines to replace human eyes to make measurements and judgments. Automatic grading of tangerine peel by machine vision can greatly improve the efficiency and effect of automatic grading of tangerine peel.

[0003] Existing automated grading machine vision systems, such as the image grading automated detection system and method based on machine vision disclosed in Chinese patent publication number CN112378916B and the cathode plate detection control system and production line based on machine vision disclosed in Chinese patent application publication number CN115343293A, are unable to perform intelligent automatic grading and sorting of tangerine peel, resulting in poor efficiency and effect of automated sorting of tangerine peel. Summary of the invention

[0004] In order to overcome the above-mentioned technical problems, the purpose of the present invention is to provide a machine vision system for automated grading of tangerine peel and a sorting control method thereof, so as to solve the problem in the prior art that the machine vision system is unable to perform intelligent automatic grading and sorting of tangerine peel, resulting in poor efficiency and effect of automated sorting of tangerine peel.

[0005] The purpose of the present invention can be achieved through the following technical solutions:

[0006] The first aspect of the present invention is to provide a machine vision system for automated grading of tangerine peel, comprising an image acquisition module, which is used to acquire tangerine peel image data, use a CNN model to identify individual tangerine peel feature pictures in the tangerine peel image data, establish a plane rectangular coordinate system based on the tangerine peel image data, mark the individual tangerine peel feature pictures in the tangerine peel image data and obtain the center coordinates of the individual tangerine peel feature pictures, and acquire the color parameters of individual tangerine peel based on the individual tangerine peel feature pictures. The image acquisition module is data-connected to a control module, which pre-collects a historical training data set and trains a machine learning model for identifying the color grade of individual tangerine peel based on the historical training data set. The control module acquires tangerine peel image data in real time through the image acquisition module, identifies the color grade of the tangerine peel image data based on the machine learning model, and uses a sorting mechanism to sort the tangerine peel individuals on a tangerine peel transport mechanism based on the color grade and the center coordinates.

[0007] As a further solution of the present invention: the coloring parameters include individual area of ​​tangerine peel, individual shape of tangerine peel, individual color of tangerine peel and individual surface texture of tangerine peel.

[0008] As a further solution of the present invention: the tangerine peel transportation mechanism spreads out individual tangerine peels, and the image acquisition module collects tangerine peel image data of the spread out individual tangerine peels;

[0009] The tangerine peel image data includes a plurality of individual characteristic pictures of tangerine peel.

[0010] As a further solution of the present invention: the number of the sorting mechanisms is at least three groups, and the three groups of sorting mechanisms are equidistantly arranged on the top surface of the tangerine peel transport mechanism, and the side of the tangerine peel transport mechanism is fixedly connected with a grading conveying trough near the sorting mechanism, and the color grades correspond to the grading conveying troughs one by one.

[0011] As a further solution of the present invention: the sorting mechanism absorbs the tangerine peel individuals based on the central coordinates of the tangerine peel individuals;

[0012] The sorting mechanism moves the individual tangerine peels into a grading conveying trough based on the color grades of the individual tangerine peels.

[0013] As a further solution of the present invention: the method for extracting the individual area of ​​tangerine peel is:

[0014] Based on the individual characteristic images of tangerine peel, the Canny edge detection algorithm is used to extract the edge contours of tangerine peel individuals;

[0015] The individual area of ​​tangerine peel is calculated based on the edge contour using the contour finding function in OpenCV.

[0016] As a further solution of the present invention: the individual shape of the tangerine peel is determined based on the circularity, eccentricity or Hu moment by extracting the individual tangerine peel in the individual characteristic picture of the tangerine peel, and the individual shape of the tangerine peel is determined based on the circularity, eccentricity or Hu moment.

[0017] As a further solution of the present invention: the method for extracting the individual colors of tangerine peel is:

[0018] Grayscale processing is performed on the individual characteristic image of tangerine peel to form a grayscale image;

[0019] Extract the grayscale value of the grayscale image;

[0020] Determine the individual color of tangerine peel based on the grayscale value.

[0021] As a further solution of the present invention: according to the individual characteristic picture of tangerine peel, the individual surface texture of tangerine peel is extracted based on the gray level co-occurrence matrix algorithm.

[0022] The second aspect of the present invention is to provide a sorting control method for automatic grading of tangerine peel. The sorting control method is implemented based on the above-mentioned machine vision system for automatic grading of tangerine peel, and comprises the following steps:

[0023] S1: putting ungraded tangerine peel into the tangerine peel delivery mechanism, and the tangerine peel delivery mechanism spreads the tangerine peels flat on the top surface of the tangerine peel transportation mechanism;

[0024] S2: The image acquisition module collects the image data of the tangerine peel on the top surface of the tangerine peel transportation mechanism in real time;

[0025] S3: The control module uses the CNN model to identify individual characteristic images of tangerine peel in the tangerine peel image data collected in real time by the image collection module;

[0026] S4: establishing a plane rectangular coordinate system based on the tangerine peel image data, marking the individual characteristic pictures of the tangerine peel in the tangerine peel image data and obtaining the central coordinates of the individual characteristic pictures of the tangerine peel, and collecting the color parameters of the individual tangerine peel based on the individual characteristic pictures of the tangerine peel;

[0027] S5: The control module trains a machine learning model for identifying the color grade of the tangerine peel image data in the following manner:

[0028] Collect a historical training data set in advance, the historical training data set includes N groups of training data, N is a positive integer, each group of training data includes feature data and label data, the feature data is the color parameter, and the label data is the color grade;

[0029] The label data is the color grade of the tangerine peel on the top surface of the tangerine peel transportation mechanism when collecting each set of training data;

[0030] Each set of feature data in the historical training data is used as the input of the machine learning model. The color grade of the tangerine peel identified by the machine learning model for each set of feature data is used as the output. The color grade of the tangerine peel in the label data corresponding to each set of feature data is used as the recognition target. The training target is to minimize the sum of all recognition differences. The calculation formula for the recognition difference is: Ai=(Bi-Ci) 2 , where Ai is the recognition difference, the recognition difference is the difference between the color grade recognized by the model and the actual color grade, Bi is the recognized color grade corresponding to the i-th group of feature data, Ci is the color grade in the i-th group of label data, and the machine learning model is trained until the sum of the recognition differences reaches convergence and the training is stopped;

[0031] S6: The control module collects the image data of the tangerine peel on the tangerine peel transportation mechanism in real time through the image acquisition module, and identifies the color grade of the tangerine peel image data based on the machine learning model;

[0032] S7: The control module uses the sorting mechanism to sort the center coordinates of the individual tangerine peels on the tangerine peel transport mechanism based on the color grade, so that the individual tangerine peels enter the grading conveying trough.

[0033] Beneficial effects of the present invention:

[0034] 1. In the present invention, through the control module that is set, the control module collects the tangerine peel image data on the tangerine peel transportation mechanism in real time through the image acquisition module, identifies the color grade of the tangerine peel image data based on the machine learning model, and then uses the sorting mechanism to sort the tangerine peel individuals on the tangerine peel transportation mechanism based on the color grade, so that the tangerine peel individuals enter the grading conveying trough, realizes the automated grading and sorting of the tangerine peel individuals, and greatly improves the efficiency and effect of the automated grading and sorting of the tangerine peel.

[0035] 2. In the present invention, a sorting mechanism is provided, the hydraulic mechanism inside the driving mechanism is opened, and the height of the adsorption seat is controlled by the lifting rod, so that the matrix adsorption plate on the bottom surface of the adsorption seat contacts the tangerine peel on the top surface of the tangerine peel transport mechanism, and then the matrix adsorption plate at the corresponding position is opened to generate adsorption force on the corresponding tangerine peel individual, and then the tangerine peel is adsorbed and then transported to the top surface of the grading conveying trough, and then the matrix adsorption plate at the corresponding position is closed, so that the tangerine peel on the bottom surface of the matrix adsorption plate automatically falls on the top surface of the grading conveying trough, thereby realizing automatic sorting of tangerine peel individual. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] The present invention will be further described below in conjunction with the accompanying drawings.

[0037] Figure 1 It is a system block diagram of a machine vision system for automatic grading of dried tangerine peel according to the present invention;

[0038] Figure 2 It is a structural schematic diagram of the tangerine peel transportation mechanism of the present invention;

[0039] Figure 3 This is a schematic diagram of the structure of the tangerine peel transportation mechanism of the present invention when viewed from above;

[0040] Figure 4 It is a top view of the tangerine peel transportation mechanism of the present invention;

[0041] Figure 5 It is a structural schematic diagram of the transport rack in the present invention;

[0042] Figure 6 is a cross-sectional view of the transport rack of the present invention;

[0043] Figure 7 It is a cross-sectional view of the tangerine peel delivery mechanism of the present invention;

[0044] Figure 8 It is a schematic diagram of the overall structure of the sorting mechanism and the grading conveying trough in the present invention;

[0045] Fig. 9 It is a structural schematic diagram of the sorting mechanism in the present invention;

[0046] Fig.10 is a side view of the sorting mechanism of the present invention;

[0047] Fig.11 is a cross-sectional view of the transverse track of the present invention;

[0048] Fig.12 It is a structural schematic diagram of the driving mechanism and the adsorption mechanism in the present invention;

[0049] Fig.13 The present invention is a flow chart of a sorting control method for automatic grading of tangerine peel.

[0050] Explanation of the reference numerals: 1. Tangerine peel transport mechanism; 11. Transport frame; 12. Active transport wheel; 13. Driven transport wheel; 14. Transport belt; 15. Built-in support seat; 16. Support roller; 17. Isolation slot; 2. Driving motor; 3. Tangerine peel delivery mechanism; 31. Delivery shell; 32. Delivery slot; 33. Guide plate; 34. Paving channel; 35. Hydraulic cylinder; 36. Rotating plate; 37. Leveling brush; 4. Sorting mechanism; 41. Horizontal track; 411. Fixed cross bar; 412. Screw; 42. Moving seat; 421. Ball nut; 43. Driving mechanism; 431. Docking block; 432. Positioning slot; 44. Adsorption mechanism; 441. Adsorption seat; 442. Matrix adsorption disk; 443. Positioning block; 444. Lifting rod; 45. Support column; 5. Grading transport slot; 6. Image acquisition module. DETAILED DESCRIPTION

[0051] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0052] Example 1

[0053] like Figure 1 - Fig.13 As shown, the present invention discloses a machine vision system for automatic grading of tangerine peel, including an image acquisition module 6, which is used to collect tangerine peel image data. It should be noted that the image acquisition module 6 can be a high-definition camera, which is used to collect high-definition pictures of tangerine peel on the top surface of the tangerine peel transport mechanism 1, that is, tangerine peel image data;

[0054] Using the CNN model to identify the individual characteristic pictures of tangerine peel in the tangerine peel image data, it should be noted that the training data of the CNN model is collected by those skilled in the art in the process of using the machine vision system, that is, a certain amount of tangerine peel image data is collected, and the specific amount of tangerine peel image data is adaptively selected by those skilled in the art according to the training situation of the CNN model to ensure that the CNN model can identify all the individual characteristic pictures of tangerine peel in the tangerine peel image data. It should be noted that usually one tangerine peel image data contains several individual characteristic pictures of tangerine peel, so the CNN model needs to identify the individual characteristic pictures of tangerine peel in the tangerine peel image data, and one individual characteristic picture of tangerine peel should only contain one tangerine peel individual, so as to ensure that the subsequent sorting of tangerine peel individuals is more accurate;

[0055] A plane rectangular coordinate system is established based on the tangerine peel image data, and the individual characteristic pictures of the tangerine peel in the tangerine peel image data are marked and the central coordinates of the individual characteristic pictures of the tangerine peel are obtained. It should be noted that the contour of the tangerine peel individual must be extracted first, and the contour is determined by the cv2.findContours function, and then the geometric moment of the tangerine peel individual is calculated. The calculation formula of the geometric moment is:

[0056] ;

[0057] in It is the p+q order moment of the individual characteristic image of tangerine peel. I (x, y) is the pixel value, x is the horizontal label of the pixel in the image, and y is the vertical label of the pixel in the image. The zero-order moment is calculated based on the calculation formula of the geometric moment. , first-order moment and , and then calculate the centroid coordinates ( , ):

[0058] ; ;

[0059] Center of mass coordinates ( , ) is the central coordinate of the individual characteristic image of tangerine peel.

[0060] Based on the individual characteristic pictures of tangerine peel, the color parameters of tangerine peel are collected. The color parameters include the individual area, individual shape, individual color and individual surface texture of tangerine peel, among which:

[0061] The extraction method of individual area of ​​tangerine peel is:

[0062] Based on the individual characteristic pictures of tangerine peel, the Canny edge detection algorithm is used to extract the edge contour of the tangerine peel. The Canny edge detection algorithm uses the Sobel operator of the image to calculate the horizontal and vertical gradients:

[0063] , ;

[0064] in is the pixel value of the individual characteristic image of tangerine peel, is the horizontal gradient, is the vertical gradient. Based on the horizontal and vertical gradients, the gradient magnitude and direction of each pixel are calculated:

[0065] ,θ( ) = arctan( / );

[0066] in is the gradient magnitude of the pixel point (x, y), θ( ) is the direction of the pixel (x, y);

[0067] After obtaining the edge contour of the individual tangerine peel in advance through the Canny edge detection algorithm, the cv2.findContours function is used to extract the contour from the binary edge map, and then the cv2.contourArea function is used to directly calculate the area of ​​each contour, which is the area of ​​the individual tangerine peel.

[0068] Alternatively, based on the individual characteristic picture of tangerine peel, a computer is used to identify the region of the tangerine peel individual in the individual characteristic picture of tangerine peel, and then the regional area of ​​the tangerine peel individual is automatically generated by the computer, and the regional area is used as the individual tangerine peel area.

[0069] The individual shape of the tangerine peel is determined by extracting the circularity, eccentricity or Hu moment of the individual tangerine peel in the individual characteristic picture of the tangerine peel based on the circularity, eccentricity or Hu moment. It should be noted that when the individual shape of the tangerine peel is determined using the circularity, the following steps are followed:

[0070] C 圆形度 =4πA / P 2 ;

[0071] Among them, C 圆形度 is the roundness of individual tangerine peel, A is the area of ​​individual tangerine peel in the individual characteristic picture of tangerine peel, P is the circumference of individual tangerine peel in the individual characteristic picture of tangerine peel, and the circumference is calculated using the cv2.arcLength algorithm. The closer the roundness is to 1, the closer the individual tangerine peel in the individual characteristic picture of tangerine peel is to a circle. The technicians in this field preset the range of roundness and determine the goodness of the shape of individual tangerine peel according to the range of roundness.

[0072] The calculation formula of eccentricity is:

[0073] ;

[0074] It should be noted that is the eccentricity, a and b are respectively the length and width of the individual tangerine peel in the individual characteristic picture of the tangerine peel. The length and width are obtained by a technician in this field using a computer, and then the eccentricity is directly calculated by the above formula. The value range of the eccentricity is [0,1]. A value close to 0 indicates a shape close to a circle, and a value close to 1 indicates a shape close to a straight line or a slender shape. The value range of the eccentricity is preset by a technician in this field, and the goodness of the shape of the individual tangerine peel is determined based on the value range of the eccentricity.

[0075] The Hu moment is to calculate the geometric moment of individual tangerine peel. The specific calculation method of the geometric moment of individual tangerine peel has been given above and will not be repeated here. The Hu moment is combined with a clustering or classification algorithm (such as KNN or SVM) to determine the shape category. The technicians in this field determine the goodness of the shape of the individual tangerine peel based on the obtained shape category.

[0076] Generally speaking, the closer the individual shape of the tangerine peel is to a circle, the better the shape of the tangerine peel is, and vice versa. The technical personnel in this field will make adaptive adjustments based on the individual shape of the tangerine peel.

[0077] The method for extracting the individual color of tangerine peel is:

[0078] Grayscale processing is performed on the individual characteristic image of tangerine peel to form a grayscale image, and the grayscale value of the grayscale image is extracted. It should be noted that the grayscale value has a value range of [0, 255], and the grayscale value range is preset by a person skilled in the art;

[0079] The individual color of the tangerine peel is determined based on the preset gray value range, and the goodness of the individual color of the tangerine peel is adaptively set by those skilled in the art based on experience.

[0080] According to the individual characteristic picture of tangerine peel, the individual surface texture of tangerine peel is extracted based on the gray level co-occurrence matrix algorithm. It should be noted that the method for determining the individual surface texture of tangerine peel is:

[0081] Step 1: Obtain the individual characteristic image of tangerine peel, and then convert the individual characteristic image of tangerine peel into a grayscale image; since the gray-level co-occurrence matrix algorithm can only work on grayscale images, the input color image needs to be converted into a grayscale image;

[0082] Step 2: Use Gaussian blur to smooth the grayscale image;

[0083] Step 3: Grayscale quantization. For example, the grayscale value of the individual feature image of tangerine peel can be reduced from [0,255] to [0,15];

[0084] Step 4: Calculate the gray level co-occurrence matrix, extract the contrast, energy, homogeneity and correlation of the gray level image based on the gray level co-occurrence matrix, and then construct the contrast, energy, homogeneity and correlation into a feature matrix;

[0085] Step 5: Use a classification algorithm (such as KNN, SVM, etc.) based on the feature matrix to group or classify the surface texture of the tangerine peel, and the technicians in this field determine the goodness of the individual tangerine peel based on the surface texture of the tangerine peel according to their experience.

[0086] The control module is connected to the image acquisition module 6, and the control module collects a historical training data set in advance, and trains a machine learning model for identifying the color grade of individual tangerine peels based on the historical training data set;

[0087] The control module collects tangerine peel image data in real time through the image acquisition module 6, identifies the color grade of the tangerine peel image data based on the machine learning model, and uses the sorting mechanism 4 to sort the tangerine peel individuals on the tangerine peel transport mechanism 1 based on the color grade and center coordinates.

[0088] The tangerine peel transport mechanism 1 spreads the tangerine peels individually, and the image acquisition module 6 collects tangerine peel image data of the spread tangerine peels individually, the tangerine peel image data including a plurality of tangerine peel individual feature pictures;

[0089] There are at least three groups of sorting mechanisms 4, which are equidistantly arranged on the top surface of the tangerine peel transport mechanism 1. A grading conveying trough 5 is fixedly connected to the side of the tangerine peel transport mechanism 1 near the sorting mechanism 4. The color grades correspond one to one with the grading conveying trough 5. The sorting mechanism 4 absorbs the tangerine peel individuals based on the central coordinates of the tangerine peel individuals, and the sorting mechanism 4 moves the tangerine peel individuals into the grading conveying trough 5 based on the color grades of the tangerine peel individuals.

[0090] Based on the color grade, the sorting mechanism 4 is used to sort the tangerine peel on the tangerine peel transport mechanism 1. It should be noted that the historical training data set includes N groups of training data, N is a positive integer, and each group of training data includes feature data and label data, the feature data is the color parameter, and the label data is the color grade;

[0091] Color parameters include individual area, individual shape, individual color and individual surface texture of tangerine peel;

[0092] The label data is the color grade of the tangerine peel on the top surface of the tangerine peel transport mechanism 1 when collecting each set of training data;

[0093] Each set of feature data in the historical training data is used as the input of the machine learning model. The color grade of the tangerine peel identified by the machine learning model for each set of feature data is used as the output. The color grade of the tangerine peel in the label data corresponding to each set of feature data is used as the recognition target. The training target is to minimize the sum of all recognition differences. The calculation formula for the recognition difference is: Ai=(Bi-Ci) 2 , where Ai is the recognition difference, the recognition difference is the difference between the color grade recognized by the model and the actual color grade, Bi is the recognized color grade corresponding to the i-th group of feature data, Ci is the color grade in the i-th group of label data, the machine learning model is trained until the sum of the recognition differences reaches convergence and the training is stopped. It should be noted that the convergence standard is set by those skilled in the art according to the specific model training situation, and the color grade is classified by those skilled in the art according to the color of the tangerine peel itself, for example, the color grade can be divided into general grade tangerine peel, medium grade tangerine peel and preferred grade tangerine peel;

[0094] The control module collects the image data of the tangerine peel on the tangerine peel transport mechanism 1 in real time through the image acquisition module 6, identifies the color grade of the tangerine peel image data based on the machine learning model, and then uses the sorting mechanism 4 to sort the tangerine peel on the tangerine peel transport mechanism 1 based on the color grade, so that the tangerine peel enters the grading conveying trough 5, thereby realizing the automated grading and sorting of the tangerine peel, and greatly improving the efficiency and effect of the automated grading and sorting of the tangerine peel.

[0095] Example 2

[0096] like Figure 1-Figure 12 As shown, the tangerine peel transport mechanism 1 includes a transport frame 11, one end of the transport frame 11 is equipped with a driving conveying wheel 12 through a bearing, the other end of the transport frame 11 is equipped with a driven conveying wheel 13 through a bearing, and the sides of the driving conveying wheel 12 and the driven conveying wheel 13 are embedded with a transport belt 14. A built-in support seat 15 is installed on the inner side of the transport frame 11, and a plurality of equidistant support rollers 16 are installed on the top surface of the built-in support seat 15. One end of the driving conveying wheel 12 is connected to a driving motor 2 through a coupling. It should be noted that the driving motor 2 and the driving conveying wheel 12 are connected through a coupling. The gear box is connected, and the output shaft of the driving motor 2 can convert the rotational speed force into torque force through the gear box, so that the active conveying wheel 12 rotates, and the rotating active conveying wheel 12 can drive the side conveying belt 14 to rotate, and the rotating conveying belt 14 can continuously transport the tangerine peel, so that the tangerine peel can be transported to the bottom of the sorting mechanism 4, and the sorting mechanism 4 can automatically sort the individual tangerine peels, and the built-in support seat 15 and the support roller 16 installed on the inner side of the transport frame 11 can be supported on the bottom surface of the conveying belt 14, so that the conveying belt 14 is more stable when transporting the tangerine peel.

[0097] like Figure 5 , Figure 6 and Figure 7 As shown, the tangerine peel delivery mechanism 3 includes a delivery shell 31, a delivery slot 32 is provided inside the delivery shell 31, a guide plate 33 is provided on the bottom surface of the delivery slot 32, a paving channel 34 is provided on the bottom of the delivery shell 31, a hydraulic cylinder 35 is rotatably connected to the inside of the delivery shell 31, a rotating plate 36 is rotatably connected to the output end of the hydraulic cylinder 35, and a leveling brush 37 is connected to one end of the rotating plate 36. It should be noted that the guide plate 33 is tilted and set, and the guide plate 33 is movably installed on the bottom surface of the delivery slot 32. In addition, a vibrator is provided on the bottom surface of the guide plate 33, so that the vibrator can make the guide plate 33 vibrate to ensure the smoothness of the delivery. In order to ensure that the dried tangerine peels in the delivery slot 32 can smoothly enter the interior of the flattening channel 34, the output shaft of the hydraulic cylinder 35 can be used to act on the rotating plate 36. The end of the rotating plate 36 away from the flattening brush 37 is rotatably installed inside the delivery shell 31. The hydraulic cylinder 35 can control the angle of the flattening brush 37 through the output shaft, so that the flattening brush 37 scrapes the top surface of the dried tangerine peels so that the dried tangerine peels are spread flat on the top surface of the conveyor belt 14. It should also be noted that the top surface of the conveyor belt 14 is provided with an isolation groove 17, which can ensure that the dried tangerine peels are spread flat on the top surface of the conveyor belt 14 in an orderly manner, so that the sorting mechanism 4 can perform sorting operations.

[0098] Example 3

[0099] like Figure 8-Figure 12 As shown, the number of sorting mechanisms 4 is at least three groups, and the three groups of sorting mechanisms 4 are equidistantly arranged on the top surface of the tangerine peel transport mechanism 1. A grading conveying trough 5 is fixedly connected to the side of the tangerine peel transport mechanism 1 near the sorting mechanism 4. The grading conveying trough 5 is provided with inclined slots the same in number as the sorting mechanisms 4. When the sorting mechanism 4 transports tangerine peel of corresponding color grade to the inclined slots, the inclined slots can further transport individual tangerine peels. It should also be noted that a storage box for storing tangerine peels can be provided on the other side of the inclined slots.

[0100] The sorting mechanism 4 includes a transverse track 41, a support column 45 is fixedly connected to the middle position of the bottom of the transverse track 41, the bottom end of the support column 45 is fixedly connected to the top surface of the tangerine peel transport mechanism 1, moving seats 42 are installed on both sides of the transverse track 41, the bottom surface of the moving seat 42 is fixedly connected to the driving mechanism 43, the bottom surface of the driving mechanism 43 is connected to the adsorption mechanism 44, a fixed cross bar 411 is arranged inside the transverse track 41, screws 412 are installed on both sides of the fixed cross bar 411 through bearings, a ball nut 421 is fixedly connected to the inner end of the moving seat 42 near the screw 412, a docking block 431 is fixedly connected to one end of the top surface of the driving mechanism 43, a positioning groove 432 is provided at the bottom end of the driving mechanism 43, and a positioning groove 432 is provided on the bottom surface of the moving seat 42 to match the docking block 431 The driving mechanism 43 has a matching docking groove, and a hydraulic mechanism is installed inside. It should be noted that the moving seat 42 can slide freely in the specified direction in the transverse track 41. It should also be noted that a motor can be set at one end of the transverse track 41. The output shaft of the motor can be transmitted to the screw 412 through a coupling, so that the screw 412 can rotate freely. The rotating screw 412 cooperates with the ball nut 421 to realize the movement of the moving seat 42. The end face of the moving seat 42 can be matched with the slide groove of the fixed cross bar 411, so that the moving seat 42 can move stably on the side of the transverse track 41. The driving mechanism 43 can be fixed to the bottom surface of the moving seat 42 through the docking block 431, so that the moving seat 42 can drive the driving mechanism 43 to move synchronously during the movement.

[0101] The adsorption mechanism 44 includes an adsorption seat 441, a matrix adsorption plate 442 is installed on the bottom surface of the adsorption seat 441, a positioning block 443 that fits with the positioning groove 432 is fixedly connected to the top surface of the adsorption seat 441, and a lifting rod 444 is fixedly connected to the top surface of the positioning block 443. It should be noted that a negative pressure pump is arranged inside the adsorption seat 441, and the input end of the negative pressure pump is connected to the matrix adsorption plate 442 on the bottom surface of the adsorption seat 441. However, a solenoid valve is arranged in the channel between the negative pressure pump and the matrix adsorption plate 442, so that the adsorption force of the matrix adsorption plate 442 can be freely controlled, and the color grade of the individual tangerine peel on the top surface of the tangerine peel transport mechanism 1 is identified through the control module, and the three groups of sorting mechanisms 4 arranged on the top surface of the tangerine peel transport mechanism 1 correspond to the general grade tangerine peel, the medium grade tangerine peel and the preferred grade tangerine peel respectively. In this way, the control module can identify the tangerine peel according to the identified color grade. The color grade controls different sorting mechanisms 4 to adsorb the corresponding tangerine peel individuals, so that the tangerine peel individuals are transported to the corresponding grading conveying trough 5. The specific process is to open the hydraulic mechanism inside the driving mechanism 43. The hydraulic mechanism can be a hydraulic push rod and can also be replaced by an electric push rod or a pneumatic push rod. The main purpose is to control the height of the adsorption seat 441 through the lifting rod 444, so that the matrix adsorption plate 442 on the bottom surface of the adsorption seat 441 contacts the tangerine peel individuals on the top surface of the tangerine peel transport mechanism 1, and then open the matrix adsorption plate 442 at the corresponding position to generate adsorption force on the corresponding tangerine peel individuals, and then adsorb the tangerine peel individuals, and then transport them to the top surface of the grading conveying trough 5, and then close the matrix adsorption plate 442 at the corresponding position, so that the tangerine peel on the bottom surface of the matrix adsorption plate 442 automatically falls on the top surface of the grading conveying trough 5, thereby realizing the automatic sorting of tangerine peel individuals.

[0102] Example 4

[0103] like Figure 1-Figure 13 As shown, the present invention discloses a sorting control method for automatic grading of tangerine peel, which is implemented based on a machine vision system for automatic grading of tangerine peel and includes the following steps:

[0104] S1: Put the ungraded tangerine peel into the tangerine peel delivery mechanism 3, and the tangerine peel delivery mechanism 3 spreads the tangerine peel on the top surface of the tangerine peel transportation mechanism 1. It should be noted that the vibrator arranged on the bottom surface of the guide plate 33 is turned on, and the vibrator can make the guide plate 33 produce a vibration effect to ensure that the tangerine peel in the delivery slot 32 can smoothly enter the interior of the paving channel 34, and the driving motor 2 is turned on. The output shaft of the driving motor 2 can convert the speed force into torque force through the gear box to rotate the active conveying wheel 12, and the rotating active conveying wheel 12 can drive the side conveying belt 14 to rotate, and the rotating conveying belt 14 can continuously transport the tangerine peel. The output shaft of the hydraulic cylinder 35 can be used to act on the rotating plate 36, and the end of the rotating plate 36 away from the leveling brush 37 is rotatably installed in the delivery shell 31. The hydraulic cylinder 35 can control the angle of the leveling brush 37 through the output shaft, so that the leveling brush 37 scrapes the top surface of the tangerine peel so that the tangerine peel is spread on the top surface of the conveying belt 14;

[0105] S2: The image acquisition module 6 acquires the image data of the tangerine peel on the top surface of the tangerine peel transport mechanism 1 in real time. It should be noted that the image acquisition module 6 may be a high-definition camera for acquiring high-definition images of the tangerine peel on the top surface of the tangerine peel transport mechanism 1, that is, the tangerine peel image data;

[0106] S3: The control module uses the CNN model to identify the individual characteristic pictures of the tangerine peel in the tangerine peel image data collected in real time by the image acquisition module 6. The training data of the CNN model is collected by those skilled in the art in the process of using the machine vision system, that is, a certain amount of tangerine peel image data is collected. The specific amount of tangerine peel image data is adaptively selected by those skilled in the art according to the training situation of the CNN model to ensure that the CNN model can identify all the individual characteristic pictures of the tangerine peel in the tangerine peel image data. It should be noted that, usually, one tangerine peel image data contains several individual characteristic pictures of the tangerine peel, so the CNN model needs to identify the individual characteristic pictures of the tangerine peel in the tangerine peel image data, and one individual characteristic picture of the tangerine peel should only contain one tangerine peel individual, so as to ensure that the subsequent sorting of the tangerine peel individuals is more accurate.

[0107] S4: Establish a plane rectangular coordinate system based on the tangerine peel image data, mark the individual characteristic picture of the tangerine peel in the tangerine peel image data and obtain the central coordinates of the individual characteristic picture of the tangerine peel. First, extract the contour of the tangerine peel individual, the contour is determined by the cv2.findContours function, and then calculate the geometric moment of the tangerine peel individual. The calculation formula of the geometric moment is:

[0108] ;

[0109] in is the p+q order moment of the individual characteristic image of tangerine peel, I (x, y) is the pixel value, and the zero-order moment is calculated based on the calculation formula of the geometric moment , first-order moment and , and then calculate the centroid coordinates ( , ):

[0110] ; ;

[0111] Center of mass coordinates ( , ) is the central coordinate of the individual characteristic image of tangerine peel.

[0112] Based on the individual characteristic pictures of tangerine peel, the color parameters of tangerine peel are collected. The color parameters include the individual area, individual shape, individual color and individual surface texture of tangerine peel, among which:

[0113] The extraction method of individual area of ​​tangerine peel is:

[0114] Based on the individual characteristic pictures of tangerine peel, the Canny edge detection algorithm is used to extract the edge contour of the tangerine peel. The Canny edge detection algorithm uses the Sobel operator of the image to calculate the horizontal and vertical gradients:

[0115] , ;

[0116] in is the pixel value of the individual characteristic image of tangerine peel, is the horizontal gradient, is the vertical gradient. Based on the horizontal and vertical gradients, the gradient magnitude and direction of each pixel are calculated:

[0117] ,θ( ) = arctan( / );

[0118] in is the gradient magnitude of the pixel point (x, y), θ( ) is the direction of the pixel (x, y);

[0119] After obtaining the edge contour of the individual tangerine peel in advance through the Canny edge detection algorithm, the cv2.findContours function is used to extract the contour from the binary edge map, and then the cv2.contourArea function is used to directly calculate the area of ​​each contour, which is the area of ​​the individual tangerine peel.

[0120] Alternatively, based on the individual characteristic picture of tangerine peel, a computer is used to identify the region of the tangerine peel individual in the individual characteristic picture of tangerine peel, and then the regional area of ​​the tangerine peel individual is automatically generated by the computer, and the regional area is used as the individual tangerine peel area.

[0121] The individual shape of the tangerine peel is determined by extracting the circularity, eccentricity or Hu moment of the individual tangerine peel in the individual characteristic picture of the tangerine peel based on the circularity, eccentricity or Hu moment. It should be noted that when the individual shape of the tangerine peel is determined using the circularity, the following steps are followed:

[0122] C 圆形度 =4πA / P 2 ;

[0123] Among them, C 圆形度 is the roundness of individual tangerine peel, A is the area of ​​individual tangerine peel in the individual characteristic picture of tangerine peel, P is the circumference of individual tangerine peel in the individual characteristic picture of tangerine peel, and the circumference is calculated using the cv2.arcLength algorithm. The closer the roundness is to 1, the closer the individual tangerine peel in the individual characteristic picture of tangerine peel is to a circle. The technicians in this field preset the range of roundness and determine the goodness of the shape of individual tangerine peel according to the range of roundness.

[0124] The calculation formula of eccentricity is:

[0125] ;

[0126] It should be noted that is the eccentricity, a and b are respectively the length and width of the individual tangerine peel in the individual characteristic picture of the tangerine peel. The length and width are obtained by a technician in this field using a computer, and then the eccentricity is directly calculated by the above formula. The value range of the eccentricity is [0,1]. A value close to 0 indicates a shape close to a circle, and a value close to 1 indicates a shape close to a straight line or a slender shape. The value range of the eccentricity is preset by a technician in this field, and the goodness of the shape of the individual tangerine peel is determined based on the value range of the eccentricity.

[0127] The Hu moment is to calculate the geometric moment of individual tangerine peel. The specific calculation method of the geometric moment of individual tangerine peel has been given above and will not be repeated here. The Hu moment is combined with a clustering or classification algorithm (such as KNN or SVM) to determine the shape category. The technicians in this field determine the goodness of the shape of the individual tangerine peel based on the obtained shape category.

[0128] Generally speaking, the closer the individual shape of the tangerine peel is to a circle, the better the shape of the tangerine peel is, and vice versa. The technical personnel in this field will make adaptive adjustments based on the individual shape of the tangerine peel.

[0129] The method for extracting the individual color of tangerine peel is:

[0130] Grayscale processing is performed on the individual characteristic image of tangerine peel to form a grayscale image, and the grayscale value of the grayscale image is extracted. It should be noted that the grayscale value has a value range of [0, 255], and the grayscale value range is preset by a person skilled in the art;

[0131] The individual color of the tangerine peel is determined based on the preset gray value range, and the goodness of the individual color of the tangerine peel is adaptively set by those skilled in the art based on experience.

[0132] According to the individual characteristic picture of tangerine peel, the individual surface texture of tangerine peel is extracted based on the gray level co-occurrence matrix algorithm. It should be noted that the method for determining the individual surface texture of tangerine peel is:

[0133] Step 1: Obtain the individual characteristic image of tangerine peel, and then convert the individual characteristic image of tangerine peel into a grayscale image; since the gray-level co-occurrence matrix algorithm can only work on grayscale images, the input color image needs to be converted into a grayscale image;

[0134] Step 2: Use Gaussian blur to smooth the grayscale image;

[0135] Step 3: Grayscale quantization. For example, the grayscale value of the individual feature image of tangerine peel can be reduced from [0,255] to [0,15];

[0136] Step 4: Calculate the gray level co-occurrence matrix, extract the contrast, energy, homogeneity and correlation of the gray level image based on the gray level co-occurrence matrix, and then construct the contrast, energy, homogeneity and correlation into a feature matrix;

[0137] Step 5: grouping or classifying the surface textures of tangerine peel using a classification algorithm (such as KNN, SVM, etc.) based on the feature matrix, and the technicians in this field determine the goodness of the individual tangerine peels based on their experience and the surface textures of the tangerine peels;

[0138] S5: The control module trains a machine learning model for identifying the color grade of the tangerine peel image data in the following manner:

[0139] Collect a historical training data set in advance, the historical training data set includes N groups of training data, N is a positive integer, each group of training data includes feature data and label data, the feature data is the color parameter, and the label data is the color grade;

[0140] The label data is the color grade of the tangerine peel on the top surface of the tangerine peel transport mechanism 1 when collecting each set of training data;

[0141] Each set of feature data in the historical training data is used as the input of the machine learning model. The color grade of the tangerine peel identified by the machine learning model for each set of feature data is used as the output. The color grade of the tangerine peel in the label data corresponding to each set of feature data is used as the recognition target. The training target is to minimize the sum of all recognition differences. The calculation formula for the recognition difference is: Ai=(Bi-Ci) 2 , where Ai is the recognition difference, Bi is the recognized color grade corresponding to the i-th group of feature data, Ci is the color grade in the i-th group of label data, and the machine learning model is trained until the sum of the recognition differences reaches convergence and the training is stopped. The convergence standard is set by those skilled in the art according to the specific model training situation, and the color grade is classified by those skilled in the art according to the color of the tangerine peel itself. For example, the color grade can be divided into general grade tangerine peel, medium grade tangerine peel and preferred grade tangerine peel.

[0142] S6: The control module collects the tangerine peel image data on the tangerine peel transport mechanism 1 in real time through the image acquisition module 6, and identifies the color grade of the tangerine peel image data based on the machine learning model;

[0143] S7: The control module uses the sorting mechanism 4 to sort the central coordinates of the individual tangerine peels on the tangerine peel transport mechanism 1 based on the color grade, so that the individual tangerine peels enter the grading conveying trough 5. The specific process is to open the hydraulic mechanism inside the driving mechanism 43. The hydraulic mechanism can be a hydraulic push rod and can also be replaced by an electric push rod or a pneumatic push rod. The main purpose is to control the height of the adsorption seat 441 through the lifting rod 444, so that the matrix adsorption plate 442 on the bottom surface of the adsorption seat 441 contacts the tangerine peel on the top surface of the tangerine peel transport mechanism 1, and then open the matrix adsorption plate 442 at the corresponding position to generate adsorption force on the corresponding tangerine peel individual, and then adsorb the tangerine peel, and then transport it to the top surface of the grading conveying trough 5, and then close the matrix adsorption plate 442 at the corresponding position, so that the tangerine peel on the bottom surface of the matrix adsorption plate 442 automatically falls on the top surface of the grading conveying trough 5, thereby realizing automatic sorting of tangerine peel.

[0144] The above is a detailed description of an embodiment of the present invention, but the content is only a preferred embodiment of the present invention and cannot be considered to limit the scope of implementation of the present invention. All equivalent changes and improvements made within the scope of the present invention should still fall within the scope of the patent coverage of the present invention.

Claims

1. A machine vision system for automated grading of dried tangerine peel, characterized in that: include: An image acquisition module (6), which is used to acquire tangerine peel image data; Use CNN model to identify individual characteristic images of tangerine peel in tangerine peel image data; Establishing a plane rectangular coordinate system based on the tangerine peel image data, marking the tangerine peel individual characteristic picture in the tangerine peel image data and obtaining the central coordinates of the tangerine peel individual characteristic picture; Based on the individual characteristic pictures of tangerine peel, the color parameters of individual tangerine peel are collected; A control module, which is data-connected to the image acquisition module (6), the control module pre-collects a historical training data set, and trains a machine learning model for identifying the color grade of individual tangerine peels based on the historical training data set; The control module collects tangerine peel image data in real time through the image acquisition module (6), identifies the color grade of the tangerine peel image data based on a machine learning model, and uses a sorting mechanism (4) to sort the tangerine peel individuals on the tangerine peel transport mechanism (1) based on the color grade and the center coordinates.

2. The machine vision system for automated grading of dried tangerine peel according to claim 1, characterized in that: The coloring parameters include individual area of ​​tangerine peel, individual shape of tangerine peel, individual color of tangerine peel and individual surface texture of tangerine peel.

3. The machine vision system for automated grading of dried tangerine peel according to claim 1, characterized in that: The tangerine peel transport mechanism (1) spreads out individual tangerine peels, and the image acquisition module (6) collects tangerine peel image data of the individual tangerine peels spread out; The tangerine peel image data includes a plurality of individual characteristic pictures of tangerine peel.

4. The machine vision system for automated grading of dried tangerine peel according to claim 1, characterized in that: The number of the sorting mechanisms (4) is at least three groups, and the three groups of sorting mechanisms (4) are equidistantly arranged on the top surface of the tangerine peel transport mechanism (1), and a grading conveying trough (5) is fixedly connected to the side of the tangerine peel transport mechanism (1) near the sorting mechanism (4), and the color grades correspond to the grading conveying troughs (5) one by one.

5. The machine vision system for automated grading of dried tangerine peel according to claim 4, characterized in that: The sorting mechanism (4) absorbs the tangerine peel individuals based on the central coordinates of the tangerine peel individuals; The sorting mechanism (4) moves the individual tangerine peels into the grading conveying trough (5) based on the color grade of the individual tangerine peels.

6. The machine vision system for automated grading of dried tangerine peel according to claim 2, characterized in that: The extraction method of the individual area of ​​tangerine peel is: Based on the individual characteristic images of tangerine peel, the Canny edge detection algorithm is used to extract the edge contours of tangerine peel individuals; The individual area of ​​tangerine peel is calculated based on the edge contour using the contour finding function in OpenCV.

7. The machine vision system for automated grading of dried tangerine peel according to claim 2, characterized in that: The individual shape of the tangerine peel is determined by extracting the circularity, eccentricity or Hu moment of the individual tangerine peel in the individual characteristic picture of the tangerine peel, and determining the individual shape of the tangerine peel based on the circularity, eccentricity or Hu moment.

8. The machine vision system for automated grading of dried tangerine peel according to claim 2, characterized in that: The extraction method of the individual color of tangerine peel is: Grayscale processing is performed on the individual characteristic image of tangerine peel to form a grayscale image; Extract the grayscale value of the grayscale image; Determine the individual color of tangerine peel based on the grayscale value.

9. The machine vision system for automated grading of dried tangerine peel according to claim 2, characterized in that: According to the individual characteristic picture of dried tangerine peel, the individual surface texture of the dried tangerine peel is extracted based on the gray level co-occurrence matrix algorithm.

10. A sorting control method for automated grading of dried tangerine peel, the sorting control method being implemented based on the machine vision system for automated grading of dried tangerine peel according to any one of claims 1 to 9, characterized in that: The following steps are involved: S1: placing ungraded tangerine peel into the tangerine peel delivery mechanism (3), and the tangerine peel delivery mechanism (3) spreads the tangerine peels flat on the top surface of the tangerine peel transport mechanism (1); S2: the image acquisition module (6) acquires the image data of the tangerine peel on the top surface of the tangerine peel transport mechanism (1) in real time; S3: The control module uses the CNN model to identify individual characteristic images of the tangerine peel in the tangerine peel image data collected in real time by the image collection module (6); S4: establishing a plane rectangular coordinate system based on the tangerine peel image data, marking the individual characteristic pictures of the tangerine peel in the tangerine peel image data and obtaining the central coordinates of the individual characteristic pictures of the tangerine peel, and collecting the color parameters of the individual tangerine peel based on the individual characteristic pictures of the tangerine peel; S5: The control module trains a machine learning model for identifying the color grade of the tangerine peel image data in the following manner: Collect a historical training data set in advance, the historical training data set includes N groups of training data, N is a positive integer, each group of training data includes feature data and label data, the feature data is the color parameter, and the label data is the color grade; The label data is the color grade of the tangerine peel on the top surface of the tangerine peel transportation mechanism (1) when collecting each set of training data; Each set of feature data in the historical training data is used as the input of the machine learning model. The color grade of the tangerine peel identified by the machine learning model for each set of feature data is used as the output. The color grade of the tangerine peel in the label data corresponding to each set of feature data is used as the recognition target. The training target is to minimize the sum of all recognition differences. The calculation formula for the recognition difference is: Ai=(Bi-Ci) 2 , where Ai is the recognition difference, the recognition difference is the difference between the color grade recognized by the model and the actual color grade, Bi is the recognized color grade corresponding to the i-th group of feature data, Ci is the color grade in the i-th group of label data, and the machine learning model is trained until the sum of the recognition differences reaches convergence and the training is stopped; S6: The control module collects the image data of the tangerine peel on the tangerine peel transport mechanism (1) in real time through the image acquisition module (6), and identifies the color grade of the tangerine peel image data based on the machine learning model; S7: The control module uses the sorting mechanism (4) to sort the central coordinates of the individual tangerine peels on the tangerine peel transport mechanism (1) based on the color grade, so that the individual tangerine peels enter the grading conveying trough (5).

Images