A visual grading method and grading production line for pear appearance quality

Abstract

The present invention is a method for visually grading the appearance quality of pears and a grading production line. The method can reflect the fruit shape, color, defect type and defect area of ​​the sample through characteristic values, and select a suitable machine learning model as a classifier according to the characteristics of the features to make accurate predictions for the above-mentioned various types of information. Finally, the information is input into a decision tree for decision-making, simulating the decision-making classification process of people to obtain the grading results. The whole grading process uses a multi-model fusion method to process various types of information in a targeted manner to avoid the occurrence of overfitting, so that the processing results are more accurate, the detection efficiency and detection accuracy are improved, and the requirements for fast and accurate detection on the assembly line are met. The method is used for the rapid grading of pears on a circular production line. The pears are graded by feature fusion, and the recognition rate is expected to reach more than 95%. It is suitable for grading the appearance quality of different types of pears.

Description

Technical Field

[0001] The invention relates to the technical field of pear appearance quality detection and grading, and in particular to a pear appearance quality visual grading method and a grading production line. Background Art

[0002] Fruit appearance quality inspection and automatic grading are of great significance to improving the market competitiveness and profit level of fresh fruits. As one of the most common fruits, it is very necessary to inspect the appearance quality of pears. At present, the main method for inspecting the appearance quality of pears is manual inspection, which consumes a lot of human resources and has a strong subjectivity in grading, and has low accuracy and efficiency. Machine vision inspection has outstanding advantages such as non-contact, non-destructive, high degree of automation, and safety and reliability. It has been successfully applied to the appearance inspection of citrus and apples. Therefore, it is feasible to use machine vision for pear appearance quality inspection.

[0003] The National Standard of the People's Republic of China GB / T 10650-2008 "Fresh Pears" is currently in effect and divides the appearance quality of pears into three grades: superior, first and second grade. The grading is based on the shape, color, single fruit weight, fruit surface defects and other characteristics of the pear. Among them, the type and area of ​​fruit surface defects are the most important indicators in the grading standard. Therefore, the type and area of ​​defects on the surface of the pear can be visually detected, and then graded according to the national standard.

[0004] The document of patent number CN201621138384.4 discloses a roxburghii grader, comprising an upstream group conveying roller, a downstream group conveying roller and a motor driving the conveying roller to rotate, the distance between two adjacent rollers in the upstream group conveying roller is 20 mm, the distance between two adjacent rollers in the downstream group conveying roller is 35 mm, and a collecting trough is provided below the upstream group conveying roller and the downstream group conveying roller. The device has a simple structure and high efficiency. However, the device only grades roxburghii according to the size characteristics of the roxburghii, the grading accuracy is low, and it cannot meet the demand for grading the appearance quality of the pear.

[0005] The existing grading device can only grade one pear in the same time period and cannot achieve efficient and continuous grading. The pears are pushed by baffles, causing them to slide directly off the platform. Collisions may occur during the process, causing secondary damage to the pears. In addition, the existing grading device does not clearly indicate the grading standards, and it is impossible to determine whether the grading meets national standards. The grading accuracy needs to be improved. Summary of the invention

[0006] In view of the gaps in existing research directions and technical deficiencies, the technical problem that the present invention intends to solve is to provide a method for visual grading of pear appearance quality and a grading production line.

[0007] The technical solution of the present invention to solve the technical problem is to provide a method for visually grading the appearance quality of pears, which includes the following contents:

[0008] Acquire color images of pear fruits to be graded to obtain sample pear images, and a large number of sample pear images form a sample pear image library;

[0009] The sample pear images are processed to extract the pear outer surface images, and a large number of pear outer surface images form a pear outer surface image database;

[0010] Obtaining the Fourier descriptor of the pear outer surface image in the pear outer surface image database, using the Fourier descriptor as input and the actual evaluation of the pear surface contour as output, training an artificial neural network model to obtain classifier 1;

[0011] Extract the B, G and R components of the pear outer surface image in the pear outer surface image database, calculate the second-order moments of the B, G and R component matrices, and obtain a 3×1-dimensional feature matrix. Take the feature matrix as input and the actual evaluation of the pear surface color as output to train a naive Bayes model and obtain classifier 2.

[0012] Performing brightness correction on the pear outer surface images in the pear outer surface image database to obtain brightness corrected images of sample pears, and a large number of brightness corrected images form a brightness corrected image library;

[0013] The brightness correction images in the brightness correction image library are segmented to obtain the defect images of the sample pears, and a large number of the defect images of the sample pears form a defect image library;

[0014] Extract the Hu invariant moment, Fourier descriptor, and gray-level co-occurrence moment of the defect image in the defect image library, take these three features as input, and take the actual type of the defect as output to train another artificial neural network to obtain classifier 3; at the same time, calculate the area of ​​the surface defect in each defect image;

[0015] The results of classifier 1, the results of classifier 2, and the number of all defect types of classifier 3 are sequentially formed into a decision vector. If the number of all defect types of classifier 3 is x, then the length of the decision vector is x+2; in the decision vector, the position corresponding to the number of defect types is filled with the area of ​​the surface defect of the sample pear corresponding to the defect type determined by the sample, and the other defect type positions are filled with 0; with the decision vector as input and the actual level of the sample pear according to the national standard as output, the decision tree is trained to obtain classifier 4;

[0016] The trained classifiers 1, 2, 3 and 4 are used to grade the appearance quality of pears.

[0017] This method is used for rapid grading of pears on a circular production line. The recognition rate of pears can reach over 95% by grading them through feature fusion. It is suitable for grading the appearance quality of different types of pears.

[0018] A production line for visual grading of pear appearance quality, characterized in that the production line comprises a conveyor belt mechanical structure, a host computer, an image acquisition system, a conveyor belt driving system, a grading execution system, a PLC and a communication system;

[0019] The conveyor belt mechanical structure is the main mechanical structure of the production line, which plays the role of transporting sample pears to different functional areas. The entire conveyor belt mechanical structure forms a circular reflux conveyor belt;

[0020] The conveyor belt drive system is used to drive the start, stop, deceleration, acceleration and uniform speed of the circular return flow conveyor belt;

[0021] The host computer directly controls the CCD industrial camera in the image acquisition system to realize image acquisition, which is used to obtain the outer surface image of the pear; the host computer communicates with the PLC through the communication system, sends instructions to the PLC, and makes the PLC control the conveyor belt drive system and the grading execution system to achieve the purpose of controlling the entire production line;

[0022] The upper computer is loaded with the visual grading method for the appearance quality of pears. The grading results are displayed on the upper computer screen and sent to the PLC at the same time.

[0023] The grading execution system includes a push-pull electromagnet, an electromagnet fixing frame and a grading trough. The push-pull electromagnet and the electromagnet fixing frame are fixed on the reflux conveyor belt behind the image acquisition system. The grading trough has multiple channels, and shock-absorbing materials are pasted inside. The grading trough is placed on the opposite side of the push-pull electromagnet. Each push-pull electromagnet is aligned with a channel, and each channel corresponds to a grading level, which is recorded as a grading area. The push-pull electromagnet is connected to the PLC. After receiving the grading result sent by the upper computer, the PLC enables the push-pull electromagnet of the corresponding level, and then sends the sample pear into the designated grading area.

[0024] The entire conveyor belt mechanical structure is composed of two linear conveyor belts and two steering gears to form an annular return conveyor belt. The two linear conveyor belts and two steering gears are driven by a stepper motor respectively. Multiple stepper motors are controlled by PLC to achieve synchronization. The two linear conveyor belts and two steering gears form an annular return conveyor belt through a transmission mechanism. The entire annular conveyor belt runs synchronously. The entire annular conveyor belt is supported by a bracket, and the image acquisition system, conveyor belt drive system and grading execution system are also installed on the bracket.

[0025] The host computer is a GUI interface made of python, and the interface includes function buttons, information display and image display areas. Users can use the interface to control the production line, obtain equipment information, view collected images and grading results; the host computer directly controls the CCD industrial camera in the image acquisition system to achieve image acquisition, which is used to obtain the outer surface image of the pear; the host computer communicates with the PLC through the communication system, sends instructions to the PLC, and enables the PLC to control the conveyor belt drive system and the grading execution system to achieve the purpose of controlling the entire device; the outer surface image of the collected pear is preprocessed, feature extracted, machine learning classified and other operations in the host computer to obtain the grading result of the sample pear, which is displayed in the display bar and sent to the PLC through the communication system;

[0026] The image acquisition system is used to collect the image of the outer surface of the sample pear. The system includes a CCD industrial camera, a lens, an LED ring light source, an image acquisition box and a power adapter. The power adapter supplies power to the image acquisition system so that the ring light source works normally, and the ring light source provides uniform brightness for the image acquisition system; the image acquisition box is placed on the bracket of the conveyor belt, and the frame of the image acquisition box is fastened to the bracket of the first straight conveyor belt by screws; a black acrylic plate is installed on the surface of the frame to isolate external light and reduce interference during image acquisition; the CCD industrial camera uses a color digital CCD industrial camera to obtain more features of the surface of the sample pear, and the CCD industrial camera is connected to the lens to obtain a clear image within an effective distance; the interface of the industrial camera is a gigabit network port, which is connected to the network cable interface of the host computer using a network cable;

[0027] The conveyor belt drive system includes a motor controller, and a PLC is used as the controller of the drive system to realize the logic control and timing control of the circuit; the motor controller is connected to the PLC, and after receiving the driving instruction sent by the host computer, the PLC provides a command signal to the motor controller, and the motor controller generates a waveform for driving the stepper motor and provides an effective voltage to realize the control of the stepper motor;

[0028] The grading execution system includes a push-pull electromagnet, an electromagnet fixing frame and a grading slot, wherein the push-pull electromagnet and the electromagnet fixing frame are fixed on the first linear conveyor belt; the grading slot is a slot-shaped area made of an acrylic plate, the grading slot has multiple channels, and a shock-absorbing material is pasted inside to buffer collisions, and the grading slot is placed on the opposite side of the push-pull electromagnet, each push-pull electromagnet is aligned with a channel, and each channel corresponds to a grading level, which is recorded as a grading area; the push-pull electromagnet is connected to the PLC, and after receiving the grading result sent by the upper computer, the PLC enables the push-pull electromagnet of the corresponding level, and then sends the sample pear into the designated grading area;

[0029] The communication system uses wireless communication methods such as ZigBee, Bluetooth, and wifi. When ZigBee is used, the computer where the host computer is located and the PLC are mounted with ZigBee communication modules, and the communication between the host computer and the PLC is realized through the ZigBee protocol. The communication content is the drive instructions sent by the host computer to the PLC, the classification results, and the image acquisition instructions sent by the PLC to the host computer;

[0030] The CCD camera model is MV-EM200C, and the photosensitive area of ​​the CCD camera is 4.8mm×3.6mm; the lens model is HX2X-T1, which is a zoom lens with a focal length of 12mm-36mm and a minimum object distance of 20cm; the LED ring light model is HDR-90-45; the PLC model is Siemens S7-200SMART, the CPU is ST40, the input I / O is 24, and the output I / O is 16; the motor controller model is TB6600, with a rated current of 4.2A and a subdivision number of 6400; the stepper motor model is 57BYG250C, with a step angle of 1.8° and a torque of 1.8N·m; the push-pull electromagnet has a stroke of 60mm and a thrust of 100N.

[0031] Compared with the prior art, the present invention has the following beneficial effects:

[0032] (1) The grading idea of ​​the grading method of the present invention is applicable to the national standard GB / T 10650-2008 "Fresh Pears", and can reflect the fruit shape, color, defect type and defect area of ​​the sample through the feature value, and select a suitable machine learning model as a classifier according to the characteristics of the feature to make accurate predictions on the above-mentioned various types of information, and finally input the information into the decision tree used for the decision-making level, simulate the human decision-making classification process, and obtain the grading result; the whole grading process uses the multi-model fusion method to process various types of information in a targeted manner, avoid the occurrence of overfitting, make the processing result more accurate, improve the detection efficiency and detection accuracy, and meet the requirements of fast and accurate detection on the assembly line.

[0033] (2) The grading production line of the present invention adopts a grading mechanism pushed by a horizontally arranged push-pull electromagnet, which is simple to control, quick to execute, and improves the grading efficiency. It is suitable for actual working environments that have certain requirements on the grading speed, and buffer materials are installed in the push-pull electromagnet and the grading area to effectively reduce the secondary mechanical damage of the fruit; the rollers of the conveyor belt are fixed with channel steel, so that the tightness of the conveyor belt can be adjusted, keeping the conveyor belt running stably for a long time, thereby improving production efficiency; the collection box uses black acrylic board as the box wall to ensure that the image collection is not affected by external light, further ensuring the accuracy of the collected image.

[0034] (3) The grading production line of the present invention integrates mechanical transmission, sensor detection, embedded, image processing and pattern recognition technologies, has complete functions, and can realize a complete production process.

[0035] (4) The grading method of the present invention can obtain a complete and clear defect area of ​​the pear appearance, fully consider the influence of the features obtained by visual inspection on the recognition effect (mainly accuracy) of the neural network algorithm, and the demand for rapid inspection of the production line, and can identify the more complex appearance information of the pear, including interference factors such as its own color, uniformity and surface spots. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] Figure 1 It is a schematic diagram of the overall structure of the production line for visual grading of pear appearance quality of the present invention;

[0037] Figure 2 This is a front view of the overall structure of the production line for visual grading of pear appearance quality according to the present invention;

[0038] Figure 3 It is a side view of the overall structure of the production line for visual grading of pear appearance quality of the present invention;

[0039] Figure 4 It is a top view of the overall structure of the production line for visual grading of pear appearance quality of the present invention;

[0040] Figure 5 A schematic diagram of the installation position of the image acquisition system used in the pear appearance quality visual grading production line of the present invention;

[0041] Figure 6 It is a schematic diagram of a grading execution system used in a pear appearance quality visual grading production line according to the present invention;

[0042] Figure 7 It is a schematic diagram of the installation structure of the linear conveyor belt used in the production line for visual grading of pear appearance quality according to the present invention;

[0043] Figure 8 It is a structural schematic diagram of a steering machine used in a pear appearance quality visual grading production line according to the present invention;

[0044] Fig. 9 It is a schematic diagram of the control structure of the production line for visual grading of pear appearance quality according to the present invention;

[0045] Fig.10 It is a schematic diagram of the process of the visual grading method for the appearance quality of pears of the present invention;

[0046] In the figure: 1. image acquisition box; 2. tray; 3. steering gear; 4. linear conveyor belt; 5. grading trough; 6. push-pull electromagnet; 7. electromagnet fixing frame; 8. motor fixing frame; 9. chain; 10. stepper motor; 11. sprocket; 101. acquisition box frame; 102. black acrylic plate; 103. CCD industrial camera; 301. steering gear belt; 302. steering gear bracket; 303. steering gear roller; 401. channel steel; 402. fixing plate; 403. conveyor belt roller; 404. conveyor belt bracket; 405. conveyor belt; 406. right-angle connector; 501. grading trough baffle; 502. grading trough bracket. DETAILED DESCRIPTION

[0047] The specific embodiments of the present invention are given below. The specific embodiments are only used to further illustrate the present invention in detail and do not limit the protection scope of the present application.

[0048] The present invention is used for a visual grading method of pear appearance quality, and the grading method is applicable to the national standard GB / T 10650-2008 "Fresh Pears". The steps of the grading method are:

[0049] Step 1: Acquire images of fruit pears to be graded through an industrial camera, obtain sample pear images to form a sample pear image library, the number of samples in the sample pear image library is N, and N is an integer greater than 200;

[0050] Step 2: Randomly select more than half of the images in the sample pear image library as the training set, and the rest as the test set. Perform grayscale transformation on the images to obtain the grayscale image of the original image. Then use the Otsu method (OTSU) to automatically select a suitable threshold to perform binarization THRESH_BINARY processing on the grayscale image, so that the pixels in the image with a grayscale less than the threshold are set to 0, and the pixels with a grayscale greater than the threshold are set to 255. After the processing, the background grayscale value of the image is 0 and the foreground grayscale value is 255, and a mask that can separate the surface of the sample pear is obtained; then the mask is corroded, expanded, opened and closed, and then the mask is multiplied by the original image to extract the image of the outer surface of the pear. A large number of pear outer surface images constitute the pear outer surface image database;

[0051] Step 3: Use the geometric features of the outer surface of the pear to determine the standard degree of the sample pear shape and calculate the Fourier descriptor of the outer surface image of the pear:

[0052] Extract the contour of the pear outer surface image in the pear outer surface image database. Assume that the contour of the pear outer surface is a digital boundary with K points. Starting from any point (x0, y0) on the boundary and moving in a counterclockwise direction on the boundary, we will encounter coordinate pairs (x1, y1), (x2, y2), …, (x K-1 ,y K-1 ), these coordinates are expressed as x(k)=xk y(k)=y k In this representation, the boundary is represented by a coordinate sequence s(k) = [x(k), y(k)], k = 0, 1, 2…, K-1.

[0053] After that, treat each coordinate pair as a complex number and get:

[0054] s(k)=x(k)+jy(k),k=0,1,2…,K-1

[0055] In this way, the x-axis is used as the real axis of the complex sequence, and the y-axis is used as the imaginary axis of the complex sequence. The two-dimensional data is simplified to one-dimensional data. The Fourier descriptor a(u) of the contour is obtained by performing a discrete Fourier transform (DFT) on s(k):

[0056]

[0057] Where u is each frequency point after Fourier transform;

[0058] The pear surface boundary is divided equally, and the Fourier descriptors corresponding to the coordinates of the K equally divided points are taken to form a K×1 matrix; the obtained feature matrix is ​​labeled and input into an artificial neural network model (ANN) for training, and the label is the actual evaluation of the pear surface contour, and the shape standard is 0, the shape with slight defects is 1, and the shape with obvious defects is 2; after the training is completed, the classifier 1 is obtained;

[0059] Step 4: Use the color features of the outer surface of the pear to determine the standard degree of the color of the sample pear. The outer surface images of the pears in the pear outer surface database are BGR images. The three components B, G and R of the outer surface images of the pears are extracted respectively, and then the second-order moments (variances) of the matrices of the three components B, G and R are calculated to obtain a 3×1-dimensional feature matrix; the obtained feature matrix is ​​labeled and input into the naive Bayesian model (NBC) for training. The label is the actual evaluation of the pear surface color. The color standard is 0, the color with a slight deviation is 1, and the color with an obvious deviation is 2; after the training is completed, classifier 2 is obtained;

[0060] Step 5: Perform color space conversion on the image selected in the second step, convert the original image from BGR image to HSV image, extract the brightness component V in HSV space, divide the brightness component V by the grayscale image, perform brightness correction, and obtain the brightness correction image of the sample pear. A large number of brightness correction images form a brightness correction image library; in the brightness correction image, the brightness of the background area and the outer surface area of ​​the pear except the defective part is suppressed, and its grayscale value increases, while the grayscale value of the defective part remains unchanged. Use the Otsu method (OTSU) to automatically select a suitable threshold to perform binarization THRESH_BINARY processing on the grayscale image. After the processing, the grayscale values ​​of the background and the pear surface become 0, and the grayscale values ​​of the pear surface defects become 255, and a mask that can separate the surface defects of the sample pear is obtained; then the mask is corroded, expanded, opened and closed, and then the mask is multiplied by the original image to extract the defective area on the outer surface of the pear to obtain the defect image of the sample pear. A large number of defective images of the sample pears form a defect image library;

[0061] Step 6: Extract geometric features and texture features using defect images in the defect image library of sample pears:

[0062] (1) Calculate the Hu invariant moment of the defect image:

[0063] For a continuous image f(x,y), the p+q order geometric moment (standard moment) m of the image pq for:

[0064]

[0065] Where x and y are the horizontal and vertical coordinates of the image pixel respectively;

[0066] p+q order center distance μ pq Defined as:

[0067]

[0068] in and is the center of gravity of the image, m pq is the p+q order geometric moment of the image;

[0069] The defect images in the defect image database are all discrete digital images, and their geometric moments and central moments are:

[0070]

[0071] Where N and M are the height and width of the image, that is, the number of rows and columns of pixels;

[0072] Then the calculated central moment is normalized to obtain the normalized center distance η pq:

[0073]

[0074] Construct seven invariant moments M1 to M7 using the second - order and third - order normalized central moments of the defect image:

[0075] M1 = η 20 + η 02

[0076]

[0077] M3=(η 30 - 3η 12 ) 2 +(3η 21 - η 03 ) 2

[0078] M4=(η 30 + η 12 ) 2 +(η 21 + η 03 ) 2

[0079] M5=(η 30 - 3η 12 )(η 30 + η 12 )((η 30 + η 12 ) 2 - 3(η 21 + η 03 ) 2 )+(3η 21 - η 03 )(η 21 + η 03 )(3(η 30 + η 12 ) 2 - (η 21 + η 03 ) 2 )

[0080] M6=(η 20 - 3η 02 )((η 30 + η 12 ) 2 - (η 21 + η 03 ) 2 )+4η 11 (η 30 + η 12 )(η 21 + η 03)M7=(3η 12 -η 30 )(η 30 +η 12 )((η 30 +η 12 ) 2 -3(η 21 +η 03 ) 2 )-(η 03 -3η 21 )(η 21 +η 03 )(3(η 30 +η 12 ) 2 -(η 21 +η 03 ) 2 )

[0081] The calculated 7 invariant moments are sequentially combined into a 7×1 matrix;

[0082] (2) Calculate the Fourier descriptor of the defect image:

[0083] The calculation method is the same as the third step, dividing the defect boundary into equal parts, taking the Fourier descriptors corresponding to the coordinates of the K equally divided points, and forming a K×1 matrix; the value of K in the calculation of the Fourier descriptors in the two places in this application has no correlation and is selected according to actual conditions.

[0084] (3) Calculate the grayscale co-occurrence moment of the defect image:

[0085] The gray-level co-occurrence matrix is ​​obtained by statistically analyzing the situation where two pixels that maintain a certain distance on the image have a certain gray level. For a continuous image f(x,y), take any point (x,y) in the image and another point (x+a,y+b) that deviates from it by a point in the horizontal coordinate and b point in the vertical coordinate, and set the gray value of this pair of points to (g,h). If (x,y) moves on the entire image, various (g,h) values ​​will be obtained. If the number of gray values ​​is k, then there are k total gray values ​​of (g,h). 2 For the whole image, count the number of times each (g, h) value appears, and then arrange them into a matrix. Use the total number of times (g, h) appears to normalize the number of times each (g, h) value appears to the probability of occurrence P(g, h). Such a matrix is ​​called a gray-level co-occurrence matrix.

[0086] The texture of the image is described by studying the spatial correlation characteristics of grayscale, and some scalars are used to characterize the characteristics of the grayscale co-occurrence moment. The following four scalars are calculated:

[0087] Contrast Con:

[0088]

[0089] Energy Asm:

[0090]

[0091] EntropyEnt:

[0092]

[0093] Corr:

[0094]

[0095] in

[0096] in

[0097] The calculated four features of contrast, energy, entropy and correlation are sequentially combined into a 4×1 matrix to obtain the grayscale co-occurrence moment of the corresponding defect image;

[0098] Step 7: Fuse the three features of Hu invariant moment, Fourier descriptor and gray-level co-occurrence moment in step 6, that is, connect the three feature matrices end to end in the order of calculation to obtain a (7+K+4)×1-dimensional matrix, add labels to the fused features and input them into another artificial neural network (ANN) to train the ANN model. The labels are the actual types of defects, thorn scratches are 0, bumps are 1, abrasions are 2, fruit rust is 3, sunburn is 4, hail is 5, insect damage is 6, diseases are 7 and insect fruit is 8; after training, classifier 3 is obtained; at the same time, the area of ​​the surface defect of each sample in the defect image is calculated. The specific calculation method is to locate the defect, find the outer contour of the defect, calculate the number of pixels, correspond the number to the area in the actual space, and then determine the area of ​​the surface defect;

[0099] Step 8: Create an empty vector with a length of 11, and take the sample's fruit shape evaluation result (i.e., the result of classifier 1) as the first element of the vector, and take the sample's color evaluation result (i.e., the result of classifier 2) as the second element of the vector. If the predicted result of the sample surface defect type (i.e., the result of classifier 3) is x (x is an integer from 0 to 8), then the area of ​​the surface defect is the x+3th element of the vector, and the remaining elements are all set to 0; the labels of the third and fourth steps and the labels and defect areas of the seventh step are combined into a new vector according to the above principles, recorded as a decision vector, and the decision vector is labeled according to the evaluation criteria for fresh pear grades in the National Standard of the People's Republic of China GB / T 10650-2008 "Fresh Pears", and then input into a decision tree (DT) for training, with the label being the actual grade of the sample pear, with superior pears being 0, first-class pears being 1, second-class pears being 2, and pears outside the grade being 3; after training, classifier 4 is obtained;

[0100] Step 9: Quick Identification:

[0101] The image of the sample pear to be classified is segmented according to the outer surface image segmentation method in the second step to obtain the outer surface image of the sample pear, and the Fourier descriptor of the outer surface of the sample pear is extracted according to the third step, and the geometric feature (Fourier descriptor) is input into the classifier 1 for prediction to obtain the fruit shape evaluation result of the sample, and the result is a value of 0, 1 or 2; the color feature of the outer surface of the sample pear obtained in the second step is extracted according to the color feature calculation method in the fourth step, and the color feature is input into the classifier 2 for prediction to obtain the color evaluation result of the sample, and the result is a value of 0, 1 or 2; the outer surface image of the sample pear obtained in the second step is The surface defect image of the sample pear is obtained according to the surface defect image segmentation method in the fifth step, and then the geometric features and texture features (Hu invariant moment, Fourier descriptor, grayscale co-occurrence moment) of the surface defects of the sample pear are extracted according to the geometric features and texture features calculation method of the defect image in the sixth step. The geometric features and texture features are input into the classifier 3 in the seventh step to obtain the prediction result of the surface defect type of the sample, which is an integer value between 0 and 8. At the same time, the area of ​​the surface defect region is obtained; the decision vector is obtained according to the principle in the eighth step, and the decision vector is input into the classifier 4 in the eighth step to output the final grading result of the sample pear.

[0102] The present invention provides a production line for visual grading of pear appearance quality (see Figure 1-Figure 8), including an image acquisition box 1, a tray 2, a diverter 3, a linear conveyor belt 4, a grading slot 5, and a push-pull electromagnet 6; two linear conveyor belts 4 and two diverters 3 are spliced ​​together and connected using a transmission mechanism to form an annular reflux conveyor belt; the image acquisition box 1 is placed on a linear conveyor belt 4, and the acquisition box frame 101 of the image acquisition box 1 is 40mm aluminum, with a slide groove on the surface, and a fixed key can be installed. The upper surface of the truss of the linear conveyor belt 4 is punched at an appropriate position, and the linear conveyor belt 4 and the fixed key are fastened by screws and nuts, so that the image acquisition box 1 and the linear conveyor belt 4 are relatively fixed; the tray 2 is placed on the annular conveyor belt; the grading slot 5 is placed behind the image acquisition box. The outside of the linear conveyor belt 4 is tightly attached to each other; the push-pull electromagnet 6 has a threaded hole, and the electromagnet fixing frame 7 has a round hole, and the two are fastened by screws. The upper surface of the truss of the linear conveyor belt 4 is punched at an appropriate position, and the current linear conveyor belt 4 and the electromagnet fixing frame 7 are fastened by screws and nuts, so that the push-pull electromagnet 6 and the current linear conveyor belt 4 are relatively fixed; the stepping motor 10 has a threaded hole, and the motor fixing frame 8 has a round hole, and the two are fastened by screws. The sides of the trusses of the two linear conveyor belts 4 are provided with a certain length of slide groove, which can pass through the screws, and the linear conveyor belt 4 and the corresponding motor fixing frame 8 are fastened by screws and nuts, so that the two stepping motors 10 and the corresponding linear conveyor belt 4 are relatively fixed;

[0103] The image acquisition box 1 includes an acquisition box frame 101, a black acrylic plate 102, a CCD industrial camera 103 and a ring light source. The surface of the aluminum material used for the acquisition box frame 101 has a slide groove, and the black acrylic plate 102 of appropriate size is stuck in the slide groove and adhered to the acquisition box frame 101 with hot melt adhesive to fix it. The CCD industrial camera 103 has a threaded hole, and the camera fixing part has a bidirectional threaded hole. Screws can be used to fasten the CCD industrial camera 103 and the acquisition box frame 101 at the same time, so that the CCD industrial camera 103 can be fixed in a suitable position, and the CCD industrial camera 103 is connected to the lens;

[0104] The CCD industrial camera 103 communicates with the host computer, the above-mentioned grading method is loaded in the host computer, the host computer and the PLC perform two-way communication, the PLC is electrically connected to the corresponding stepping motor through the motor controller, and a photoelectric limit switch is installed near the push-pull electromagnet corresponding to each channel of the grading slot. The photoelectric limit switch is installed at a position 10 cm in front of the corresponding push-pull electromagnet in the pear conveying direction to control the push-pull electromagnet action of the corresponding channel. A photoelectric limit switch is installed at the entrance of the image acquisition box, and all photoelectric limit switches and push-pull electromagnets communicate with the PLC;

[0105] The CCD camera model is MV-EM200C, with a resolution of 1280*960, a photosensitive area of ​​4.8mm×3.6mm, and three image acquisition modes: continuous, external trigger and soft trigger; the lens model is HX2X-T1, which is a zoom lens with a focal length of 12mm-36mm and a minimum object distance of 20cm; the LED ring light model is HDR-90-45;

[0106] In the present application, two cameras can be set up, and the two cameras are placed on the collection box frame 101 on both sides of the top of the image collection box 1. The sample pear is transported to the middle area between the two cameras to start collection. In this way, an image of the entire circumference of the pear surface is obtained as large as possible, so that the grading result is as accurate as possible. When two cameras are set up, the images collected by the two cameras are spliced ​​and then processed using the above-mentioned grading method.

[0107] The steering gear 3 includes a steering gear belt 301, a steering gear bracket 302 and a steering gear roller 303. The steering gear bracket 302 includes a vertical bracket and a horizontal arc bracket. The arc side of the arc bracket is perforated at a certain interval, and a plurality of steering gear rollers 303 are installed. The surfaces of the plurality of steering gear rollers are sleeved with the steering gear belt. The steering gear roller 303 is a column-shaped roller with a larger diameter at one end and a smaller diameter at the other end, so that the surface of the steering gear belt 301 sleeved on the surface can move at the same angular velocity at all places.

[0108] The linear conveyor belt 4 includes a channel steel 401, a fixing plate 402, a conveyor belt roller 403, a conveyor belt bracket 404, a conveyor belt 405 and a right-angle connector 406. The conveyor belt bracket 404 includes a vertical bracket and a horizontal truss. Channel steels 401 are extended along the conveying direction at both ends of the truss, and channel steels are arranged on the left and right sides of the truss. The channel steels 401 are connected to both ends of the truss, and the two are connected together by a right-angle connector.

[0109] The axle of the conveyor belt roller 403 is a stepped axle, and a through hole is punched at the side of the end of the channel steel 401 away from the truss. The size of the through hole is adapted to the inner end of the roller axle, and the inner ends of the axles of the two conveyor belt rollers 403 pass through the corresponding through holes;

[0110] There is an opening at the center of the fixing plate 402, and the size of the opening is adapted to the outer end of the roller axle. The fixing plate 402 is sleeved on the outer end of the axle of the roller 403, and small holes are punched around the central opening of the fixing plate. At the same time, holes are punched at the corresponding positions of the channel steel 401, and the fixing plate and the channel steel are fastened with screws and nuts so that the roller 403 can rotate normally without lateral shaking; the conveyor belt is sleeved on two conveyor belt rollers 403, one conveyor belt roller is connected to the stepper motor through a chain as an active conveyor belt roller, and the other is a driven conveyor belt roller; the active conveyor belt roller 403 is installed on the side of the channel steel 401 at one end, and is connected to the stepper motor through a chain 11. The active conveyor belt roller is punched close to the truss end, and the outer side of the truss where the active conveyor belt roller is located is A slide groove is arranged on the surface, and the channel steel can move relative to the truss along the conveying direction under the drive of the slide groove. The right-angle connector 406 is a connector with a right angle corner. Both end faces of the right-angle connector 406 have round holes. The two right-angle connectors form a pair, and one of each pair of right-angle connectors is fastened to the channel steel 401 by screws and nuts, and the other is fastened to the slide groove of the truss by screws and nuts. The slide groove can move relative to the truss, and then each pair of right-angle connectors 406 is fastened by screws and nuts to fix the active conveyor belt roller 403 and the conveyor belt bracket 404 relatively. The conveyor belt 405 is sleeved on the conveyor belt roller 403. Due to the existence of the slide groove, the channel steel 401 at one end of the active conveyor belt roller 403 can adjust its position to ensure that the conveyor belt 405 is in a taut state;

[0111] The grading trough 5 comprises a parallel grading trough baffle, a divergent grading trough baffle 501 and a grading trough bracket 502. Both grading trough baffles 501 are made of acrylic plates. The entrance of the grading trough 5 is provided with a plurality of parallel grading trough baffles. The parallel grading trough baffles are perpendicular to the bottom of the trough. The plurality of parallel grading trough baffles are arranged in parallel along the depth direction of the trough. The rear end of the parallel grading trough baffle is connected to the front end of the divergent grading trough baffle. The plurality of divergent grading trough baffles 501 are arranged in a divergent manner toward the tail end as a whole. The inner spacing increases toward the tail end, and the bottom of the trough also increases. The grading trough is tilted downward to facilitate the storage of sample pears. The tail of the divergent grading trough baffle is connected to the tail of two adjacent divergent grading trough baffles and the bottom of the trough by a horizontal plate. The parallel grading trough baffle and the divergent grading trough baffle 501 and the bottom of the trough are formed as a whole and placed on the grading trough bracket 502, so that the entrance of the grading trough is opposite to the tray outlet at the corresponding position of the linear conveyor belt. The number of channels in the grading trough is consistent with the level of the classifier 4, and each channel is determined as a grading area of ​​one level. In this embodiment, four levels are set and four channels are set.

[0112] The stepper motor 10 is of model 57BYG250C, with a step angle of 1.8° and a torque of 1.8 N·m. The motor shaft is tightly connected to the sprocket 11 by interference fit, and is connected to the sprocket on the axle of the active conveyor belt roller 403 by a chain 9 to form a transmission structure; the motor controller is of model TB6600, with a rated current of 4.2A and a subdivision number of 6400, the PLC model is Siemens S7-200SMART, the CPU is ST40, the input I / O is 24, and the output I / O is 16. The input of the stepper motor 10 is connected to the output end of the motor controller by a wire, and the input end of the motor controller is connected to the output I / O of the PLC by a wire;

[0113] The working principle and workflow of the pear appearance quality visual grading production line and grading method of the present invention are as follows:

[0114] The working principle of the grading production line of the present invention is:

[0115] After the sample pear is transported to the image acquisition area (inside the image acquisition box) by the conveyor belt, the photoelectric limit switch arranged in the image acquisition box at a position 10 cm away from the CCD camera in the conveying direction of the conveyor belt sends a signal to the PLC controlling the conveyor belt; after receiving the signal, the PLC changes the output of the motor controller to stop the stepper motors 10 driving the two conveyor belts, and stops the sample pear within the specified image acquisition range. When the stepper motor finally stops the conveyor belt after the deceleration process, the sample pear is located at a suitable position for the CCD camera image acquisition. After the PLC controls all the stepper motors to stop, it sends a command to the host computer through the communication system to start acquisition; the host computer receives the acquisition command, controls the CCD industrial camera 10 to acquire a frame of image, and the image is transmitted to the host computer through the Gigabit network cable via the network port of the computer. The host computer processes the image and extracts features, and inputs the extracted feature values ​​into the designed and trained classifier to classify the relevant features (the host computer loads and executes this The classification result (which type or types of surface defects the defect is, the total area of ​​defects, the grade of the pear) is displayed on the upper computer interface, and the upper computer sends the classification result to the PLC through the communication system; after receiving the classification result, the PLC controls the rotation of all stepper motors 10 to transport the sample pear out of the image acquisition area, and makes the corresponding photoelectric limit switch near the entrance of the grading slot (set in the conveying direction of the conveyor belt and 10 cm away from the entrance of the grading slot); the sample pear is transported to the grading area of ​​the specified level, and the photoelectric limit switch corresponding to the grading area sends a signal to the PLC controlling the conveyor belt through the communication system; after receiving the signal, the PLC changes the output of the motor controller to stop all stepper motors 10 from rotating, and stops the sample pear within the specified grading execution range, and the PLC sends instructions to the corresponding push-pull electromagnet 6 at the same time; after receiving the signal, the push-pull electromagnet 6 executes a pushing action to push the sample pear into the corresponding grading slot 5 channel; the grading is completed.

[0116] The workflow of the grading production line of the present invention comprises the following steps:

[0117] Step 1, place the sample pear on the tray 2, the conveyor belt 405 is driven by the stepper motor 10 to move the tray 2 to the designated position, that is, inside the image acquisition box 1, the photoelectric limit switch detects that the sample pear has reached the designated position, and sends a level signal to the PLC through the communication system;

[0118] Step 2: After receiving the signal from the photoelectric limit switch, the PLC sends a command to the upper computer through the communication system. The upper computer sends a series of commands to the CCD industrial camera 103 for turning on the camera, focusing, setting parameters such as frame rate and delay, collecting images, turning off the camera, releasing the camera, etc., to complete image acquisition;

[0119] Step 3, the digital image input is transmitted to the host computer through a gigabit network cable; the grading method of the pear appearance quality is loaded by the software;

[0120] The process of the pear appearance quality grading method is: extract the region of interest (ROI) of the collected image so that the pear is in the center of the image; perform preprocessing operations such as brightness correction, threshold segmentation and fuzzy processing on the image to obtain the defect area on the pear surface; extract the surface defect contour, calculate the Hu invariant moment, Fourier descriptor and grayscale co-occurrence moment of the defect; fuse the extracted features; input the fused features into the pre-trained neural network learning model for classification: output the classification result; its specific steps are:

[0121] (1) Obtaining images of pears to be graded using an industrial camera;

[0122] (2) The image is gray-scaled to obtain a gray-scale image of the original image, and then the OTSU method is used to automatically select a suitable threshold to perform binarization THRESH_BINARY processing on the gray-scale image, so that the pixels in the image with a grayscale less than the threshold are set to 0, and the pixels with a grayscale greater than the threshold are set to 255. After the processing, the background grayscale value of the image is 0 and the foreground grayscale value is 255, thereby obtaining a mask that can separate the surface of the sample pear; the mask is then corroded, expanded, opened, and closed, and then the mask is multiplied by the original image to extract the image of the outer surface of the pear;

[0123] (3) Extract the contour of the pear surface image in (2). Assume that the contour of the pear surface is a digital boundary with K points. Then, starting from any point (x0, y0) on the boundary and moving in a counterclockwise direction on the boundary, we will encounter coordinate pairs (x1, y1), (x2, y2), …, (x K-1 ,y K-1 ), these coordinates are expressed as x(k)=x k y(k)=y k In this representation, the boundary is represented as a coordinate sequence s(k)=[x(k),y(k)], k=0,1,2…,K-1. K is set according to the boundary length, K=16.

[0124] After that, treat each coordinate pair as a complex number and get:

[0125] s(k)=x(k)+jy(k),k=0,1,2…,K-1

[0126] In this way, the x-axis is used as the real axis of the complex sequence, and the y-axis is used as the imaginary axis of the complex sequence. The two-dimensional data is simplified to one-dimensional data. The Fourier descriptor a(u) of the contour is obtained by performing a discrete Fourier transform (DFT) on s(k):

[0127]

[0128] Where u is each frequency point after Fourier transform;

[0129] The surface boundary of the pear is divided equally, and the Fourier descriptors corresponding to the coordinates of the K equally divided points are taken to form a K×1 matrix; the obtained feature matrix is ​​input into the classifier 1, and the result is the fruit shape evaluation of the pear to be graded;

[0130] (4) extracting the three components B, G and R of the pear outer surface image in (2), and then calculating the second-order moment (variance) of the three component matrices of B, G and R to obtain a 3×1 dimensional feature matrix; inputting the obtained feature matrix into classifier 2, and obtaining the result as the color evaluation of the pear to be graded;

[0131] (5) Performing color space conversion on the image obtained in (1), converting the original image from a BGR image to an HSV image, extracting the brightness component V in the HSV space, dividing the grayscale image by the brightness component V, performing brightness correction, and obtaining a brightness-corrected image of the sample pear; using the Otsu method (OTSU) to automatically select a suitable threshold to perform THRESH_BINARY processing on the grayscale image after brightness correction, and obtaining a mask for separating surface defects of the sample pear; then performing erosion, expansion, opening and closing operations on the mask, and then multiplying the mask with the original image to extract the defect area on the outer surface of the pear, and obtaining a defect image of the sample pear;

[0132] (6) Extract geometric features and texture features of the defect image in (5):

[0133] ① Calculate the Hu invariant moment of the defect image:

[0134] For a continuous image f(x,y), the p+q order geometric moment (standard moment) m of the image pq for:

[0135]

[0136] Where x and y are the horizontal and vertical coordinates of the image pixel respectively;

[0137] p+q order center distance μ pq Defined as:

[0138]

[0139] in and is the center of gravity of the image, m pq is the p+q order geometric moment of the image;

[0140] The defect images in the database are all discrete digital images, and their geometric moments and central moments are:

[0141]

[0142] Where N and M are the height and width of the image, that is, the number of rows and columns of pixels;

[0143] Then the calculated central moment is normalized to obtain the normalized center distance η pq :

[0144]

[0145] Seven invariant moments M1~M7 are constructed using the second-order and third-order normalized central moments of the defect image:

[0146] M1=η 20 +η 02

[0147]

[0148] M3=(η 30 -3η 12 ) 2 +(3η 21 -η 03 ) 2

[0149] M4=(η 30 +η 12 ) 2 +(η 21 +η 03 ) 2

[0150] M5=(η 30 -3η 12 )(η 30 +η 12 )((η 30 +η 12 ) 2 -3(η 21 +η 03 ) 2 )+(3η 21 -η 03 )(η 21 +η 03 )(3(η 30 +η 12 ) 2 -(η 21 +η 03 ) 2 )

[0151] M6=(η 20-3η 02 )((η 30 +η 12 ) 2 -(η 21 +η 03 ) 2 )+4η 11 (η 30 +η 12 )(η 21 +η 03 )M7=(3η 12 -η 30 )(η 30 +η 12 )((η 30 +η 12 ) 2 -3(η 21 +η 03 ) 2 )-(η 03 -3η 21 )(η 21 +η 03 )(3(η 30 +η 12 ) 2 -(η 21 +η 03 ) 2 )

[0152] The calculated 7 invariant moments are sequentially combined into a 7×1 matrix;

[0153] ② Calculate the Fourier descriptor of the defect image:

[0154] The calculation method is the same as (3). Divide the defect boundary into equal parts, take the Fourier descriptors corresponding to the coordinates of the K equally divided points, and form a K×1 matrix; in this case, K is 24.

[0155] ③ Calculate the grayscale co-occurrence moment of the defect image:

[0156] The gray-level co-occurrence matrix is ​​obtained by statistically analyzing the situation where two pixels that maintain a certain distance on the image have a certain gray level. For a continuous image f(x,y), take any point (x,y) in the image and another point (x+a,y+b) that deviates from it, and set the gray value of this pair of points to (g,h). If (x,y) moves on the entire image, various (g,h) values ​​will be obtained. If the number of gray values ​​is k, then there are k total gray values ​​of (g,h). 2 For the whole image, count the number of times each (g, h) value appears, and then arrange them into a matrix. Use the total number of times (g, h) appears to normalize the number of times each (g, h) value appears to the probability of occurrence P(g, h). Such a matrix is ​​called a gray-level co-occurrence matrix.

[0157] The texture of the image is described by studying the spatial correlation characteristics of grayscale, and some scalars are used to characterize the characteristics of the grayscale co-occurrence moment. The following four scalars are calculated:

[0158] Contrast Con:

[0159]

[0160] Energy Asm:

[0161]

[0162] EntropyEnt:

[0163]

[0164] Corr:

[0165]

[0166] in

[0167] in

[0168] The calculated four features of contrast, energy, entropy and correlation are sequentially combined into a 4×1 matrix;

[0169] (7) The three features of Hu invariant moment, Fourier descriptor and gray-level co-occurrence moment in (6) are fused, that is, the three feature matrices are connected end to end in the order of calculation to obtain a (7+K+4)×1-dimensional matrix. The fused features are labeled and input into classifier 3 to obtain the defect type; the defect area is calculated;

[0170] (8) Create an empty vector with a length of 11, with the fruit shape evaluation result in (3) as the first element of the vector, and the color evaluation result in (4) as the second element of the vector. According to the principle that the predicted result of the sample surface defect type is x (x is an integer from 0 to 8), the area of ​​the surface defect is the x+3th element of the vector, and the remaining elements are all set to 0, the defect area in (7) is written into the vector; the vector is input into the classifier 4 to obtain the classification result;

[0171] Step 4, the grading result obtained in step 3 is sent to the PLC through the communication system. The PLC controls the rotation of the stepper motor 10 according to the information, and transports the tray 2 carrying the sample pears out of the image acquisition box 1 to the grading area of ​​the specified level. The PLC enables the corresponding push-pull electromagnet 6, and the push-pull electromagnet 6 performs the grading action and pushes the sample pears into the grading slot 5 to complete the grading process.

[0172] The recognition rate of pear appearance can reach over 95% through feature fusion.

[0173] The present invention does not describe the prior art.

Claims

1. A method for visually grading the appearance quality of pears, characterized in that: The method includes the following: Acquire color images of pear fruits to be graded to obtain sample pear images, and a large number of sample pear images form a sample pear image library; The sample pear images are processed to extract the pear outer surface images, and a large number of pear outer surface images form a pear outer surface image database; Obtaining the Fourier descriptor of the pear outer surface image in the pear outer surface image database, using the Fourier descriptor as input and the actual evaluation of the pear surface contour as output, training an artificial neural network model to obtain classifier 1; Extract the B, G and R components of the pear outer surface image in the pear outer surface image database, calculate the second-order moments of the B, G and R component matrices, and obtain a 3×1-dimensional feature matrix. Take the feature matrix as input and the actual evaluation of the pear surface color as output to train a naive Bayes model and obtain classifier 2. Performing brightness correction on the pear outer surface images in the pear outer surface image database to obtain brightness corrected images of sample pears, and a large number of brightness corrected images form a brightness corrected image library; The brightness correction images in the brightness correction image library are segmented to obtain the defect images of the sample pears, and a large number of the defect images of the sample pears form a defect image library; Extract the Hu invariant moment, Fourier descriptor, and gray-level co-occurrence moment of the defect image in the defect image library, take these three features as input, and take the actual type of the defect as output to train another artificial neural network to obtain classifier 3; at the same time, calculate the area of ​​the surface defect in each defect image; The results of classifier 1, the results of classifier 2, and the number of all defect types of classifier 3 are sequentially formed into a decision vector. If the number of all defect types of classifier 3 is x, then the length of the decision vector is x+2; in the decision vector, the position corresponding to the number of defect types is filled with the area of ​​the surface defect of the sample pear corresponding to the defect type determined by the sample, and the other defect type positions are filled with 0; Taking the decision vector as input and the actual grade of the sample pears according to the national standard as output, the decision tree is trained to obtain classifier 4; The trained classifiers 1, 2, 3 and 4 are used to grade the appearance quality of pears.

2. The classification method according to claim 1, characterized in that: The process of processing the sample pear image to extract the pear outer surface image is as follows: grayscale transformation is performed on the sample pear image to obtain a grayscale image of the original image, and then the OTSU method (OTSU) is used to automatically select a suitable threshold to perform binarization THRESH_BINARY processing on the grayscale image, so that pixels in the image with a grayscale less than the threshold are set to 0, and pixels with a grayscale greater than the threshold are set to 255. After the processing, the background grayscale value of the image is 0, and the foreground grayscale value is 255, so as to obtain a mask that can separate the surface of the sample pear; then the mask is corroded, expanded, opened and closed, and then the mask is multiplied by the original image to extract the image of the pear outer surface.

3. The classification method according to claim 1, characterized in that: The calculation process of the Fourier descriptor of the pear surface image is: Extract the contour of the pear outer surface image in the pear outer surface image database. Assume that the contour of the pear outer surface is a digital boundary with K points. Starting from any point (x0, y0) on the boundary and moving in a counterclockwise direction on the boundary, we will encounter coordinate pairs (x1, y1), (x2, y2), …, (x K-1 ,y K-1 ), these coordinates are expressed as x(k)=x k y(k)=y k In this representation, the boundary is represented by a coordinate sequence s(k)=[x(k),y(k)], k=0,1,2…,K-1; After that, treat each coordinate pair as a complex number and get: s(k)=x(k)+jy(k),k=0,1,2…,K-1 In this way, the x-axis is used as the real axis of the complex sequence, and the y-axis is used as the imaginary axis of the complex sequence, simplifying the two-dimensional data into one-dimensional data; performing discrete Fourier transform (DFT) on s(k) will obtain the Fourier descriptor a(u) of the contour: Where u is each frequency point after Fourier transform; Divide the pear surface boundary into equal parts, take the Fourier descriptors corresponding to the coordinates of the K equally divided points, and form a K×1 matrix.

4. The classification method according to claim 1, characterized in that: The labels of classifier 1 are the actual evaluation of the pear surface contour, with 0 for standard shape, 1 for shape with minor defects, and 2 for shape with obvious defects; The label of classifier 2 is the actual evaluation of the color of the pear surface, with 0 for standard color, 1 for color with slight deviation, and 2 for color with obvious deviation; The labels of classifier 3 are the actual types of defects, including thorn scratches as 0, bruises as 1, abrasions as 2, rust as 3, sunburn as 4, hail as 5, insect damage as 6, disease as 7, and insect-infested fruit as 8; The labels of classifier 4 are the actual grades of the sample pears, 0 for superior pears, 1 for first-class pears, 2 for second-class pears, and 3 for pears outside the grade.

5. The classification method according to claim 1, characterized in that: The process of brightness correction is: convert the color space of the pear outer surface image, convert the original image from BGR image to HSV image, extract the brightness component V in HSV space, divide the brightness component V by the grayscale image, perform brightness correction, and obtain the brightness correction image of the sample pear; The process of obtaining the defect image of the sample pear is as follows: in the brightness correction image, the brightness of the background area and the outer surface area of ​​the pear except the defective part is suppressed, and its grayscale value increases, while the grayscale value of the defective part remains unchanged. The Otsu method is used to automatically select a suitable threshold to perform binarization THRESH_BINARY processing on the grayscale image. After the processing, the grayscale values ​​of the background and the pear surface become 0, and the grayscale values ​​of the pear surface defects become 255, and a mask that can separate the surface defects of the sample pear is obtained; then the mask is corroded, expanded, opened and closed, and then the mask is multiplied by the original image to extract the defective area on the outer surface of the pear to obtain the defect image of the sample pear.

6. A production line for visual grading of pear appearance quality, characterized in that: The production line includes conveyor belt mechanical structure, host computer, image acquisition system, conveyor belt drive system, grading execution system, PLC and communication system; The conveyor belt mechanical structure is the main mechanical structure of the production line, which plays the role of transporting sample pears to different functional areas. The entire conveyor belt mechanical structure forms a circular reflux conveyor belt; The conveyor belt drive system is used to drive the start, stop, deceleration, acceleration and uniform speed of the circular return flow conveyor belt; The host computer directly controls the CCD industrial camera in the image acquisition system to realize image acquisition, which is used to obtain the outer surface image of the pear; the host computer communicates with the PLC through the communication system, sends instructions to the PLC, and makes the PLC control the conveyor belt drive system and the grading execution system to achieve the purpose of controlling the entire production line; The grading method according to any one of claims 1 to 5 is loaded in the host computer, and the grading result is displayed on the display screen of the host computer and sent to the PLC at the same time; The grading execution system includes a push-pull electromagnet, an electromagnet fixing frame and a grading trough. The push-pull electromagnet and the electromagnet fixing frame are fixed on the reflux conveyor belt behind the image acquisition system. The grading trough has multiple channels, and shock-absorbing materials are pasted inside. The grading trough is placed on the opposite side of the push-pull electromagnet. Each push-pull electromagnet is aligned with a channel, and each channel corresponds to a grading level, which is recorded as a grading area. The push-pull electromagnet is connected to the PLC. After receiving the grading result sent by the upper computer, the PLC enables the push-pull electromagnet of the corresponding level, and then sends the sample pear into the designated grading area.

7. The production line according to claim 6, characterized in that: The conveyor belt drive system includes a motor controller, which is connected to a PLC. After receiving a drive instruction sent by a host computer, the PLC provides a command signal to the motor controller. The motor controller generates a waveform for driving a corresponding stepper motor on a reflow conveyor belt and provides an effective voltage to realize control of the stepper motor. The image acquisition system includes a CCD industrial camera, a lens, an LED ring light source, an image acquisition box and a power adapter. The power adapter supplies power to the image acquisition system so that the ring light source can work normally, and the ring light source provides uniform brightness for the image acquisition system. The image acquisition box is fixed on the bracket of the reflow conveyor belt, and a black acrylic plate is installed on the surface of the frame of the image acquisition box to isolate external light and reduce interference during image acquisition. The CCD industrial camera uses a color digital CCD industrial camera, which can obtain more features of the surface of the sample pear. By connecting it to the lens, a clear image can be obtained within an effective distance. The interface of the CCD industrial camera is a gigabit network port, which is connected to the network cable interface of the host computer using a network cable. The communication system uses ZigBee, Bluetooth or WiFi.

8. The production line according to claim 7, characterized in that: The annular reflux conveyor belt includes two straight conveyor belts and two steering gears, which are spliced ​​together and connected by a transmission mechanism to form an annular reflux conveyor belt; the image acquisition box is placed on a straight conveyor belt, the frame of the image acquisition box is 40mm aluminum, with a slide groove on the surface, and a fixed key is installed in the slide groove. Holes are punched at appropriate positions on the upper surface of the straight conveyor belt truss, and the straight conveyor belt and the fixed key are fastened by screws and nuts, so that the image acquisition box and the straight conveyor belt are relatively fixed; the tray is placed on the annular reflux conveyor belt; the grading trough is placed behind the image acquisition box The outer side of the square linear conveyor belt is tightly attached to each other; the push-pull electromagnet is provided with a threaded hole, and the electromagnet fixing frame is provided with a round hole, and the two are fastened by screws. The upper surface of the linear conveyor belt truss is punched at an appropriate position, and the current linear conveyor belt and the electromagnet fixing frame are fastened by screws and nuts, so that the push-pull electromagnet and the current linear conveyor belt are relatively fixed; the stepper motor is provided with a threaded hole, and the motor fixing frame is provided with a round hole, and the two are fastened by screws. The sides of the two linear conveyor belt trusses are provided with sliding grooves, and the linear conveyor belts and the corresponding motor fixing frames are fastened by screws and nuts, so that the two stepper motors and the corresponding linear conveyor belts are relatively fixed; A photoelectric limit switch is installed near the push-pull electromagnet corresponding to each channel of the grading tank to control the action of the push-pull electromagnet of the corresponding channel. A photoelectric limit switch is installed at the entrance of the image acquisition box. All photoelectric limit switches and push-pull electromagnets communicate with the PLC. The linear conveyor belt includes channel steel, a fixing plate, a conveyor belt roller, a conveyor belt bracket, a conveyor belt and a right-angle connector. The conveyor belt bracket includes a vertical bracket and a horizontal truss. Channel steels are extended along the conveying direction at both ends of the truss, and channel steels are arranged on the left and right sides of the truss. The channel steels are connected to both ends of the truss, and the two are connected together by a right-angle connector. The conveyor belt is mounted on two conveyor belt rollers, one of which is connected to the stepper motor through a chain as the active conveyor belt roller, and the other is the driven conveyor belt roller; the active conveyor belt roller is installed on the side of the channel steel at one end, and is connected to the stepper motor through a chain. The active conveyor belt roller is punched near the truss end, and a slide groove is set on the outer surface of the truss where the active conveyor belt roller is located. The channel steel can move relative to the truss along the conveying direction driven by the slide groove. Due to the existence of the slide groove, the conveyor belt is kept in a tight state; The grading trough comprises a parallel grading trough baffle, a divergent grading trough baffle and a grading trough bracket. Both grading trough baffles are made of acrylic plates. A plurality of parallel grading trough baffles are placed in parallel at the entrance of the grading trough. The parallel grading trough baffle is perpendicular to the trough bottom. The plurality of parallel grading trough baffles are arranged in parallel along the trough depth direction. The rear end of the parallel grading trough baffle is connected to the front end of the divergent grading trough baffle. The plurality of divergent grading trough baffles are placed in a divergent manner toward the tail end as a whole. The inner spacing increases toward the tail end, and the trough bottom is also inclined downward. The tail of the divergent grading trough baffle is connected to the tail end of two adjacent divergent grading trough baffles by a horizontal plate. The whole formed by the parallel grading trough baffle, the divergent grading trough baffle and the trough bottom is placed on the grading trough bracket, so that the entrance of the grading trough is opposite to the tray outlet at the corresponding position of the linear conveyor belt.

9. The production line according to claim 7, characterized in that: The host computer includes a GUI interface made in Python, which includes function buttons, information display and image display areas. Users can use the interface to control the production line, obtain equipment information, and view collected images and grading results; The model of the CCD industrial camera is MV-EM200C, and the photosensitive area of ​​the CCD industrial camera is 4.8mm×3.6mm; the model of the lens is HX2X-T1, which is a zoom lens with a focal length of 12mm-36mm and a minimum object distance of 20cm; the model of the LED ring light source is HDR-90-45; the model of the PLC is Siemens S7-200SMART, the CPU is ST40, the input I / O is 24, and the output I / O is 16; the model of the motor controller is TB6600, with a rated current of 4.2A and a subdivision number of 6400; the model of the stepper motor is 57BYG250C, with a step angle of 1.8° and a torque of 1.8N·m; the stroke of the push-pull electromagnet is 60mm and the thrust is 100N.

Images