Malformed potato defect segmentation method based on machine vision
Through the machine vision-based deformed potato detection and segmentation method, the problem of low efficiency in potato quality detection is solved, and efficient and accurate potato quality assessment and resource optimization are achieved, which is suitable for the detection needs of different hardware equipment.
Patent Information
- Application Number
- CN202510747957.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-06-06
- Publication Date
- 2025-09-09
AI Technical Summary
The existing technology for potato quality detection is inefficient, with inconsistent results and a lack of quantitative description, making it difficult to meet the needs of modern processing. The traditional manual detection model is difficult to adapt to the scale and standardization of the industry.
A machine vision-based method for detecting and segmenting deformed potatoes was adopted, including image preprocessing, Douglas-Peucker contour simplification, Graham scanning convex hull analysis, and support vector machine classification. An adaptive segmentation algorithm was designed in combination with concave point scanning to achieve automatic extraction and quantitative evaluation of potato contour features.
It achieves efficient and automated testing, improves detection efficiency and accuracy, reduces resource waste, provides a reliable basis for quality assessment, and supports hardware equipment adaptation in different scenarios.
Smart Images
Figure CN120612685A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of image classification, and in particular relates to a method for detecting and segmenting deformed potatoes based on machine vision. Background Art
[0002] As an important dual-purpose grain and vegetable crop and the fourth largest staple food in my country, the high-quality development of the potato industry is of strategic significance for ensuring food security and promoting agricultural modernization. Potato deformities primarily manifest as morphological variations such as hollow potatoes, cracked potatoes, and misshapen tubers. These defects not only affect product appearance and storage performance, but also significantly reduce processing suitability and end-product yield, becoming a key bottleneck hindering improvements in the industry's quality and efficiency.
[0003] my country's current potato quality inspection system is still dominated by manual spot checks. Inspectors visually assess the degree of deformity according to standards such as GB 18133-2012, "Seed Potatoes," and then manually remove defective areas for grading. This traditional method has significant limitations: First, manual inspection is inefficient, with an hourly processing capacity of less than 200 potatoes, which is insufficient to meet the demands of modern processing lines that process over 10 tons per hour. Second, inspection results are significantly affected by personnel experience, with the consistency of grading between inspectors for the same batch of samples being only 68%-75%. Third, the lack of quantitative descriptions of defect characteristics makes it impossible to establish an accurate quality control database for traceability. Under the dual pressures of rising labor costs and rising consumption, traditional inspection models are no longer adapting to the demands of industrial scale and standardization.
[0004] Chinese invention patent CN115365162A discloses a machine vision-based potato grading device and shape detection method. However, existing literature still requires the extraction and computation of numerous potato shape features, and only provides simple assessments of deformed potatoes. There is little assessment of the usability of detected deformed potatoes, especially since the weight of the deformed portion is a crucial metric in spot checks. Summary of the Invention
[0005] To address the above issues, the present invention proposes a method for detecting and segmenting deformed potatoes based on machine vision. The method proposed in the present invention comprises the following steps:
[0006] (1) Screening deformed potatoes, collecting deformed potato image information on the device, performing preprocessing operations on it, and extracting potato masks.
[0007] The preprocessing operations include: converting the image from RGB format to HSV, setting an appropriate threshold, and then performing dilation and corrosion operations on the image.
[0008] (2) Extract the potato contour based on the obtained mask and calculate the contour's perimeter, area, length, width and other external feature parameters; simplify the potato contour using the Douglas-Peucker method and store the contour vertices.
[0009] (3) The obtained contour vertices are used to calculate the convex hull of the potato contour using the Graham scanning method, and the external feature parameters of the convex hull are extracted.
[0010] (4) The external feature parameters of the potato convex hull contour, such as perimeter, area, length and width, are compared with the external features of the potato contour itself. The data collected through support vector machine training is then used to predict whether the potato is deformed.
[0011] (5) When the potato is deformed, scan the concave points inside the potato contour, extract the concave points, start the algorithm segmentation, and obtain the optimal segmentation path.
[0012] (6) The area and perimeter of the segmented potato contour are extracted to predict the weight of the potato after removing the deformed part.
[0013] Beneficial effects of the present invention:
[0014] 1. Efficient and automated testing to reduce labor costs
[0015] Through image preprocessing (RGB-HSV conversion, dilation and corrosion operations) and mask extraction technology, the potato body and background are quickly separated. Combined with Douglas-Peucker contour simplification and Graham scanning convex hull analysis, the potato contour features are automatically extracted and parameter calculation is achieved, avoiding the subjectivity and inefficiency of traditional manual screening and significantly improving detection efficiency.
[0016] 2. High-precision deformity identification and quantitative assessment
[0017] By comparing the perimeter, area, length, and width of the original potato outline with its convex hull, combined with a support vector machine (SVM) classification model, the system transforms traditional qualitative analysis into quantitative analysis, effectively improving the accuracy and interpretability of deformity identification. By using concave point scanning and an optimal segmentation path algorithm, the system precisely locates deformed areas, avoiding the loss of normal potato parts due to missegmentation.
[0018] 3. Dynamically optimize the segmentation path to reduce weight loss
[0019] In view of the concave characteristics of deformed potatoes, an adaptive segmentation algorithm based on the concave points of the contour is designed. By calculating the optimal segmentation path, the normal potato area is retained to the maximum extent. The weight after de-deformation is predicted by combining the area, perimeter and other parameters after segmentation, providing a reliable basis for subsequent quality grading and processing, and reducing resource waste.
[0020] 4. Technology Integration and Scalability
[0021] By integrating image processing, geometric feature analysis, machine learning classification and dynamic path planning technologies, a complete potato quality assessment system is formed. The modular design of the algorithm can adapt to hardware equipment in different scenarios (such as sorting lines and quality inspection instruments), providing a technical reference for defect detection and quality grading of other fruits and vegetables. BRIEF DESCRIPTION OF THE DRAWINGS
[0022] Figure 1 Flowchart for detection and segmentation of deformed potatoes;
[0023] Figure 2 For potato samples;
[0024] Figure 3 Preprocessing of potato images;
[0025] Figure 4 Simplified potato outline for Douglas-Peucker;
[0026] Figure 5 Compute the convex hull of the potato outline for the Graham scanning method;
[0027] Figure 6 to split deformed potatoes;
[0028] Figure 7 Extraction of key areas for deformed potatoes;
[0029] Figure 8 It is the sunken area of the potato;
[0030] Figure 9 It is an algorithm for segmenting deformed potatoes;
[0031] Figure 10 is the error threshold selection;
[0032] Figure 11 Predicted and actual quality for potatoes. DETAILED DESCRIPTION
[0033] The process of detecting, segmenting and evaluating deformed potatoes is as follows: Figure 1 As shown in the figure, the selection and extraction of deformed potato image features determines the accuracy of deformed potato recognition. The degree of convexity and concavity of the edge contour of a deformed potato is significantly different from that of a normal potato, and this degree of concavity and convexity can be quantified using a specific algorithm. Therefore, the convex hull contour extracted from the potato is compared with the shape features of the potato edge contour to determine the potato deformity. The separation point between the potato contour and the convex hull contour is then found, and the optimal segmentation point for the deformed potato is found. All potential segmentation methods are traversed to segment the potato until the deformed potato is detected as a normal potato, thus completing the segmentation of the deformed potato.
[0034] All potatoes used in the experiment were from Xueyu No. 1 variety of potatoes produced by Xuechuan Liupanshan Food (Ningxia) Co., Ltd. in Guyuan City, Ningxia Hui Autonomous Region. A total of 692 deformed potatoes and 140 normal potatoes were selected for the experiment. Figure 2 As shown, sample selection was done to cover nearly all possible potato sizes and shapes. Normal potatoes were oval or sub-round, while deformed tubers were selected, including those with one or more small, tumor-like tubers protruding from the tuber's eye, and those with secondary growth deformities, such as runners resembling candied haws chains.
[0035] Potato pretreatment process Figure 3 To extract the potato outline, the image is converted from the RGB color model to the HSV color model. A dilation and erosion operation is performed based on the HSV color threshold range. Binarization is then performed to construct a mask. The mask is then used to find the local maximum of the gradient using the Canny operator. A Gaussian smoothing filter is used for convolution noise reduction. A pair of convolution arrays is used to calculate edge gradients and directions. Finally, non-maximum suppression is used to remove non-edge lines. Hysteresis thresholds (high and low thresholds) are used to detect and connect edges. The contour with the largest area is used as the potato outline.
[0036] like Figure 4 As shown in the figure, the Douglas-Peucker algorithm is used to reduce the number of curve representation points in order to reduce redundant points in the trajectory while maintaining the overall shape of the curve, thereby reducing data complexity and improving processing speed. The extraction of potato contours will produce a lot of noise points that affect data processing, so the Douglas-Peucker algorithm is used to reduce the noise of the potato contours. Figure 5 The following figure shows a partial curve of a potato outline, consisting of a set of vertices S = {P0, P1, P2, P3, P4, P5}. P1, P2, P3, P4, and P5 are connected in pairs, and an error threshold δ is set. The distances d1, d2, d3, and d4 between P1, P2, P3, and P4 and the line are calculated, and the maximum distance d3 is found. D3 is compared with the error threshold δ. If d3 > δ, vertex P3 is retained. Since d1, d2, and d4 are all less than the threshold δ, P1, P2, and P4 are discarded. P0, P3, and P5 are connected to form a line segment. The curve segment formed by the line segments P0P3 and P3P5 is the desired curve segment. This completes the curve noise removal, successfully simplifies the potato outline, and provides the data foundation for subsequent image analysis. Figure 5 The first row of two pictures shows the point set S={P0,P1,P2....P 11}, α 32 >α 43 , so discard point P3 and push P4 into the stack. Figure 5 The second row of two pictures α 42 <α54 , push P5 into the stack; discard point P9 and push P10 into the stack. 65 >α 76 , so discard point P6 and push P7 into the stack. Figure 5 The third row shows the convex hull of the points in the stack. The potato outline is drawn as a solid line, and the convex hull outline is drawn as a dotted line.
[0037] This paper uses Graham's scan to calculate the convex hull of the potato contour. Given a two-dimensional point set S0 = {P0, P1, P2....P n}, set the coordinate axis to the lower left of all two-dimensional point sets, select the lower left point P0 and the point with the smallest distance from the coordinate origin, and set it as the starting point; the remaining points are sorted in the counterclockwise direction of point P0, and the remaining point set after sorting is {P0,P1,P2....P n}, let the stack Stack be empty, Stack[i] represents the i+1th element in the stack Stack, and use k to represent the number of elements in the stack; if there are k elements in the stack, then Stack[k-1] is the top element of the stack; for each PUSH operation, the number of stack elements k=k+1; for each POP operation, the number of stack elements k=k-1.
[0038] The specific operation to obtain the convex hull of the potato contour is as follows Figure 5 As shown, it is known that the point set after the potato contour is sorted is S'={P'0,P'1,P'2....P' n}, select the lower left point P'0 as the starting point, push P'0 and P'1 into the stack; calculate The angle with the positive direction of the X axis is denoted as α ij , at this time ij=1; if α ij ≤α n(n-1) , The angle with the positive direction of the X axis, (n>i), then point P i Perform PUSH operation and push into stack Stack, k=k+1,j=i,i=i+1; if α ij >α n(n-1) , then i=n, calculate Angle α with the positive direction of the X axis ij , repeat the above operation; finally, the point set in stack S1 is the convex hull contour of the potato, with K vertices.
[0039] After obtaining the potato outline and its convex hull contour, we extract its contour features. The contour of a deformed potato differs significantly from its convex hull contour. This paper extracts shape features such as the potato's contour area, contour length, convex hull contour area, and convex hull contour length, and then applies them to a support vector machine model for shape classification.
[0040] After completing the classification of deformed and non-deformed potatoes, the deformed parts of the deformed potatoes are located and segmented. Common segmentation methods for deformed potatoes are as follows: Figure 6 As shown in the figure, the red part is the part of the deformed potato to be removed. Finding the best splitting method generally follows two principles: (1) the potatoes after splitting are normal, non-deformed potatoes; (2) as much of the deformed potato as possible should be retained to improve its utilization rate and reduce the loss of deformed potatoes. Research and observation of manual splitting methods have shown that the essence of deformed potatoes is to remove the raised part, and the splitting position is located in the concave area of the potato.
[0041] First, the key points in the potato image are detected. Key points are usually located in deformed parts or uneven areas on the surface, which are important for subsequent segmentation. Then, based on the key points, the optimal segmentation method is tried. Figure 7 The figure shows a deformed potato. When the deformity is removed, it is found that when the segmentation key point is located at the vertex of the concave area of the potato, it is in the best segmentation position. It can not only accurately segment the deformed part of the potato, but also greatly retain the usable part of the potato.
[0042] like Figure 8 As shown, by combining the potato convex hull contour two-dimensional point set S1 with the potato contour two-dimensional point set S0, the potato concave area contour point set S2 = S0-S1+S3 = {A, B, P0, P1...P n .},in The point set S3 = {A, B} is the set of points separating the convex hull from the potato. Find the concave vertex inside, that is, the vertex P(X0, Y0) farthest from line AB. Let the coordinates of point A be (X1, Y1), the coordinates of point B be (X2, Y2), and the key point in the potato outline be P(X0, Y0).
[0043] It can be used to find the distance d from a point to a line:
[0044]
[0045] Based on the detected key points, the image segmentation algorithm is used to segment the potato image into normal parts and deformed parts. In the segmentation process, assuming that there are n (n>1) key points, there are There are two segmentation methods, connecting the segmentation key points in pairs to form a segmentation line to segment the potato; if there is one key point, there are two segmentation methods, connecting the two adjacent vertices to form a straight line as the segmentation line, extracting the contour features of the segmented potato and sending it to SVM for prediction. If the result is returned as a normal potato, the contour of this group of segmented potatoes is compared with the previous group, and the group with a larger area is retained; if the result is returned as a deformed potato, continue to segment the next key point, repeat the above steps until the result is returned as a normal potato, and compare it with the retained group, and retain the group with a larger area.
[0046] like Figure 9 The first figure shows the segmentation process of a deformed potato. The key point detection algorithm found a total of 3 key points, so there are 3 segmentation methods. The first round of segmentation of the deformed potatoes was performed, and the external contour feature parameters of the segmented potatoes were pushed into the trained deformity prediction model. It was found that the first and second groups of potatoes had been segmented. The third group of potatoes was still deformed, so the segmentation of the third group of potatoes continued. Figure 9 As shown in the second figure, when segmenting the third group of potatoes, it was discovered that the deformed potatoes had only one key segmentation point. The adjacent point straight line segmentation method was used. The areas of all potatoes that met the criteria after segmentation were compared, and the segmentation method that retained the largest area was selected as the optimal segmentation method. The deformed potato segmentation algorithm now completes the segmentation.
[0047] In Douglas-Peucker contour simplification, the choice of error threshold determines the effectiveness of contour simplification. To compare the contour data after potato image preprocessing with the contour data after Douglas-Peucker simplification, 20 deformed potatoes were selected to test the area loss rate and vertex number simplification rate of the potato contours at different thresholds, where:
[0048]
[0049] like Figure 10 As shown in the figure, the error threshold δ = 17 was set in the experiment, which effectively reduced the number of vertices without causing much loss in the contour area. The results show that the Douglas-Peucker simplification algorithm can accurately reflect the actual characteristics of the potato contour while effectively reducing the number of contour vertices.
[0050] A total of 189 potatoes were tested. 70 deformed potatoes and 63 normal potatoes were selected as the training set; 29 deformed potatoes and 27 normal potatoes were selected as the test set. Different combinations of potato contour features were tested for classification using the SVM. Table 1 shows that the original area, original perimeter, and area ratio contour features performed best for deformed potato classification, with an overall accuracy of 95.8%. The classification success rates for deformed potatoes were 96.0% and for normal potatoes were 95.6%. Therefore, the original area, original perimeter, and area ratio contour features were selected to create the dataset for testing the performance of the SVM classification model.
[0051] Table 1 Test results of different feature quantity combinations
[0052]
[0053] After the model was established, 50 normal potatoes and 50 deformed potatoes were selected for online prediction to test the accuracy of the model. In order to make the prediction data more reasonable and reliable, this paper conducted mixed experiments on normal potatoes and deformed potatoes, and comprehensively analyzed and calculated the detection accuracy.
[0054] Table 2 Deformed potato identification test results
[0055]
[0056] The classification results are shown in Table 2. Two deformed potatoes were identified as normal potatoes, with an accuracy rate of 96.0%. The misidentification occurred because the potatoes had a slight bulge, making the deformity less obvious. Three normal potatoes were identified as deformed potatoes, with an accuracy rate of 94.0%.
[0057] To verify the effectiveness of our segmentation algorithm, this experiment used existing mainstream deep learning image segmentation models, U-Net, FastSCNN, and Deeplabv3+, for comparative testing. We constructed a deformed potato segmentation dataset for training and testing our algorithm model. The dataset contains 1,380 images. An experienced professional labeled the deformed potato images using Labelme software, and the image labels were saved in PNG format. The training set and validation set ratio was 8:2. The validation set data was used to compare our segmentation algorithm with the segmentation model. The model training learning rate was 0.005, the batch size was 1, and the number of iterations was 200. We selected 276 deformed potato images for validation testing. The results showed that our segmentation algorithm achieved an average accuracy of 95.38% and a recall of 95.77%.
[0058] The algorithm extracts the morphological features of deformed potatoes after segmentation and uses them to predict the weight of the usable portion of the deformed potatoes. A total of 84 images and corresponding data on the weight of the potatoes after segmentation were collected for testing and validation of the algorithm. As shown in Table 3, a linear regression analysis was conducted using the area, perimeter, length, and width of the segmented potatoes as independent variables and the post-segment weight as the dependent variable. The model R-squared value was 0.884, indicating that the area, perimeter, length, and width explained 88.4% of the variation in post-segment weight. The model passed the F-test (F=150.443, p=0.000<0.05), indicating that the area, perimeter, length, and width have an impact on the post-segment weight.
[0059] Table 3 Multiple linear regression results of potato quality
[0060]
[0061] Figure 11 The distribution of estimated and actual weights obtained using the potato prediction model is shown. The data is evenly distributed within each weight range, with similar estimation errors. While some individual data are of relatively poor quality and exhibit relatively large errors, accounting for errors in the shape of individual potatoes, the model achieves good results overall, with a calculated RMSE of 19.63 g.
Claims
1. A method for segmenting deformed potato defects based on machine vision, characterized in that: The steps include: Step 1: Collect deformed potato image information, perform preprocessing operations, and extract potato masks; Step 2: Extract the potato outline based on the obtained mask and calculate the external feature parameters of the outline; simplify the potato outline using the Douglas-Peucker method and store the outline vertices; Step 3: Calculate the convex hull of the potato contour using the Graham scanning method and extract the external feature parameters of the convex hull; Step 4: Compare the external feature parameters of the potato convex hull with the external features of the potato's own contour, and then predict whether the potato is deformed by training the collected data with a support vector machine (SVM); Step 5: When the potato is deformed, scan the concave points of the potato contour, extract the concave points, start the algorithm segmentation, obtain the optimal segmentation path, and extract the area and perimeter of the segmented potato contour to predict the weight of the potato after removing the deformed part.
2. The method for segmenting deformed potato defects based on machine vision according to claim 1, characterized in that: The specific implementation process of extracting the potato mask is: converting the image from the RGB color model to the HSV color model, setting the threshold range of the color in the HSV to perform dilation and corrosion operations, and then performing binarization processing to construct the mask.
3. The method for segmenting deformed potato defects based on machine vision according to claim 2, characterized in that: The specific implementation process of step 2 is as follows: For the mask, the Canny operator is used to find the local maximum of the gradient. A Gaussian smoothing filter is used for convolution noise reduction. A pair of convolution arrays is used to calculate the edge gradient and direction. Finally, non-maximum suppression is used to remove non-edge lines. A hysteresis threshold is used to detect and connect edges. The contour with the largest area is used as the potato outline. The Douglas-Peucker algorithm is used to reduce the noise of the potato contour and obtain the contour vertices.
4. The method for segmenting deformed potato defects based on machine vision according to claim 3, characterized in that: The specific implementation process of step 3 is as follows: Step 3.1, use Graham scanning method to calculate the convex hull of potato contour, given the two-dimensional point set S0 of contour vertices = {P0, P1, P2....P n }, set the coordinate axis to the lower left of all two-dimensional point sets, select the lower left point P0 and the point with the smallest distance from the coordinate origin, and set it as the starting point; the remaining points are sorted in the counterclockwise direction of point P0, and the remaining point set after sorting is {P0,P1,P2....P n }, let Stack be empty, Stack[i] represents the i+1th element in the stack, and k represents the number of elements in the stack; if there are k elements in the stack, then Stack[k-1] is the top element of the stack; for each PUSH operation, the number of stack elements k=k+1; for each POP operation, the number of stack elements k=k-1; Step 3.2: The point set after the potato outline is sorted in step 3.1 is S' = {P'0, P'1, P'2....P' n }, select the lower left point P'0 as the starting point, push P'0 and P'1 into the stack; calculate The angle with the positive direction of the X axis is denoted as α ij , at this time ij=1; if α ij ≤α n(n-1) , The angle with the positive direction of the X axis, (n>i), then point P i Perform PUSH operation and push into stack Stack, k=k+1,j=i,i=i+1; if α ij >α n(n-1) , then i=n, calculate Angle α with the positive direction of the X axis ij , repeat the above operation; finally, the point set in stack S1 is the convex hull contour of the potato, with K vertices. Step 3.3: After obtaining the potato contour and the potato convex hull contour, extract the potato contour shape features, and extract the potato contour area, contour length, convex hull contour area, and convex hull contour length.
5. The method for segmenting deformed potato defects based on machine vision according to claim 4, characterized in that: The specific implementation process of step 5 is as follows: By combining the potato convex hull contour two-dimensional point set S1 with the potato contour two-dimensional point set S0, we can find the potato concave area contour point set S2 = S0-S1+S3 = {A, B, P0, P1...P n .}, where S0-S1={x∈S0 and }, point set S3 = {A, B} is the set of separation points between the convex hull and the potato, find the inner concave vertex, that is, find the vertex P(X0, Y0) farthest from the line AB, let the coordinates of point A be (X1, Y1), the coordinates of point B be (X2, Y2), and the key point in the potato outline be P(X0, Y0); According to the detected key points, the image segmentation algorithm is used to segment the potato image into normal part and deformed part. In the segmentation process, assuming that there are n (n>1) key points, there are There are two segmentation methods: the key points are connected in pairs to form a segmentation line to segment the potato. If there is only one key point, there are two segmentation methods: the two adjacent vertices are connected to form a straight line as the segmentation line. The contour features of the segmented potato are extracted and sent to the SVM for prediction. If the result returns a normal potato, the contour of this group of segmented potatoes is compared with the previous group, and the group with the larger area is selected and retained. If the result is a deformed potato, continue segmenting the next key point and repeat the above steps until the result is a normal potato. Compare it with the retained group and retain the group with the larger area.