A method for automatically locating the outermost single zongzi leaf from a stack of zongzi leaf images

Abstract

This invention discloses a method for automatically locating the outermost single zongzi leaf from an image of a stack of zongzi leaves. The method includes the following steps: S1, locating the main vein of the outermost zongzi leaf; S2, automatically segmenting the entire stack of zongzi leaves from the background; S3, automatically segmenting the outermost single zongzi leaf from the entire stack. The advantages are: high accuracy in automatically detecting the main vein of the outermost zongzi leaf in an image of a stack of zongzi leaves; and automatic and rapid identification of the outermost single zongzi leaf and calculation of its various digital features, such as the coordinates of the main vein, the centroid of the zongzi leaf, and the area of ​​the zongzi leaf. This method provides a purely vision-based mathematical algorithm for robotic arms in industrial production to accurately grasp single zongzi leaves.

Description

Technical Field

[0001] This invention relates to the field of digital image processing technology, and in particular to a method for automatically locating the outermost single leaf from an image of a stack of zongzi leaves. Background Technology

[0002] Digital image processing is a rapidly emerging and popular technology, encompassing techniques such as image segmentation, image recognition, and image restoration or enhancement. As computer technology becomes increasingly integrated with industrial production, the application areas of digital image processing technology are constantly expanding, with industrial automation being a significant one. Meanwhile, image segmentation and edge detection are hot topics in the practical application of digital image processing technology in production, and also among the most challenging problems in image processing.

[0003] Image segmentation, due to its inherent importance and difficulty, has attracted significant research efforts since the 1970s. Existing image segmentation methods can be broadly categorized as follows: threshold-based segmentation, region-based segmentation, edge detection-based segmentation, deep learning-based segmentation, and segmentation methods based on specific theories such as wavelet analysis, genetic algorithms, and active contour models. However, to date, different methods have their advantages and disadvantages for different segmentation tasks, and a universally applicable and perfect image segmentation method still exists.

[0004] Currently, the wrapping process in the zongzi (sticky rice dumpling) industry is largely manual, resulting in high labor intensity, demanding technical skills, and low efficiency, severely hindering the industry's development. The application of mechatronics technology can reduce labor intensity and improve production efficiency, expanding production scale. This involves the identification of zongzi leaves. By positioning the outermost leaves, a robotic arm can automatically and accurately locate and sort the outermost leaves from a stack of leaves, allowing them to proceed to the feeding stage. Accurate leaf positioning effectively reduces the rate of defective products, promoting standardized and regulated production.

[0005] Currently, technologies related to image recognition of zongzi leaves are mostly used to inspect the quality of zongzi leaves to reduce manual sorting costs, or to extract them into individual leaves using mechanical methods such as suction cups before positioning and subsequent operations. However, research on how to directly locate the surface leaves within a stack of zongzi leaves is limited. Furthermore, in automated production processes, recognition speed is crucial for productivity, and some basic image segmentation algorithms cannot meet the accuracy and speed requirements of industrial production when dealing with overlapping zongzi leaf images with large overlapping areas and high similarity. In conclusion, no effective solution has yet been proposed for identifying the surface leaves within overlapping zongzi leaves to meet industrial production requirements. Summary of the Invention

[0006] The purpose of this invention is to provide a method for automatically locating the outermost single leaf from an image of a stack of zongzi leaves, thereby solving the aforementioned problems existing in the prior art.

[0007] To achieve the above objectives, the technical solution adopted by the present invention is as follows:

[0008] A method for automatically locating the outermost single leaf from an image of a stack of zongzi leaves includes the following steps:

[0009] S1. Location of the main vein of the outer layer of bamboo leaves:

[0010] For the RGB channel color image of the zongzi leaf, the gradient operator is applied to the saturation channel image of the RGB channel color image of the zongzi leaf to obtain an image containing the main vein and the edge of the zongzi leaf. The Hough transform algorithm is then used to detect lines in the image to automatically locate the main vein of the surface zongzi leaf.

[0011] S2. Automatic segmentation of the entire stack of zongzi leaves from the background:

[0012] Based on the prior geometric feature of the long and narrow shape of the zongzi leaves, a long and narrow prior ellipse containing the outermost whole zongzi leaf with its major axis coinciding with the main vein is set. The long and narrow prior ellipse is combined with edge detection based on the structure tensor to construct the first image segmentation energy function. The first image segmentation energy function is solved by the graph maximum flow algorithm to segment the whole stack of zongzi leaves from the background in the image.

[0013] S3. Automatic separation of the outermost single bamboo leaf from the entire stack of bamboo leaves:

[0014] Based on the prior spatial position of the outermost leaf inside the stack of leaves, this prior spatial position is combined with tensor-based edge detection to construct a second image segmentation energy function. The second image segmentation energy function is solved using a graph-based maximum flow algorithm to segment the outermost single leaf from the stack of leaves.

[0015] Preferably, step S1 specifically includes the following:

[0016] S11. Convert the RGB channel color image of the zongzi leaf into a color space composed of hue, saturation and brightness;

[0017] S12. Calculate the gradient of the saturation channel image, and obtain a binary image of the image formed by the magnitude of the gradient using a set empirical threshold. In the binary image, the main vein is represented by the longest white straight line segment.

[0018] S13. Set the pixel values ​​of the binary image near the image edge to 0 to eliminate the influence of the long straight line segments around the boundary of the binary image on the main vein.

[0019] S14. Use the Hough transform algorithm to detect line segments in the binary image processed in step S13, and extract the longest line segment as the main vein of the surface zongzi leaf.

[0020] Preferably, step S2 specifically includes the following:

[0021] S21. Using each pixel of the zongzi leaf image as v x For each vertex, two additional special vertex sources S and sink T are added to form an undirected ST graph.

[0022] The set of vertices of graph G It contains two types of vertices. The first type is the discrete pixel v of the zongzi leaf image. x The second category consists of source S and sink T; the set of edges of graph G. It contains two types of edges, the first type is the pixel v x The edges connected to S and T respectively are called S-connections and T-connections. The second type is the pixel v. x It has m connected neighbors around it Inner pixel v y An edge connecting two points is called an N-connection;

[0023] S22, For pixels Set the S-connection weight to +∞ and the T-connection weight to 0;

[0024] Construct an elliptical region with the midpoint of the main vein as the center of the narrow a priori ellipse, and the major semi-axis coinciding with the main vein. For pixels Set the S-connection weight to +∞ and the T-connection weight to 0;

[0025] The four borders of the image of the zongzi leaf Belongs to the background, pixels Satisfy in minimum cut For these pixels, set the S-connection weight to 0 and the T-connection weight to +∞;

[0026] in, and In any cut of graph G In the middle, if pixel point Then the pixel v is determined x In the image background, conversely, if pixels... Then the pixel v is determined x Inside the whole stack of bamboo leaves;

[0027] S23. Determine the largest eigenvalue of the smooth structure tensor matrix at each pixel, and use the largest eigenvalue to determine the pixel v. x With its four connected neighbors Inner pixel v y N-connection weights;

[0028] S24. Construct the first image segmentation energy function based on the S-connection weights, T-connection weights, and N-connection weights of each pixel.

[0029]

[0030] Where, ε 1 (φ) is the energy function for the first image segmentation; d 1 (x) represents the S-connection weights; d 2 (x) represents the T-connection weights; w(x,y) represents the N-connection weights; φ is the characteristic function of the graph cut, if the pixel point Then φ(x) takes the value 1, and conversely, if φ(x) takes a value of 0; if the pixel point Then φ(y) takes the value 1, and conversely, if φ(y) takes the value 0;

[0031] S25. Solve the first image segmentation optimization model using the maximum flow algorithm for graphs, as expressed below.

[0032]

[0033] Preferably, step S3 specifically includes the following:

[0034] S31, using each pixel of the entire stack of zongzi leaves as v x′ As vertices, two additional special vertex sources S′ and sink T′ are added to form an undirected S′-T′ graph.

[0035] The set of vertices of graph G′ It contains two types of vertices. The first type is the discrete pixel v of the entire stack of zongzi leaves. x′ The second category consists of the source S′ and sink T′; the set of edges of graph G′. It contains two types of edges, the first type is the pixel v x′ The edges that connect to S′ and T′ respectively are called S′-connections and T′-connections. The second type is the pixel v. x′ It has m connected neighbors around it Inner pixel v y′ The edges connecting them are called N'-connections;

[0036] S32, For pixels Set the S'-connection weight to +∞ and the T'-connection weight to 0;

[0037] Construct an elliptical region with the midpoint of the main vein as the center of the narrow a priori ellipse, and the major semi-axis coinciding with the main vein. For pixels Set the S'-connection weight to +∞ and the T'-connection weight to 0;

[0038] The edges of the whole stack of bamboo leaves Belongs to the background, pixels In the minimum cut, v satisfies x′ ∈ For these pixels, set S'-connection weight to 0 and T'-connection weight to +∞;

[0039] in, and In any cut of graph G′ In the middle, if pixel point Then the pixel v is determined x′ On the entire stack of bamboo leaves, beyond the outermost single bamboo leaf, conversely, if pixel... Then the pixel v is determined x′ On the outermost single leaf of the zongzi;

[0040] Indicative function 1-φ * Perform a circular expansion operation on the kernel with radius to construct a circular region. For pixels Set the S'-connection weight to 0 and the T'-connection weight to +∞;

[0041] S33. Determine the largest eigenvalue of the smooth structure tensor matrix at each pixel, and use the largest eigenvalue to determine the pixel v. x′ With its four connected neighbors Inner pixel v y′ N'-connection weights;

[0042] S34. Based on the S'-connection weights, T'-connection weights, and N'-connection weights of each pixel, construct the second image segmentation energy function.

[0043]

[0044] Where, ε 2 (φ) is the energy function for the second image segmentation; d 1 (x′) represents the S'-connection weights; d 2 (x′) is the T′-connection weight; w(x′,y′) is the N′-connection weight; φ is the characteristic function of the graph cut, if the pixel point Then φ(x′) takes the value 1, and conversely, if φ(x′) takes a value of 0; if the pixel point Then φ(y′) takes the value 1, and conversely, if φ(y′) takes the value 0;

[0045] S35. Solve the second image segmentation energy function using the maximum flow algorithm for graphs, as expressed below.

[0046]

[0047] Preferably, step S23 specifically involves converting the RGB channel color image of the zongzi leaf into a LAB color image, which is represented as follows: Its gradient Represented as,

[0048]

[0049] Based on gradient Obtain the smooth structure tensor matrix H.

[0050]

[0051] Among them, f 1 ,f 2 ,f 3 G represents the components of the three LAB channels in the LAB color image f; σ σ represents a Gaussian kernel with parameter σ, and * represents convolution; Represent a bivariate function f j Regarding the i-th variable x i The partial derivatives; let λ be the maximum eigenvalues ​​of H at x and y for each pixel. max (x) and λ max (y), then pixel v x With its four connected neighbors Inner pixel v y The N-connection weights are set to,

[0052]

[0053] The beneficial effects of this invention are: 1. The method of this invention can automatically and quickly extract the main vein, edge contour, and centroid of the outermost single zongzi leaf in images with high overlap and similarity. 2. Compared with existing segmentation methods, the method of this invention can obtain accurate global solutions faster, significantly improving accuracy and speed in practical zongzi leaf segmentation problems, which is of great significance to improving industrial automation productivity. 3. The method of this invention performs secondary refinement segmentation based on coarse segmentation, improving segmentation accuracy and better handling the problem of the overlap between the outermost zongzi leaf contour and the lower zongzi leaf edge; the two-stage segmentation and recognition improves accuracy while ensuring high segmentation speed. Attached Figure Description

[0054] Figure 1 This is a flowchart illustrating the method in an embodiment of the present invention;

[0055] Figure 2 This is a schematic diagram of the method in an embodiment of the present invention;

[0056] Figure 3 This is a schematic diagram illustrating the extraction effect of the main vein of the outermost layer of zongzi leaves in an embodiment of the present invention;

[0057] Figure 4 This is a schematic diagram of the segmentation effect of the outermost layer of zongzi leaves in an embodiment of the present invention. Detailed Implementation

[0058] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0059] like Figure 1 and Figure 2 As shown in this embodiment, a method for automatically locating the outermost single leaf from a stack of zongzi leaves is provided. This method can automatically and quickly extract the main vein, edge contour, and centroid of the outermost single leaf in zongzi leaf images with high overlap and similarity. The method includes three parts:

[0060] I. Locating the main veins of the outermost layer of bamboo leaves

[0061] This step aims to locate and extract the main veins of the surface layer of the bamboo leaves, thereby completing the positioning of the surface layer of the bamboo leaves.

[0062] Specifically: For the RGB channel color image of the zongzi leaf, the gradient operator is applied to the saturation channel image of the RGB channel color image of the zongzi leaf to obtain an image containing the main vein and the edge of the zongzi leaf, and the Hough transform algorithm is used to detect lines in the image to automatically locate the main vein of the surface zongzi leaf.

[0063] In this embodiment, the identification of the main vein of the given original image of the bamboo leaf can be approximated as the problem of identifying the longest line segment. Therefore, the main vein of the surface bamboo leaf can be accurately identified using the Hough transform line detection method. Specifically, the steps are as follows:

[0064] 1. Convert the RGB channel color image of the zongzi leaf into the HSV color space, which consists of hue, saturation, and value / brightness.

[0065] 2. Since the main vein of the leaf presents a significantly different feature from other parts in the S channel, the gradient of the image in the S channel is calculated, and a binary image is obtained by taking the magnitude of the gradient and using a set empirical threshold (such as 0.0008). In the binary image, the pixel value 1 (white) represents the edge of the leaf and other details in the image, and 0 (black) is the background. The main vein of the leaf is roughly represented by the longest white straight line segment.

[0066] 3. In order to eliminate the influence of long straight line segments that may be generated around the image boundary on the identification of the main leaf vein, the pixel values ​​near the image edge of the binary image are set to 0;

[0067] 4. Use the Hough transform algorithm to detect line segments in the binary image from the previous step, and extract the longest line segment as the main vein of the surface leaf.

[0068] II. Automatic segmentation of stacked zongzi leaves from the background

[0069] The core idea of ​​this part is as follows: Based on the prior geometric feature that the zongzi leaves are long and narrow, a long and narrow prior ellipse containing the outermost whole zongzi leaf with its major axis coinciding with the main vein is set. The long and narrow prior ellipse is combined with edge detection based on structure tensor to construct the first image segmentation energy function. The first image segmentation energy function is solved based on the graph maximum flow algorithm to segment the whole stack of zongzi leaves from the background in the image.

[0070] like Figure 3 As shown, in this embodiment, based on the results of the main vein extraction in the first part, to improve the segmentation accuracy, this invention sets a narrow, elongated prior ellipse whose major axis coincides with the main vein, based on the prior geometric feature that the zongzi leaf is narrow and elongated. This ellipse is contained within the outermost layer of the entire zongzi leaf. This step can provide geometric and spatial location priors for subsequent image segmentation algorithms, improving the accuracy of image segmentation. The above ellipse prior is combined with the graph cut segmentation optimization model, and the model is solved using a graph-based maximum flow algorithm to automatically segment the entire stack of zongzi leaves from the background in the image. This invention, based on the graph cut algorithm, uses various location priors to automatically segment the entire stack of zongzi leaves from the background. This algorithm can obtain a more accurate global solution faster than existing graph cut methods, significantly improving accuracy and speed in practical zongzi leaf segmentation problems, which is of great significance for improving industrial automation productivity. Both theoretical analysis and experimental results show that the segmentation method used in this invention is highly competitive compared to some existing segmentation methods.

[0071] This invention combines prior spatial location information between the stack of zongzi leaves and the background image to redesign a graph such that the minimum cut of this graph is the ideal segmentation result. Furthermore, it provides a mathematically theoretical optimization problem corresponding to this minimum cut problem. The maximum flow algorithm is then used to solve this optimization problem. This algorithm performs excellently in the segmentation problem applied to this example. The algorithm in this invention improves the speed and accuracy of automatic segmentation of the stack of zongzi leaves and the background, which is highly significant for increasing production capacity in automated production. Specifically, it includes the following steps:

[0072] 1. Using each pixel of the zongzi leaf image as a reference... x For each vertex, two additional special vertex sources S and sink T are added to form an undirected ST graph. The source point is the center point of the entire stack of zongzi leaves, and the sink point is the center point of the zongzi leaf image.

[0073] The set of vertices of graph G It contains two types of vertices. The first type is the discrete pixel v of the zongzi leaf image. x The second category consists of source S and sink T; represented as a set of vertices. Ω represents the set of pixel coordinates.

[0074] The set of edges of graph G It contains two types of edges, the first type is the pixel v x The edges that connect to S and T respectively are called S-connections and T-connections, and are represented as:

[0075] The second category is pixels v. x It has m connected neighbors around it Inner pixel v y An edge connecting two points is called an N-connection, represented as: For v x ,v y The subscripts x and y are used to distinguish different pixels, such as v1, v2; when Ω is the set of positive integers {1, 2, ..., n}, the relevant pixels can be represented as v1, v2, ..., v n .

[0076] Therefore, all edges of graph G can be represented as: Each edge (v x ,v y There is a weight that symbolizes the transmission capacity. Cutting in Figure G This refers to removing some edges from G, which would allow the vertex set to be... Divide into two connected sets with no intersection. and Make and Therefore, the weights of the edges removed in each cut will form a cost.

[0077]

[0078] In any cut of this graph, if the pixel point Then v x Segmented into image background points, i.e., pixels v x In the background of the image. Conversely, That is, to identify v x Within the entire stack of bamboo leaves, the minimum cut is the one that minimizes the cost. The minimum graph cut. The minimum cut of a graph can be computed using its dual problem, the maximum flow. The next step will utilize the prior spatial positions of the bamboo leaves and the background in the image to construct a special ST graph such that the image segmentation corresponding to the minimum cut of the graph can precisely separate the entire stack of bamboo leaves from the image background.

[0079] 2. Use the prior position of the stack of zongzi leaves and the background to define the S-connection and T-connection weights of graph G.

[0080] (1) First, the pixel v at the position of the main vein extracted in the first step x The pixel should be located within the bamboo leaf area; such a pixel should satisfy the minimum cut condition. Therefore, S-connection weights are set. (In the actual algorithm, a very large positive number 9 × 10 is used) 9 To replace +∞), T-connection weights This weight setting ensures that the T-connection is definitely one of the edges removed in the minimum cut (otherwise, the energy of the minimum cut would be +∞, which contradicts the definition of the minimum cut). Therefore, we have

[0081] (2) Then, by utilizing the geometric prior of the narrow shape of the zongzi leaf, the prior comfort is expanded to improve the accuracy of the segmentation algorithm. The length of the extracted main leaf vein straight line segment is calculated as l. Taking the midpoint of the main leaf vein as the center of the narrow prior ellipse, the semi-major axis coincides with the main leaf vein, and the length is a = αl (in the algorithm, it is taken as l). The minor semi-axis b = β (β = 40 pixels in the algorithm) is used to construct an elliptical region. For pixels Set the S-connection weight to (In the actual algorithm, a very large positive number 9 × 10 is used) 9 To replace +∞) and T-connection weights The reason for setting this weight is the same as in step (1).

[0082] The four borders of the image of the zongzi leaf Belongs to the background, pixels Satisfy in minimum cut Therefore, for these pixels, S-connection weights are set. and T-connection weights (In the actual algorithm, a very large positive number 9 × 10 is used) 9 To replace +∞). In this case, such pixels should have their S-connections removed in the minimum cut (otherwise the energy of the minimum cut would be +∞, which contradicts the definition of the minimum cut), therefore

[0083] 3. Use image boundary information to set N-connection weights

[0084] The purpose of setting N-links is to ensure that the edges removed in the minimum cut fall precisely at the boundary between the stack of zongzi leaves and the background image. Therefore, pixels at the boundary need to be assigned smaller weights, while other areas can be assigned larger weights. First, a boundary detection function is constructed to roughly detect the boundary between the stack of zongzi leaves and the background image. This invention uses the maximum eigenvalue of the structure tensor of a vector-valued function to construct an edge detection function. Compared to existing edge detection methods, the method used in this invention can more accurately detect the boundary of the stack of zongzi leaves. To more accurately detect the boundary of the zongzi leaves, the RGB image is first converted to the LAB color space. For ease of expression, this LAB color image is denoted as... x is the pixel position, f 1 ,f 2 ,f 3 These represent the components of the LAB channels in the color image f. Their gradients... It is a 3×2 matrix.

[0085]

[0086]

[0087] The smoothed structure tensor matrix H is a 3×3 matrix. Represent a bivariate function f j Regarding the i-th variable x i The partial derivatives of G. By calculating the eigenvalues ​​of matrix H, the maximum eigenvalue of H at each pixel can be obtained. σ It is a Gaussian kernel with parameter σ, and * represents convolution. Let the maximum eigenvalues ​​of H at each pixel x and y be λ. max (x) and λ max (y). The magnitude of this function value reflects the strength of the boundary of the vector image and can be regarded as a boundary detection function for zongzi leaves.

[0088] Then pixel vx With its four connected neighbors Inner pixel v y The N-connection weights are set to,

[0089]

[0090] 4. Construct the first image segmentation energy function based on the S-connection weights, T-connection weights, and N-connection weights of each pixel.

[0091]

[0092] Where, ε 1 (φ) is the energy function for the first image segmentation; d 1 (x) represents the S-connection weights, i.e. d 2 (x) represents the T-connection weights, i.e. w(x,y) represents the N-connection weights, i.e. φ is the characteristic function of the graph cut, if the pixel point Then φ(x) takes the value 1, and conversely, if φ(x) takes a value of 0; if the pixel point Then φ(y) takes the value 1, and conversely, if φ(y) takes the value 0.

[0093] 5. Solve the first image segmentation optimization model using the maximum flow algorithm for graphs, as follows:

[0094]

[0095] It can be mathematically proven that the minimum element φ of the above energy... * The minimum cut of graph G corresponding to the above construction The characteristic function is given (detailed proof omitted). Therefore, this invention utilizes the max-flow algorithm to obtain the minimum element φ of the segmentation energy function. * Separate the stack of zongzi leaves from the background of the image.

[0096] In this embodiment, the constructed graph G fully utilizes the prior spatial position of the main vein of the zongzi leaf extracted in the first step and the entire stack of zongzi leaves located in the middle of the image, resulting in a segmentation energy and algorithm with distinctive features. This step can accurately separate the entire stack of zongzi leaves from the image background.

[0097] III. Automatic separation of the outermost single bamboo leaf from the entire stack of bamboo leaves

[0098] The core idea of ​​this part is as follows: based on the prior spatial position of the outermost zongzi leaf inside the whole stack of zongzi leaves, the prior spatial position is combined with tensor-based edge detection to construct a second image segmentation energy function. The second image segmentation energy function is solved by the graph maximum flow algorithm to segment the outermost single zongzi leaf from the whole stack of zongzi leaves.

[0099] like Figure 4 As shown, this part requires the segmentation results of the entire stack of zongzi leaves extracted in the second part. Because the overlapping of the zongzi leaves causes their edges to intersect or overlap, this step is the most difficult part in segmenting the outermost single zongzi leaf. Based on the elongated shape of the zongzi leaves, using the length and position information of the main vein, an ellipse of a preset size with the main vein as its major axis is defined. A certain range inside the ellipse must be contained within the outermost zongzi leaf. Simultaneously, using the prior knowledge that the outermost single zongzi leaf is roughly located within the entire stack, a segmentation model is constructed. Due to the use of this special prior, this algorithm can extract the outermost single zongzi leaf more accurately than existing image segmentation algorithms. Specifically, it includes the following steps:

[0100] 1. Using each pixel of the entire stack of zongzi leaves as an example. x′ As vertices, two additional special vertex sources S′ and sink T′ are added to form an undirected S′-T′ graph. The source point is the center of the outermost layer of bamboo leaves, and the sink point is the center of the entire stack of bamboo leaves.

[0101] The set of vertices of graph G′ It contains two types of vertices. The first type is the discrete pixel v of the entire stack of zongzi leaves. x′ The second category consists of the source S′ and sink T′; the set of edges of graph G′. It contains two types of edges, the first type is the pixel v x′ The edges that connect to S′ and T′ respectively are called S′-connections and T′-connections. The second type is the pixel v. x′ It has m connected neighbors around it Inner pixel v y′ The edges connecting them are called N'-connections;

[0102] 2. For pixels Set S'-connection weights and T'-connection weights

[0103] Construct an elliptical region with the midpoint of the main vein as the center of the narrow a priori ellipse, and the major semi-axis coinciding with the main vein. For pixels Set S'-connection weights and T'-connection weights

[0104] The edges of the whole stack of bamboo leaves Belongs to the background, pixels Satisfy in minimum cut For these pixels, set the S'-connection weights. and T'-connection weights

[0105] in, and In any cut of graph G′ In the middle, if pixel point Then the pixel v is determined x′ On the entire stack of bamboo leaves, beyond the outermost single bamboo leaf, conversely, if pixel... Then the pixel v is determined x′ On the outermost single leaf of the zongzi;

[0106] Indicative function 1-φ * Perform a circular expansion operation on the kernel with radius to construct a circular region. For pixels Set S'-connection weights and T'-connection weights

[0107] 3. Determine the largest eigenvalue of the smooth structure tensor matrix at each pixel, and use the largest eigenvalue to determine the pixel v. x′ With its four connected neighbors Inner pixel v y′ The N'-connection weights are denoted as

[0108] 4. Based on the S'-connection weights, T'-connection weights, and N'-connection weights of each pixel, construct a second image segmentation energy function.

[0109]

[0110] Where, ε 2 (φ) is the energy function for the second image segmentation; d 1 (x′) represents the S′-connection weights. That is; d 2 (x′) represents the T′-connection weight, i.e. w(x′,y′) is the N'-connection weight, i.e.

[0111] 5. The second image segmentation energy function is solved using the maximum flow algorithm for graphs, as expressed below.

[0112]

[0113] In this embodiment, Figure G′ relates to the position of the main leaf vein and the pixel v with the midpoint of the main leaf vein as the ellipse.x′ The S'-connection and T'-connection weights of the vertices are set exactly the same as those in Figure G in Part 2. The N-connections of Figure G' are also set exactly the same as those in Figure G in Part 2. The mathematical expression for the segmentation energy is also consistent with the segmentation energy in Part 2. The difference is that this part uses the indicator function 1-φ of the image background segmentation result obtained in Part 2. * A circular dilation operation is performed on the binary image with a kernel of radius 5 pixels. Let the dilated background region be... In order to utilize this background prior, Vertex v of the inner graph G′ x In the minimum cut, there should be Therefore, for Set S'-connection weights and T'-connection weights (In the actual algorithm, a very large positive number 9 × 10 is used) 9 To replace +∞). In this case, the S'-connection should be removed from the minimum cut (otherwise the energy of the minimum cut would be +∞, which contradicts the definition of the minimum cut), thus satisfying

[0114] Then, the maximum flow algorithm is used to find the minimum cut in graph G′, thereby separating the outermost single leaf from the entire stack of bamboo leaves. This completes the location and extraction of the outermost bamboo leaf.

[0115] By adopting the above-disclosed technical solution of this invention, the following beneficial effects are obtained:

[0116] This invention discloses a method for automatically locating the outermost single zongzi leaf from an image of a stack of zongzi leaves. The method can automatically and quickly extract the main vein, edge contour, and centroid of the outermost single zongzi leaf from images of zongzi leaves with high overlap and similarity. Because this method utilizes prior knowledge of the zongzi leaf image and shape, it significantly improves accuracy and speed in practical zongzi leaf segmentation compared to existing methods, which is highly significant for improving industrial automation productivity. This invention improves the accuracy of outer leaf segmentation by separating the entire stack of zongzi leaves from the background and then separating the single leaf from the entire stack through a two-stage segmentation process. It also better handles the problem of the overlap between the surface leaf contour and the edge of the lower leaf. This two-stage segmentation and recognition ensures high segmentation speed while also improving accuracy.

[0117] The above description is only a preferred embodiment of the present invention. It should be noted that for those skilled in the art, several improvements and modifications can be made without departing from the principle of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for automatically locating the outermost individual rice dumpling leaf from a stack of rice dumpling leaf images, characterized by: Includes the following steps, S1. Location of the main vein of the outer layer of bamboo leaves: For the RGB channel color image of the zongzi leaf, the gradient operator is applied to the saturation channel image of the RGB channel color image of the zongzi leaf to obtain an image containing the main vein and the edge of the zongzi leaf. The Hough transform algorithm is then used to detect lines in the image to automatically locate the main vein of the surface zongzi leaf. S2. Automatic segmentation of the entire stack of zongzi leaves from the background: Based on the prior geometric feature of the long and narrow shape of the zongzi leaves, a long and narrow prior ellipse containing the outermost whole zongzi leaf with its major axis coinciding with the main vein is set. The long and narrow prior ellipse is combined with edge detection based on the structure tensor to construct the first image segmentation energy function. The first image segmentation energy function is solved by the graph maximum flow algorithm to segment the whole stack of zongzi leaves from the background in the image. S3. Automatic separation of the outermost single bamboo leaf from the entire stack of bamboo leaves: Based on the prior spatial position of the outermost leaf inside the stack of leaves, this prior spatial position is combined with tensor-based edge detection to construct a second image segmentation energy function. The second image segmentation energy function is solved using a graph-based maximum flow algorithm to segment the outermost single leaf from the stack of leaves.

2. The method for automatically locating the outermost single bamboo leaf from a stack of bamboo leaf images according to claim 1, characterized in that: Step S1 specifically includes the following: S11. Convert the RGB channel color image of the zongzi leaf into a color space composed of hue, saturation and brightness; S12. Calculate the gradient of the saturation channel image, and obtain a binary image of the image formed by the magnitude of the gradient using a set empirical threshold. In the binary image, the main vein is represented by the longest white straight line segment. S13. Set the pixel values ​​of the binary image near the image edge to 0 to eliminate the influence of the long straight line segments around the boundary of the binary image on the main vein. S14. Use the Hough transform algorithm to detect line segments in the binary image processed in step S13, and extract the longest line segment as the main vein of the surface zongzi leaf.

3. The method for automatically locating the outermost single bamboo leaf from a stack of bamboo leaf images according to claim 2, characterized in that: Step S2 specifically includes the following: S21. Using each pixel of the zongzi leaf image as v x For each vertex, two additional special vertex sources S and sink T are added to form an undirected ST graph. The set of vertices of graph G It contains two types of vertices. The first type is the discrete pixel v of the zongzi leaf image. x The second category consists of source S and sink T; the set of edges of graph G. It contains two types of edges, the first type is the pixel v x The edges connected to S and T respectively are called S-connections and T-connections. The second type is the pixel v. x It has m connected neighbors around it Inner pixel v y An edge connecting two points is called an N-connection; S22, For pixels Set the S-connection weight to +∞ and the T-connection weight to 0; Construct an elliptical region with the midpoint of the main vein as the center of the narrow a priori ellipse, and the major semi-axis coinciding with the main vein. For pixels Set the S-connection weights to +∞ and the T-connection weights to 0; The four edges θΩ of the zongzi leaf image belong to the background, and the number of pixels... Satisfy in minimum cut For these pixels, set the S-connection weight to 0 and the T-connection weight to +∞; in, and In any cut of graph G In the middle, if pixel point Then the pixel v is determined x In the image background, conversely, if pixels... Then the pixel v is determined x Inside the whole stack of bamboo leaves; S23. Determine the largest eigenvalue of the smooth structure tensor matrix at each pixel, and use the largest eigenvalue to determine the pixel v. x With its four connected neighbors Inner pixel v y N-connection weights; S24. Construct the first image segmentation energy function based on the S-connection weights, T-connection weights, and N-connection weights of each pixel. Where, ε 1 (φ) is the energy function for the first image segmentation; d 1 (x) represents the S-connection weights; d 2 (x) represents the T-connection weights; w(x, y) represents the N-connection weights; φ is the characteristic function of the graph cut, if the pixel point Then φ(x) takes the value 1, and conversely, if φ(x) takes a value of 0; if the pixel point Then φ(y) takes the value 1, and conversely, if φ(y) takes the value 0; S25. Solve the first image segmentation optimization model using the maximum flow algorithm for graphs, as expressed below.

4. The method for automatically locating the outermost single bamboo leaf from a stack of bamboo leaf images according to claim 3, characterized in that: Step S3 specifically includes the following: S31, using each pixel of the entire stack of zongzi leaves as v x′ As vertices, two additional special vertex sources S′ and sink T′ are added to form an undirected S′-T′ graph. The set of vertices of graph G′ It contains two types of vertices. The first type is the discrete pixel v of the entire stack of zongzi leaves. x′ The second category consists of the source S′ and sink T′; the set of edges of graph G′. It contains two types of edges, the first type is the pixel v x′ The edges that connect to S′ and T′ respectively are called S′-connections and T′-connections. The second type is the pixel v. x′ It has m connected neighbors around it Inner pixel v y′ The edges connecting two points are called N′-connections; S32, For pixels Set the S′-connection weight to +∞ and the T′-connection weight to 0; Construct an elliptical region with the midpoint of the main vein as the center of the narrow a priori ellipse, and the major semi-axis coinciding with the main vein. For pixels Set the S′-connection weight to +∞ and the T′-connection weight to 0; The edges of the whole stack of bamboo leaves Belongs to the background, pixels Satisfy in minimum cut For these pixels, set S′-connection weight to 0 and T′-connection weight to +oo; in, and In any cut of graph G′ In the middle, if pixel point Then the pixel v is determined x′ On the entire stack of bamboo leaves, beyond the outermost single bamboo leaf, conversely, if pixelated... Then the pixel v is determined x′ On the outermost single leaf of the zongzi; Indicative function 1-φ * Perform a circular expansion operation on the kernel with radius to construct a circular region. For pixels Set the S′-connection weight to 0 and the T′-connection weight to +oo; S33. Determine the largest eigenvalue of the smooth structure tensor matrix at each pixel, and use the largest eigenvalue to determine the pixel v. x′ With its four connected neighbors Inner pixel v y′ N′-connection weights; S34. Construct a second image segmentation energy function based on the S′-connection weights, T′-connection weights, and N′-connection weights of each pixel. Where, ε 2 (φ) is the energy function for the second image segmentation; d 1 (x′) represents the S′-connection weights; d 2 (x′) represents the T′ connection weight; w(x′, y′) represents the N′ connection weight; φ is the characteristic function of the graph cut, if the pixel point Then φ(x′) takes the value 1, and conversely, if φ(x′) takes the value 0; if the pixel point Then φ(y′) takes the value 1, and conversely, if φ(y′) takes the value 0; S35. Solve the second image segmentation energy function using the maximum flow algorithm for graphs, as expressed below.

5. The method for automatically locating the outermost single bamboo leaf from a stack of bamboo leaf images according to claim 4, characterized in that: Step S23 specifically involves converting the RGB channel color image of the zongzi leaf into a LAB color image, which is represented as f = (f 1 f 2 f 3 ): Its gradient Represented as, Based on gradient Obtain the smooth structure tensor matrix H. Among them, f 1 f 2 f 3 G represents the components of the three LAB channels in the LAB color image f; σ σ represents a Gaussian kernel with parameter σ, and * represents convolution; Represent a bivariate function f j Regarding the i-th variable x i The partial derivatives; let λ be the maximum eigenvalues ​​of H at x and y for each pixel. max (x) and λ max (y), then pixel v x With its four connected neighbors Inner pixel v y The N-connection weights are set to,

Images