Cigar core / sleeve classification method and device based on image processing technology

Abstract

The invention belongs to the technical field of cigar quality evaluation, and particularly relates to a cigar core / cigar cover classification method and device based on an image processing technology. The method comprises the following steps: firstly, identifying defects of tobacco leaves in an eggplant core / eggplant sleeve image, and extracting a minimum bounding rectangle and a central point of the tobacco leaves from the eggplant core / eggplant sleeve image; then, a rectangle group is constructed, all rectangles in the rectangle group take the center point of the minimum bounding rectangle of the tobacco leaves as the center point of the rectangle, and the lengths, the widths or the rotation angles of all the rectangles are different; screening out a rectangle of which all defects are not in the rectangle and the area is the largest from the rectangle group, wherein the rectangle is the largest effective use area of the tobacco leaves; and finally, the eggplant cores / eggplant sleeves are classified according to the length and the width of the maximum effective use area of the tobacco leaves. Compared with a manual sorting mode, the sorting speed can be greatly increased, the labor cost is saved, and errors possibly existing in the current manual sorting process are avoided.

Description

Technical Field

[0001] The invention belongs to the technical field of cigar quality evaluation, and in particular relates to a cigar core / binder classification method and device based on image processing technology. Background Art

[0002] In recent years, the development trend of domestic cigars has gradually increased, and the public's demand for cigars has increased day by day, which has led to higher requirements for the appearance quality, taste and flavor of cigars. Although the research work on cigars in China started late, driven by the market, my country's traditional cigar production technology is facing new challenges.

[0003] The components of traditional cigars include wrapper, binder and core. The wrapper is a layer of tobacco leaves wrapped around the outside of the cigar, which is a direct factor affecting the appearance of the cigar; the core is the filling tobacco leaves inside the cigar, which is an important factor directly affecting the taste of the cigar; the binder is used to wrap the core and shape it. A good cigar needs to be not only ornamental, but more importantly, its taste and flavor. Cigars with better quality and mellower taste often have a larger market, which will not only bring huge economic benefits, but also give consumers a better experience.

[0004] The cigar core is a key factor affecting the taste and flavor of cigars, and the cigar wrapper also has a great impact on the taste of cigars. Due to the unevenness of tobacco raw materials, there may be damage on the obtained tobacco leaves, and the presence of these damages will have different degrees of impact on the taste and flavor of cigars. Therefore, in order to improve the quality of cigars and reduce the waste of tobacco leaves, it is necessary to select the tobacco leaves during the production of cigars, and classify the tobacco leaves for the production of cigars of different standards. At present, the main method used in the selection of cigar core / cigar wrapper tobacco leaves in China is manual selection, which relies on artificial senses to identify and classify tobacco leaves. The disadvantage of this method is that it relies too much on artificial senses, and some defects on tobacco leaves may be difficult to identify with the naked eye, which will affect the quality of tobacco leaf selection and ultimately affect the quality of finished cigars. In addition, the speed and accuracy of manual selection are low, and the labor cost is high. Summary of the invention

[0005] The purpose of the present invention is to provide a cigar filler / binder classification method and device based on image processing technology to solve the problems of low efficiency and low precision caused by the manual whole selection method in the prior art.

[0006] In order to solve the above technical problems, the present invention provides a cigar filler / binder classification method based on image processing technology, comprising the following steps:

[0007] 1) Identify the defects of tobacco leaves in the filler / binder image, and extract the minimum circumscribed rectangle of the tobacco leaves and its center point on the filler / binder image;

[0008] 2) constructing a rectangular group, wherein each rectangle in the rectangular group takes the center point of the smallest circumscribed rectangle of the tobacco leaf as the center point of the rectangle, and each rectangle has a different length, width or rotation angle; selecting from the rectangular group a rectangle in which all defects are not inside the rectangle and which has the largest area, and the rectangle is the maximum effective use area of ​​the tobacco leaf;

[0009] 3) Classify the filler / binder according to the length and width of the maximum effective use area of ​​the tobacco leaf.

[0010] Furthermore, the following method is used to determine whether the defect is inside the rectangle:

[0011] calculate Where P represents the coordinates corresponding to the defect, P 1 , P 2 , P 3 , P 4 Represent the coordinates of the four vertices of the rectangle, and P 1 and P 4 are the coordinates of the two vertices on the diagonal corners of the rectangle; calculate and The angle A and and Angle B;

[0012] If one of the angles A and B is an obtuse angle, the defect is not located inside the rectangle.

[0013] If both angle A and angle B are less than or equal to 90°, further calculation and The angle C between them; if A≤C and Then the defect is judged to be inside the rectangle, if A≤C and The defect is not inside the rectangle if A ≥ C and Then the defect is judged to be inside the rectangle, A ≥ C and Then the defect is not located inside the rectangle; where P 1 P 2 Indicates P 1 and P 2 The distance between 1 P stands for P 1 The distance between P and 1 P 3 Indicates P 1 and P 3 The distance between.

[0014] Furthermore, the following method is used to determine whether the defect is inside the rectangle: if and Where P represents the coordinates corresponding to the defect, P 1 , P 2 , P 3 , P 4 Represent the coordinates of the four vertices of the rectangle, and P 1 and P 3 If the coordinates of the two vertices on the diagonal corners of the rectangle are the coordinates of the two vertices on the diagonal corners of the rectangle, the defect is judged to be inside the rectangle, otherwise the defect is judged to be not inside the rectangle.

[0015] Furthermore, the process of step 2) screening and obtaining the maximum effective utilization area of ​​the tobacco leaves is: first calculate the area of ​​each rectangle in the rectangular group, and sort the rectangles according to the size of the area; then judge whether all defects are contained in each rectangle in order from large to small in area, and the first rectangle that does not contain all defects is the maximum effective utilization area of ​​the tobacco leaves.

[0016] Furthermore, deep learning methods are used to identify tobacco leaf defects in filler / binder images.

[0017] Furthermore, in the process of extracting the minimum circumscribed rectangle of the tobacco leaf and its center point on the cigar core / cigarette wrapper image, the cigar core / cigarette wrapper image is firstly subjected to a binarization process.

[0018] In order to solve the above technical problems, the present invention also provides a cigar core / binder classification device based on image processing technology, including a memory and a processor, wherein the processor is used to execute computer program instructions stored in the memory to implement the above-mentioned cigar core / binder classification method based on image processing technology.

[0019] The beneficial effects of the present invention are as follows: the present invention is a pioneering invention, and the present invention applies image processing technology to the extraction of the maximum effective use area of ​​cigar binder / cigar core tobacco leaves. The specific means are to first identify the defects on the tobacco leaves in the binder / cigar core image, and then extract the minimum circumscribed rectangle of the tobacco leaves and determine the coordinates of the center point of the minimum circumscribed rectangle, and then use the center point as the center point of the rectangle to construct a rectangular group containing multiple rectangles, and then find the rectangle that does not contain all defects and has the largest area according to the positional relationship between the rectangle and the defect in the rectangular group. The rectangle avoids the area of ​​the damaged position, which is the maximum effective use area of ​​the tobacco leaves. Then, the maximum effective use area of ​​the tobacco leaves can be used to automatically grade the cigar binder / cigar core. Compared with the manual selection method, the present invention can greatly improve the speed of selection, save labor costs, and avoid the errors that may exist in the current manual selection process, effectively improving the quality of tobacco leaf selection. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 is a flow chart of a cigar core classification method based on image processing technology according to Example 1 of the method of the present invention;

[0021] Figure 2 This is an original image of the filler tobacco leaf of Example 1 of the method of the present invention;

[0022] Figure 3 is a defect detection diagram of filler tobacco leaves in Example 1 of the method of the present invention;

[0023] Figure 4 is an extraction diagram of the effective area of ​​the filler tobacco leaf in Example 1 of the method of the present invention;

[0024] Figure 5 is a flow chart of a cigar wrapper classification method based on image processing technology according to a second embodiment of the method of the present invention;

[0025] Figure 6 This is an original image of the cigar wrapper tobacco leaves in Example 2 of the method of the present invention;

[0026] Figure 7 is a diagram of tobacco leaf defect detection of a cigar wrapper in Example 2 of the method of the present invention;

[0027] Figure 8 This is a diagram of extracting the effective area of ​​cigar wrapper tobacco leaves in Example 2 of the method of the present invention. DETAILED DESCRIPTION

[0028] The main idea of ​​the present invention is to apply image processing technology to the extraction of the maximum effective use area of ​​cigar binder / cigar core tobacco leaves, specifically, first identify the defects on the tobacco leaves in the binder / cigar core image, then determine the center point of the minimum circumscribed rectangle of the tobacco leaves, then use the center point as the center point of the rectangle to construct a rectangular group containing multiple rectangles, and then find the rectangle that does not contain all defects and has the largest area according to the positional relationship between the rectangles and the defects in the rectangular group. This rectangle is the maximum effective use area of ​​the tobacco leaves. Then, the maximum effective use area of ​​the tobacco leaves is used to grade the cigar binders / cigar cores. The present invention uses image processing technology instead of manual labor to improve work efficiency, save costs, and improve recognition accuracy. In order to make the purpose, technical solutions and advantages of the present invention clearer, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments.

[0029] In order to make the purpose, technical solutions and advantages of the present invention more clear, the present invention is further described in detail below with reference to the accompanying drawings and embodiments.

[0030] Method Example 1:

[0031] The object to be classified in this embodiment is the cigar core. The specific process is as follows: Figure 1 As shown:

[0032] S1: Identify defects on tobacco leaves in the filler image and record the location information.

[0033] Get Figure 2 The cigar core image shown in the figure uses a deep learning method to identify damage defects on the collected cigar core image and extract the feature points of the damaged position of the tobacco leaves on the cigar core image. The extraction results are shown in Figure 3 As shown, the coordinate information of the feature point is recorded, and the coordinate information of all the feature points at the damaged positions on the tobacco leaves is used to generate a defect point set. The model used when using the deep learning method can be any deep learning model in the prior art, such as a convolutional neural network model (CNN), a recursive neural network model (RNN), etc. The model needs to be trained first, and defect identification can be performed after the training is completed.

[0034] S2: Identify the outline of the tobacco leaf in the cigar filler image and mark the minimum circumscribed rectangle of the tobacco leaf.

[0035] Convert the citrus core image into a binary image, rotate the boundary of the object by a certain angle increment each time, and record the maximum and minimum x and y values ​​of the boundary points of the circumscribed rectangle in the direction of its coordinate system each time. After rotating to a certain angle, the area of ​​the circumscribed rectangle reaches the minimum. Take the parameters of the circumscribed rectangle with the smallest area as the length and width in the sense of the main axis, record the length of the circumscribed rectangle as l, and the width as w, then the area calculation formula of the circumscribed rectangle is:

[0036] S=l×w

[0037]

[0038] In the formula, (x 11 ,y 11 ) and (x 12 ,y 12 ) is a vertex on one side of the rectangle, (x 13 ,y 13 ) and (x 14 ,y 14 ) is a vertex on the other side of the rectangle, and (x 11 ,y 11 ) and (x 13 ,y 13 ) are the coordinates of the two vertices on the opposite corners of the minimum enclosing rectangle.

[0039] The coordinates of the center point (x, y) of the minimum enclosing rectangle can be calculated using the coordinates of the four vertices of the minimum enclosing rectangle. The calculation formula is:

[0040]

[0041] In the formula, (x 11 ,y11 ) and (x 13 ,y 13 ) are the coordinates of the two vertices on the opposite corners of the minimum enclosing rectangle.

[0042] S3: Extracting rectangular groups on the citrus core image. Specifically:

[0043] The center point of the smallest circumscribed rectangle obtained in step S2 is used as the center point of the rectangular group, and different rotation angles and lengths and widths of the rectangular frame are set, thereby obtaining n rectangular frames of different angles and sizes, obtaining the coordinate information of the four vertices of the rectangular frame, and grouping the n rectangular frames into a rectangular group. The area of ​​each rectangle is calculated using the vertex coordinate information of the rectangular group, and then the rectangular frames are sorted according to the area size.

[0044] S4: Determine whether the defect point set exists inside each rectangle in the rectangular group (including the case where the defect point is located on the edge of the rectangle) to determine the maximum effective use area of ​​the tobacco leaf. Specifically:

[0045] S41: The defect point is recorded as point P, and the four vertices of the rectangle are arranged in the order of y first and then x, recorded as P 1 , P 2 , P 3 , P 4 , the coordinates are (x 1 ,y 1 )、(x 2 ,y 2 )、(x 3 ,y 3 )、(x 4 ,y 4 ), calculated according to the given coordinates (x, y) of point P The size of is calculated as:

[0046]

[0047] It should be noted that the defect itself is not a point here. The minimum enclosing rectangle of the defect is extracted, and the center point of the minimum enclosing rectangle of the defect is taken as the defect point.

[0048] S42: Calculate separately and and The angles are recorded as A and B respectively, and the calculation formula is:

[0049]

[0050] S43: Determine the angles A and B:

[0051] If one of the angles A and B is an obtuse angle, then the point P is outside the rectangle, that is, it is not inside the rectangle;

[0052] If the angles A and B are both less than or equal to 90°, further calculation and The angle C between them is calculated as follows:

[0053]

[0054] If A≤C and Then the defect is judged to be inside the rectangle, if A≤C and Then the defect is not located inside the rectangle;

[0055] If A≥C and Then the defect is judged to be inside the rectangle, A ≥ C and Then the defect is not located inside the rectangle.

[0056] Among them, P 1 P 2 Indicates P 1 and P 2 The distance between 1 P stands for P 1 The distance between P and 1 P 3 Indicates P 1 and P 3 The distance between 1 and P 4 are the coordinates of the two vertices on opposite corners of the rectangle.

[0057] S44: When the cycle reaches a rectangle in the rectangular group where there is no point in the defect point set, the rectangle is the maximum effective use area of ​​the tobacco leaf. The extraction result is as follows: Figure 4 As shown, the area is the maximum effective utilization area of ​​tobacco leaves.

[0058] S5: Classify tobacco leaves according to their maximum effective use area.

[0059] Record the coordinate information of the four vertices of the rectangle found in step S4, calculate the area of ​​the rectangle, and calculate the side length of the rectangle based on the coordinate information of the four vertices of the rectangle, where the longer side is recorded as length l and the shorter side is recorded as width w. Identify and classify the tobacco leaves according to the classification standard of filler tobacco leaves:

[0060] When the length of the maximum effective use area is greater than or equal to 9.0 cm, the width is greater than or equal to 5.0 cm, and the leaf length (referring to the length of the entire leaf) is greater than or equal to 18.0 cm, it belongs to Class 1 filler;

[0061] When the leaf length of the maximum effective use area is greater than or equal to 10.0 cm and less than or equal to 18.0 cm, it belongs to Class 2 filler.

[0062] If the above conditions are not met, the eggplant core residue will be recycled.

[0063] In summary, the cigar core classification method provided by the present invention fully combines deep learning technology to locate the damaged positions on cigar tobacco leaves and obtain the location information of the defects. When selecting tobacco leaves, the damaged positions can be completely avoided and the largest effective use area of ​​the tobacco leaves can be identified. Moreover, the present invention avoids the errors that may exist in the current manual selection process and greatly improves the selection speed. In addition, it also saves a lot of labor costs for manual selection.

[0064] Method Example 2:

[0065] The object to be classified in this embodiment is the cigar wrapper. The specific process is as follows: Figure 5 As shown:

[0066] S1: Use deep learning to identify tobacco leaf defects in binder images. Specifically:

[0067] Get Figure 6 The binder image shown in the figure uses deep learning to identify the damaged defects on the tobacco leaves. After accurately identifying the defects on the tobacco leaves, the coordinate information of the defect feature points is recorded. The coordinates of all defect feature points constitute a defect point set, such as Figure 7 As shown. The model used when using the deep learning method can be any deep learning model in the prior art, such as a convolutional neural network model (CNN), a recurrent neural network model (RNN), etc. The model needs to be trained first, and defect identification can be performed after the training is completed.

[0068] S2: Extract the minimum circumscribed rectangle and center point coordinates of the tobacco leaf on the binder image. Specifically:

[0069] The image of the tobacco wrapper is processed using image processing technology. First, the outer contour of the tobacco leaf in the image is detected and all contour points are stored. The pixel position difference between two adjacent contour points does not exceed 1. The positions of all contour points form a point set, and the outermost rectangular boundary of the point set is calculated to obtain the vertex coordinates (x 1 ,y 1 ), (x 2 ,y 2 ), (x 3 ,y 3 ), (x 4 ,y 4 ), calculate the area of ​​n rectangles, the calculation formula is as follows:

[0070]

[0071] In the formula, (x 1 ,y 1 ) and (x 2 ,y 2 ) are vertices on the same side of the rectangle, (x 3 ,y 3 ) and (x 4 ,y 4 ) is a vertex on the other side of the rectangle.

[0072] By comparing the areas of n rectangles, the smallest enclosing rectangle is obtained, which is the minimum enclosing rectangle of the tobacco leaf. The center point (x, y) of the minimum enclosing rectangle is obtained by calculation. The specific calculation formula is as follows:

[0073]

[0074] In the formula, (x 1 ,y 1 ) and (x 3 ,y 3 ) are the coordinates of the two vertices on opposite corners of the rectangle.

[0075] S3: Obtain a rectangular group on the binder image. Specifically:

[0076] Using the center point obtained in step S2 as the center point of the rectangle group, expand the rectangle from the center point to different angles with different step lengths to obtain m rectangles with different rotation angles, lengths and widths. These m rectangles form a rectangle group. Using the vertex coordinates of the m rectangles in the rectangle group, calculate the area of ​​the rectangle to obtain S 1 , S 2 , S 3 , ……S m The area of ​​m rectangles.

[0077] S4: Determine whether the defect point set is inside each rectangle in the rectangular group to determine the maximum effective use area of ​​the tobacco leaf.

[0078] Specifically:

[0079] Arrange the m rectangles in the rectangle group from large to small in area, and then determine whether the point in the defect point set exists inside the rectangle. The point in the defect point set is recorded as point P, and the four vertices of the rectangle are recorded as P 1 , P 2 , P 3 , P 4 To determine whether point P is inside the rectangle, we can transform it into determining whether point P is between the upper and lower sides and the left and right sides of the rectangle. Therefore, we can use the directionality of the vector cross product to determine whether the angle exceeds 180°. The specific calculation formula is as follows:

[0080]

[0081] Where P 1 , P 2 , P 3 , P 4 are the four vertices of the rectangle, and P 1 and P 3 are the coordinates of the two vertices on opposite corners of the rectangle.

[0082] If point P satisfies the above formula, it means that point P is inside the rectangle, otherwise point P is not inside the rectangle.

[0083] Verify the points in the defect point set against the rectangles in the rectangle group one by one from large to small. When the first rectangle does not contain any points in the defect point set, the rectangle is the maximum effective use area of ​​the tobacco leaf. The maximum effective use area extraction result is as follows: Figure 8 shown.

[0084] S5: Classify tobacco leaves according to their maximum effective use area.

[0085] According to the vertex coordinates P of the tobacco leaf effective use area obtained in step S4 1 , P 2 , P 3 , P 4 , calculate P 1 P 2 , P 2 P 3 The longer distance is taken as the length l, and the shorter distance is taken as the width w. According to the classification standard of binder tobacco leaves, the l and w of tobacco leaves are classified, and the type of tobacco leaves is finally obtained:

[0086] When the maximum effective use area satisfies l>32, w≥5, it belongs to Class 1 binder;

[0087] When the maximum effective use area satisfies 23≤l≤32, w≥5, it belongs to Class 2 binder;

[0088] When the maximum effective use area satisfies 20≤l<23, w≥6.5, it belongs to Class 3 binder;

[0089] If the above classification requirements are not met, they will be downgraded to cigar filler.

[0090] The units of the boundary parameters between the various categories (including 32, 5, etc.) are all centimeters.

[0091] In summary, the binder classification method of the present invention can be used in the tobacco leaf selection process to replace manual labor in the classification and selection of tobacco leaves, which can save a lot of labor costs and improve work efficiency; moreover, the method is combined with a deep learning method to identify defects on the leaves, thereby avoiding the impact of defects on the quality of tobacco leaves; on the whole, the present invention avoids the selection errors caused by the differences in artificial sensory recognition, and effectively improves the quality of tobacco leaves.

[0092] Device Example:

[0093] An embodiment of a cigar core / binder classification device based on image processing technology of the present invention includes a memory, a processor and an internal bus, and the processor and the memory communicate and exchange data with each other through the internal bus. The memory includes at least one software function module (i.e., computer program instructions) stored in the memory. The processor executes various functional applications and data processing by running the software program and modules stored in the memory, thereby realizing a cigar core / binder classification method based on image processing technology introduced in method embodiment 1 or method embodiment 2 of the present invention. Among them, the processor can be a processing device such as a microprocessor MCU, a programmable logic device FPGA, etc.; the memory can be various memories that use electrical energy to store information, such as RAM, ROM, etc., or it can be a memory of other forms.

[0094] Specific implementation methods are given above, but the present invention is not limited to the described implementation methods. The basic idea of ​​the present invention lies in the above basic scheme. For ordinary technicians in this field, it does not take creative work to design various deformed models, formulas, and parameters according to the teachings of the present invention. Changes, modifications, substitutions, and variations of the implementation methods without departing from the principles and spirit of the present invention still fall within the scope of protection of the present invention.

Claims

1. A cigar filler / binder classification method based on image processing technology, characterized in that: The steps include: 1) Identify the defects of tobacco leaves in the filler / binder image, and extract the minimum circumscribed rectangle of the tobacco leaves and its center point on the filler / binder image; 2) constructing a rectangular group, wherein each rectangle in the rectangular group takes the center point of the smallest circumscribed rectangle of the tobacco leaf as the center point of the rectangle, and each rectangle has a different length, width or rotation angle; selecting from the rectangular group a rectangle in which all defects are not inside the rectangle and which has the largest area, and the rectangle is the maximum effective use area of ​​the tobacco leaf; 3) Classify the filler / binder according to the length and width of the maximum effective use area of ​​the tobacco leaf.

2. The cigar filler / binder classification method based on image processing technology according to claim 1, characterized in that: The following method is used to determine whether the defect is inside the rectangle: calculate Where P represents the coordinates corresponding to the defect, P1, P2, P3, and P4 represent the coordinates of the four vertices of the rectangle, and P1 and P4 are the coordinates of the two vertices on the diagonal corners of the rectangle; calculate and The angle A and and Angle B; If one of the angles A and B is an obtuse angle, the defect is not located inside the rectangle. If both angle A and angle B are less than or equal to 90°, further calculation and The angle C between them; if A≤C and Then the defect is judged to be inside the rectangle, if A≤C and The defect is not inside the rectangle if A ≥ C and Then the defect is judged to be inside the rectangle, A ≥ C and Then the defect is not located inside the rectangle; where P1P2 represents the distance between P1 and P2, P1P represents the distance between P1 and P, and P1P3 represents the distance between P1 and P3.

3. The cigar filler / binder classification method based on image processing technology according to claim 1, characterized in that: The following method is used to determine whether the defect is inside the rectangle: If it satisfies and Where P represents the coordinates corresponding to the defect, P1, P2, P3, and P4 represent the coordinates of the four vertices of the rectangle respectively, and P1 and P3 are the coordinates of the two vertices on the diagonal corners of the rectangle. If so, the defect is judged to be inside the rectangle, otherwise, the defect is judged not to be inside the rectangle.

4. The cigar filler / binder classification method based on image processing technology according to claim 1, characterized in that: Step 2) The process of screening and obtaining the maximum effective use area of ​​tobacco leaves is as follows: first calculate the area of ​​each rectangle in the rectangular group, and sort the rectangles according to the size of the area; then judge whether all defects are contained in each rectangle in descending order of area, and the first rectangle that does not contain all defects is the maximum effective use area of ​​tobacco leaves.

5. The cigar filler / binder classification method based on image processing technology according to any one of claims 1 to 4, characterized in that: Identify tobacco leaf defects in filler / binder images using deep learning methods.

6. The cigar filler / binder classification method based on image processing technology according to any one of claims 1 to 4, characterized in that: In the process of extracting the minimum circumscribed rectangle of the tobacco leaf and its center point on the cigar core / cigarette wrapper image, the cigar core / cigarette wrapper image is firstly subjected to binarization processing.

7. A cigar filler / binder classification device based on image processing technology, comprising a memory and a processor, characterized in that: The processor is used to execute computer program instructions stored in the memory to implement the cigar filler / binder classification method based on image processing technology as described in any one of claims 1 to 6.

Images