Tobacco leaf flatness measuring device and method based on machine vision

Abstract

The invention discloses a tobacco leaf flatness measuring device and method based on machine vision, and relates to the technical field of cigar tobacco leaf flatness measurement. The tobacco leaf flatness measuring device comprises a main body table top, a tobacco leaf placing table is arranged on the main body table top, an industrial camera is arranged on one side of the tobacco leaf placing table, and laser range finders are arranged above the tobacco leaf placing table in a transverse and longitudinal sliding mode; a rotary lifting frame is arranged on one side of the tobacco leaf containing table in the longitudinal direction in a sliding mode, projection lamps are evenly distributed on the rotary lifting frame, and a leveling device is arranged on one side of the tobacco leaf containing table in the longitudinal direction in a sliding mode. The method solves the problems that in production in the traditional technology, the method for measuring the flatness of the tobacco leaves still adopts manual subjective judgment, errors are large, clear and unified standards do not exist and the like, and the manual judgment method mostly carries out indirect judgment through the flatness degree of the main veins and the branch veins of the tobacco leaves. And direct measurement of the flatness of the complete leaf surface of the cigar tobacco cannot be realized.

Description

Technical Field

[0001] The invention relates to the technical field of measuring the flatness of cigar tobacco leaves, and in particular to a device and method for measuring the flatness of tobacco leaves based on machine vision. Background Art

[0002] In recent years, my country's cigar market has developed rapidly, and the cigar planting area has been expanding year by year. The development of the cigar industry has gradually shifted from emphasizing output to improving quality. All parts of a cigar are rolled from whole tobacco leaves, and the outermost layer of tobacco is called the wrapper, which has the greatest impact on the appearance quality of the cigar. The quality of my country's wrapper tobacco leaves is inferior to that of foreign high-quality production areas. Domestic high-end cigars basically use imported wrappers, and are heavily dependent on foreign raw materials, which restricts the development of my country's cigar industry. The flatness of tobacco leaves is an important factor affecting their appearance quality. At present, with the improvement of my country's wrapper production level, the thickness and flatness of the veins of domestic wrappers have been improved, reaching the level of foreign high-quality production areas.

[0003] The prior art discloses a patent with a publication number of CN117587151A. In this scheme, the two homozygous genotypes GG and AA varieties at this locus show significant differences in the phenotype of leaf flatness. In the GWAS population, the GG genotype variety phenotype is characterized by a more wrinkled leaf surface (i.e., the leaf flatness is greater than or equal to 3), while the AA genotype variety phenotype is characterized by a relatively flat leaf surface (i.e., the leaf flatness is less than or equal to 2). This provides an efficient, simple and reliable reference marker for the screening of cigar varieties, effectively improving the efficiency of molecular breeding and genetic improvement of cigars.

[0004] As the existing devices and methods are used, the shortcomings of the technology are gradually exposed, mainly in the following aspects:

[0005] First, in the existing research field of cigar leaf flatness, the method for measuring the flatness of cigar leaves still relies on subjective human judgment, which has problems such as large errors and no clear and unified standards. In addition, the human judgment method mostly makes indirect judgments based on the flatness of the main and branch veins of the leaves, and cannot achieve direct measurement of the flatness of the entire leaf surface of the cigar leaf.

[0006] Second, in addition, there is currently no device on the market that can measure the flatness of cigar tobacco leaves, and a constant and unified measurement environment cannot be achieved. Therefore, the lack of relevant measurement methods and devices has hindered the development of research on the flatness of domestic cigar wrapper tobacco leaves.

[0007] In summary, the prior art obviously has inconveniences and defects in practical use, so it is necessary to improve it. Summary of the invention

[0008] In view of the defects in the prior art, the present invention solves the problem that in the traditional technology, the method for determining the flatness of tobacco leaves in production still relies on subjective human judgment, which has large errors and no clear and unified standards. In addition, the human judgment method mostly makes indirect judgments based on the flatness of the main and branch veins of the tobacco leaves, and cannot achieve direct measurement of the flatness of the complete leaf surface of cigar tobacco leaves.

[0009] To solve the above problems, the present invention provides the following technical solutions:

[0010] The tobacco leaf flatness measuring device based on machine vision comprises a main body table, a tobacco leaf placing table is arranged on the main body table, an industrial camera is arranged on one side of the tobacco leaf placing table, and a laser rangefinder is arranged above the tobacco leaf placing table for sliding in the horizontal and vertical directions.

[0011] A rotating lifting frame is provided on one side of the tobacco placement platform for sliding along the longitudinal direction, and projection lamps are evenly distributed on the rotating lifting frame.

[0012] A leveling device is provided on one side of the tobacco leaf placement platform for longitudinal sliding.

[0013] As an optimized solution, a parallel fill light is provided on the main table top and moves along an arc track, and the center of the arc track is located at the center position of the tobacco leaf placement table.

[0014] As an optimized solution, a grid structure consisting of thin metal rods crossing each other horizontally is provided in the rotating lifting frame, and the projection lamp is fixed at the intersection.

[0015] As an optimized solution, a avoidance hole for avoiding the camera head of the industrial camera is provided at the center position of the rotating lifting frame.

[0016] As an optimized solution, the leveling device includes a hydraulic lifting rod, a mechanical moving groove is installed on the top of the hydraulic lifting rod, roller-type leveling rods are installed in parallel in the horizontal direction in the mechanical moving groove, and several roller-type leveling rods move in a diffuse manner in the horizontal direction, and the roller-type leveling rods are equipped with a soft roller cylinder.

[0017] As an optimized solution, a moving track for a leveling device is provided on one side of the main table top, and the leveling device is slidably installed in the moving track for the leveling device, and the moving track for the leveling device is located in the direction of the midpoint of the tobacco placement table.

[0018] As an optimized solution, one side of the main table top is provided with a projection device moving track arranged in parallel with the leveling device moving track, a rotating lifting hydraulic rod is slidably provided in the projection device moving track, and the rotating lifting frame is fixedly connected to the top of the rotating lifting hydraulic rod.

[0019] As an optimized solution, the areas on both sides of the tobacco placement table on the main table top are provided with tracks extending in the longitudinal direction, a rangefinder moving track is provided between the two tracks, and the laser rangefinder is installed on the rangefinder moving track along the transverse sliding direction.

[0020] The present invention also discloses a method for measuring the flatness of tobacco leaves based on machine vision, which comprises the following steps:

[0021] S1: Selection of tobacco leaves to be tested and setting of device parameters; according to the experimental requirements, select several tobacco leaves of different treatments, varieties, parts, etc., remove tobacco leaves with large damage, deformation, and infectious pathogens, and number the remaining tobacco leaves one by one in order;

[0022] S2: Natural and flattened state image acquisition of cigar tobacco leaves; the operator places the cigar tobacco leaves on the measuring table and starts the experiment when the leaves are naturally stretched. Under the control of the programming system, the judgment point projection module moves to the vertical position directly above the cigar tobacco leaves, turns on the judgment point projection, and the industrial camera takes pictures of the tobacco leaves with the judgment points. Then the judgment point projection is turned off, the parallel fill light is turned on, and the industrial camera takes pictures of the tobacco leaves under different angles of illumination;

[0023] S3: Measurement and analysis of the flatness of cigar tobacco leaves. After obtaining the tobacco leaf image, the built-in computer runs the machine vision system to identify the tobacco leaf image based on its features, and uses the MiDaS depth map estimation to predict the depth of the image to obtain the ridge images of various parts of the tobacco leaf.

[0024] S4: Tobacco leaf depth image correction and improvement work; after the tobacco leaf image analysis is completed, the laser ranging system is turned on to measure the distance of the four corners, the center point and the randomly selected measurement points to correct the measurement data;

[0025] S5: Analysis of tobacco leaf flatness characteristics and model training; after correcting the depth information of the tobacco leaf image, the built-in computer runs the convolutional neural network to analyze the multiple images obtained in this experiment, extract features of the height changes on the tobacco leaf surface and each measurement point, and conduct training based on actual data.

[0026] As an optimized solution, in step S1, the device console is turned on, the number of tobacco leaves measured this time and the number setting are input into the console, the judgment point projection light is turned on, the "standard horizontal plane" calculation mode is selected, the parallel fill light is selectively turned on for depth information correction, the laser rangefinder is turned on, a number of judgment points are randomly selected for sampling and detection, and the cigar tobacco leaf flattening device is selectively turned on.

[0027] As an optimized solution, in step S2, the tobacco leaf flattening device is turned on, and the tobacco leaf flattening device moves to the top of the cigar tobacco leaves, and then descends to press the tobacco leaves onto the measuring table. The flattening rods are spread out to both sides to press and flatten the cigar tobacco leaves. The shooting steps are the same as above. After the shooting is completed, the industrial camera transmits the image to the built-in computer.

[0028] As an optimization solution, in step S3, the tobacco leaf image with the determination point is identified, and the ridge and wrinkle height and direction of the tobacco leaf at that point are calculated based on the size and shape of the determination point. Contour lines are drawn in combination with the image depth map to form a flatness image of the tobacco leaf.

[0029] As an optimized solution, in step S4, the contour image of the tobacco leaf is identified. If the highest point and the lowest point are found to be within the measurement point interval and are completely enclosed by the contour lines without measurement data, the coordinates are calculated based on the position in the image, and the laser ranging module is moved to this point to measure the height of the tobacco leaf at the pole alone. After completing the measurement of this tobacco leaf, the experimenter takes it away and places the next numbered tobacco leaf, and then continues the experiment on the console.

[0030] As an optimization solution, in step S5, after the training is completed, the measured images are repeatedly extracted. If overfitting or underfitting occurs, or errors occur in the data, the experimenter is reminded to change the training samples and parameters.

[0031] Compared with the prior art, the present invention has the following beneficial effects:

[0032] 1. Compared with the current methods for measuring the flatness of cigar tobacco leaves, the present invention adopts machine vision measurement technology, which can effectively eliminate the errors caused by human subjective factors, and the measurement structure is more accurate and objective, and the measurement speed is fast, which has the advantage of high efficiency.

[0033] 2. Compared with the current surface flatness and defect detection methods for cigar tobacco leaves, workpieces and quality inspection, this method introduces the MiDaS depth map estimation principle based on a lightweight convolutional neural network model for measurement, and has the advantages of ease of use, cross-platform and wide compatibility.

[0034] 3. Compared with the current surface flatness detection methods in the field of cigar tobacco leaves, 3D reconstruction and scanning, the present invention introduces a projection judgment point calculation method, which has the characteristics of simple implementation and low equipment requirements. At the same time, the present method introduces the laser ranging principle for correction to further reduce errors and has high accuracy.

[0035] 4. Compared with the traditional methods for judging the flatness of various types of tobacco leaves, the method of the present invention can directly measure the flatness of each part of the complete tobacco leaf, rather than indirectly judging through the main and branch veins. It has high accuracy and the results are more reliable.

[0036] 5. Compared with the traditional method for judging the flatness of tobacco leaves, the present invention introduces computer programming control, which can realize full automation of the experimental measurement process. Compared with the previous method mainly based on manual operation, the measurement is programmed and the manual operation error is small. In addition, the present method realizes a consistent measurement environment for tobacco leaves and can eliminate environmental interference.

[0037] 6. The device proposed in the present invention makes up for the lack of devices in the field of cigar tobacco flatness measurement research. In addition, the method has a deep learning function and can train a model on the characteristic data of cigar tobacco. The model can be continuously iterated as the experimental data becomes richer. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following is a brief introduction to the drawings required for the specific embodiments or the description of the prior art. In all the drawings, similar elements or parts are generally identified by similar reference numerals. In the drawings, the elements or parts are not necessarily drawn according to the actual scale.

[0039] Figure 1 It is a schematic diagram of the present invention in a top view direction;

[0040] Figure 2 It is a schematic diagram of the three-dimensional structure of the present invention;

[0041] Figure 3 It is a structural schematic diagram of the rotary lifting frame of the present invention;

[0042] Figure 4 It is a structural schematic diagram of the leveling device of the present invention;

[0043] Figure 5 Principles for analyzing the determination points of the tobacco leaf flatness detection method of the present invention Figure 1 ;

[0044] Figure 6 Principles for analyzing the determination points of the tobacco leaf flatness detection method of the present invention Figure 2 ;

[0045] Figure 7 Principles for analyzing the determination points of the tobacco leaf flatness detection method of the present invention Figure 3 ;

[0046] Figure 8 Principles for analyzing the determination points of the tobacco leaf flatness detection method of the present invention Figure 4 ;

[0047] Fig. 9 It is a schematic diagram of a comparative example of the present invention;

[0048] Fig.10 This is a schematic diagram of object extraction of tobacco leaves in the comparative example of the present invention.

[0049] In the figure: 1-main table, 2-control console, 3-tobacco leaf placement table, 4-industrial camera, 5-distance meter moving track, 6-laser distance meter, 7-arc track, 8-parallel fill light, 9-projection device moving track, 10-rotating lifting frame, 11-projection lamp, 12-leveling device moving track, 13-leveling device, 14-roller leveling rod. DETAILED DESCRIPTION

[0050] The following embodiments of the technical solution of the present invention are described in detail in conjunction with the accompanying drawings. The following embodiments are only used to more clearly illustrate the technical solution of the present invention, and are therefore only used as examples, and cannot be used to limit the protection scope of the present invention.

[0051] like Figures 1 to 10 As shown, the tobacco leaf flatness measuring device based on machine vision comprises a main body table 1, a tobacco leaf placing table 3 is arranged on the main body table 1, an industrial camera 4 is arranged on one side of the tobacco leaf placing table 3, and a laser rangefinder 6 is arranged above the tobacco leaf placing table 3 to slide in the horizontal and vertical directions.

[0052] A rotating lifting frame 10 is provided on one side of the tobacco placement platform 3 for sliding along the longitudinal direction, and projection lamps 11 are evenly distributed on the rotating lifting frame 10.

[0053] A leveling device 13 is provided on one side of the tobacco leaf placement platform 3 so as to slide longitudinally.

[0054] A parallel fill light 8 is provided on the main table 1 along an arc track 7, and the center of the arc track 7 is located at the center of the tobacco leaf placement table 3. The arc track 7 has a structure of 90°.

[0055] The structure of the parallel fill light 8 is that the top is provided with a parallel light source, a luminous lamp bead with adjustable brightness and a reflector.

[0056] A grid structure consisting of thin metal rods crossing each other at the horizontal axis is arranged inside the rotating lifting frame 10 , the grid aperture is 5 cm, and the projection lamp 11 is fixed at the intersection point.

[0057] The projection lamp 11 has a fixed angle, and the brightness of its lamp beads is different from the size of the projection area, but the brightness of the image projected in the plane of the tobacco leaf placement table 3 is the same; the projection figures of the projection modules at different positions are rectangles of the same width but different lengths, and the area is proportional to the distance from the projection point to the lens of the industrial camera 4, so as to achieve that when all projection modules are projected on the top surface of the tobacco leaf placement table 3, they are of the same size in the photos taken by the industrial camera 4, and the angle is perpendicular to the line connecting the projection point and the center of the top surface of the tobacco leaf placement table 3.

[0058] A clearance hole for evading the camera head of the industrial camera 4 is provided at the center of the rotating lifting frame 10 .

[0059] The leveling device 13 includes a hydraulic lifting rod, a mechanical moving groove is installed on the top of the hydraulic lifting rod, and roller-type leveling rods 14 are installed in parallel in the horizontal direction in the mechanical moving groove. Several roller-type leveling rods 14 move in a diffuse manner in the horizontal direction. The roller-type leveling rods 14 are equipped with soft roller cylinders. The roller-type leveling rods 14 are in five copies. A driving structure that independently drives the roller-type leveling rods 14 to move is set in the mechanical moving groove. The driving structure is common in daily life and is not an innovation of this solution, so it will not be elaborated here.

[0060] The length of the roller-type leveling rod 14 covers the entire top surface of the tobacco leaf placement platform 3, and the multiple roller-type leveling rods 14 can move in a diffuse manner to spread the tobacco leaves from the middle to both sides;

[0061] A leveling device moving track 12 is provided on one side of the main table top 1 , and a leveling device 13 is slidably installed in the leveling device moving track 12 , and the leveling device moving track 12 is located in the direction of the midpoint of the tobacco placement platform 3 .

[0062] A projection device moving track 9 arranged in parallel with the leveling device moving track 12 is provided on one side of the main table top 1. A rotating lifting hydraulic rod is slidably provided in the projection device moving track 9. A rotating lifting frame 10 is fixedly connected to the top of the rotating lifting hydraulic rod.

[0063] The main table top 1 is provided with tracks extending in the longitudinal direction in the areas on both sides of the tobacco placement table 3, and a rangefinder moving track 5 is provided between the two tracks. The laser rangefinder 6 is installed on the rangefinder moving track 5 in a transverse sliding manner.

[0064] The main table 1 is in the shape of a rectangular box table. A built-in computer is installed inside the main table 1. The built-in computer has an image processor, a programming control system, a machine vision system, and a convolutional neural network. The built-in computer controls the power supply of other devices, connects the control lines of functional components such as industrial cameras 4, mobile tracks, and motorized devices through the programming control system, and connects the industrial cameras 4 and laser rangefinders 6 through data transmission devices. A control console 2 is installed on one side of the main table 1. The structure is a touch monitor, which is an interactive channel between the experimenter and the built-in computer / this device.

[0065] The tobacco leaf placement table 3 is in the shape of a rectangular raised table top, and the top surface is made of white frosted acrylic material;

[0066] The industrial camera 4 is an inverted "L" shaped structure;

[0067] A coordinate system is provided on the top surface of the tobacco placement table 3, and the built-in computer controls the movement of the laser rangefinder 6 according to the coordinate calculation;

[0068] All the above-mentioned moving tracks and motorized devices are equipped with stepping motors and are controlled by computer programming. Among them, the rangefinder moving track 5 is a plane type and can be moved accurately according to the coordinates.

[0069] The present invention also discloses a method for measuring the flatness of tobacco leaves based on machine vision, which comprises the following steps:

[0070] S1: Selection of tobacco leaves to be tested and setting of device parameters; according to the experimental requirements, select several tobacco leaves of different treatments, varieties, parts, etc., remove tobacco leaves with large damage, deformation, and infectious pathogens, and number the remaining tobacco leaves one by one in order;

[0071] S2: Natural and flattened state image acquisition of cigar tobacco leaves; the operator places the cigar tobacco leaves on the measuring table and starts the experiment when the leaves are naturally stretched. Under the control of the programming system, the judgment point projection module moves to the vertical position directly above the cigar tobacco leaves, turns on the judgment point projection, and the industrial camera takes pictures of the tobacco leaves with the judgment points. Then the judgment point projection is turned off, the parallel fill light is turned on, and the industrial camera takes pictures of the tobacco leaves under different angles of illumination;

[0072] S3: Measurement and analysis of the flatness of cigar tobacco leaves. After obtaining the tobacco leaf image, the built-in computer runs the machine vision system to identify the tobacco leaf image based on its features, and uses the MiDaS depth map estimation to predict the depth of the image to obtain the ridge images of various parts of the tobacco leaf.

[0073] S4: Tobacco leaf depth image correction and improvement work; after the tobacco leaf image analysis is completed, the laser ranging system is turned on to measure the distance of the four corners, the center point and the randomly selected measurement points to correct the measurement data;

[0074] S5: Analysis of tobacco leaf flatness characteristics and model training; after correcting the depth information of the tobacco leaf image, the built-in computer runs the convolutional neural network to analyze the multiple images obtained in this experiment, extract features of the height changes on the tobacco leaf surface and each measurement point, and conduct training based on actual data.

[0075] In step S1, the device console is turned on, the number and number settings of the tobacco leaves to be measured are input into the console, the projection light of the judgment point is turned on, the "standard horizontal plane" calculation mode is selected, the parallel fill light is selectively turned on for depth information correction, the laser rangefinder is turned on, several judgment points are randomly selected for sampling and detection, and the cigar tobacco leaf flattening device is selectively turned on.

[0076] In step S2, the tobacco leaf flattening device is turned on, and the tobacco leaf flattening device moves to the top of the cigar tobacco leaves, and then descends to press the tobacco leaves onto the measuring table. The flattening rods are spread out to both sides to press and flatten the cigar tobacco leaves. The shooting steps are the same as above. After the shooting is completed, the industrial camera transmits the image to the built-in computer.

[0077] In step S3, the tobacco leaf image with the determination point is identified, and the ridge and wrinkle height and direction of the tobacco leaf at that point are calculated according to the size and shape of the determination point. Contour lines are drawn in combination with the image depth map to form a flatness image of the tobacco leaf.

[0078] In step S4, the contour image of the tobacco leaf is identified. If the highest point and the lowest point are found to be within the measurement point interval and are completely enclosed by the contour lines without measurement data, the coordinates are calculated based on the position in the image, and the laser ranging module is moved to this point to measure the height of the tobacco leaf at the pole alone. After completing the measurement of this tobacco leaf, the experimenter takes it away and places the next numbered tobacco leaf, and then continues the experiment on the console.

[0079] In step S5, after the training is completed, the measured images are repeatedly extracted. If overfitting or underfitting occurs, or errors occur in the data, the experimenter is reminded to change the training samples and parameters.

[0080] The detection process of the present invention is:

[0081] 1. Selection of cigar tobacco leaves to be tested and setting of device parameters

[0082] Select the middle loin with a length of not less than 50cm and a width of not less than 25cm, remove the tobacco leaves with large damage, deformation and serious infection of infectious pathogens, leave no less than 10 tobacco leaves suitable for making cigar wrappers for each treatment, number them in order, and balance them for 48 hours at 22℃ and 60% relative humidity. After the tobacco leaves are balanced, turn on the control console 2, enter the number of experimental treatment groups and the number of tobacco leaves in each group for this measurement, turn on the projection module 11, and choose to calculate the deviation as the standard plane relative to the tobacco leaf placement table 3; turn on the parallel fill light 8, and set the movement to fill light in two directions with an angle of 90°; turn on the laser rangefinder 6, select 1 at each of the four corners of the table, 1 at the midpoint and 5 random points, a total of 10 points for detection and verification. Selectively turn on the leveling device 13, which is set to collect images again after the tobacco leaves are leveled.

[0083] 2. Image collection of cigar leaves in natural and flattened states

[0084] Place the cigar tobacco leaves on the top surface of the tobacco leaf placement platform 3, and when the tobacco leaves are naturally stretched and have no large curls, start the experiment on the console 2. The rotating lifting frame 10 moves along the projection device moving track 9 to one side of the tobacco leaf placement platform 3, then rotates to the vertically above the tobacco leaf placement platform 3, and then rises to the same height as the camera of the industrial camera 4, and then each projection module 11 is turned on. The industrial camera 4 collects the image of the cigar tobacco leaves with the projection point. After the collection is completed, the projection module 11 is turned off, and the parallel fill light 8 is turned on to illuminate the cigar tobacco leaves in two directions, and the depth image of the tobacco leaves is constructed according to the shadow changes. After the shooting is completed, the parallel fill light 8 is turned off. If the cigar tobacco leaf flattening function is turned on in the previous step, after shooting the image, the flattening device 13 moves along the track to the top of the tobacco leaf placement platform 3, and then descends to the roller-type flattening rod 14 to lightly press on the cigar tobacco leaves. The roller-type flattening rod 14 spreads from the center to both sides until the cigar tobacco leaves are rolled flat, and the above shooting steps are repeated again after flattening.

[0085] 3. Cigar leaf image analysis and flatness calculation

[0086] After acquiring the image of tobacco leaves, the convolutional neural network model is used to identify the features of the image, and then the depth of the image is predicted using the MiDaS depth map estimation to obtain the changes in the concave and convex trends of various parts of the tobacco leaves; after the depth image is constructed by combining the shadow images under different directions of light, the distance is measured according to the size of the judgment point on the image. The relevant distance settings are as follows:

[0087] Computer measurement principles, such as Figures 5 to 8 As shown, the distance between the reference point and the camera is L0, the distance between two adjacent projection points is L, and the distance between the projection point n and the camera is L n , let the length of the reference point be R0, and the length of the projection point (XY) be R (X.Y)a ,

[0088] Then the length of the projection point with coordinates (XY) is: R (X.Y) =R0(√L0 2 +(X 2 +Y 2 )L) / L0

[0089] The height of this point from the tobacco placement table 3 plane

[0090] H (X.Y) =L0-√{√L0 2 +(X 2 +Y 2 )L 2 *[R0*√L0 2 +(X 2 +Y 2 )L)] / (R (X.Y)a *L0)}2 -(X 2 +Y 2 )L 2

[0091] After obtaining the elevation heights of different parts of the cigar tobacco leaf relative to the plane of the tobacco leaf placement table 3, contour lines are drawn in combination with the image depth map to produce a tobacco leaf flatness image.

[0092] 4. Tobacco leaf depth image correction and improvement work

[0093] After the analysis of the tobacco leaf image is completed, the built-in computer obtains the coordinates of the judgment points that need to be checked in advance, and the laser rangefinder 6 moves to the inspection points along the rangefinder moving track 5 to measure the distance, and transmits the obtained data to the built-in computer, which assists the computer to correct the depth image and contour lines of the tobacco leaf. The built-in computer identifies the contour lines of the image, and according to the closure of the contour lines, determines the positions of the highest and lowest points of the tobacco leaf, and obtains the corresponding coordinates. The laser rangefinder 6 moves to the measurement point to obtain the height of the extreme point of the tobacco leaf, and then improves the depth image of the tobacco leaf. After completing the measurement of this tobacco leaf, the measurement of the next labeled tobacco leaf is carried out.

[0094] 5. Analysis of tobacco leaf flatness characteristics and model training

[0095] After completing this experiment, the convolutional neural network was run to analyze the multiple depth images obtained this time, extract features of the height changes on the tobacco leaf surface and at each measurement point, and train with actual data. After training, the measured images were repeatedly extracted. If overfitting or underfitting occurred, or errors occurred in the data, the experimenter was reminded to change the training samples and parameters.

[0096] Comparative Example

[0097] like Fig. 9 As shown, in the image of tobacco leaves ( Figure 9-1 ) After edge extraction and depth map estimation, the height variation trend of different parts of the tobacco leaf was obtained ( Figure 9-2 After combining the ridge heights of different parts of the tobacco leaves obtained from each judgment point, a tobacco leaf flatness image was obtained with 0.5 cm as an interval ( Figure 9-3 ), it can be seen that when this tobacco leaf is placed on a flat surface, the main distribution height is between 0.5-1cm.

[0098] Depend on Fig.10It can be seen that after object extraction of the tobacco leaves, when the 0.5-1cm interval is selected, the histogram shows that the total number of pixels of the tobacco leaves in this 5mm interval is 294288. When the whole leaf is selected, it can be seen that the total number of pixels of the tobacco leaves is 433087. It can be obtained that the area of ​​the tobacco leaves in the 5mm interval is 67.95%, and the overall flatness is not good, which is not conducive to use as high-quality cigar wrapper tobacco leaves.

[0099] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein by equivalents. These modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present invention, and they should all be included in the scope of the claims and specification of the present invention.

Claims

1. A tobacco leaf flatness measuring device based on machine vision, characterized in that: The invention comprises a main body table (1), wherein a tobacco leaf placement table (3) is provided on the main body table (1), an industrial camera (4) is provided on one side of the tobacco leaf placement table (3), and a laser rangefinder (6) is provided above the tobacco leaf placement table (3) so as to slide in the horizontal and vertical directions. A rotating lifting frame (10) is provided on one side of the tobacco placement platform (3) for sliding along the longitudinal direction, and projection lamps (11) are evenly distributed on the rotating lifting frame (10). A leveling device (13) is provided on one side of the tobacco leaf placement platform (3) for sliding along the longitudinal direction.

2. The tobacco leaf flatness measuring device based on machine vision according to claim 1, characterized in that: A parallel fill light (8) is provided on the main table top (1) so as to move along an arc track (7), and the center of the arc track (7) is located at the center position of the tobacco leaf placement table (3).

3. The tobacco leaf flatness measuring device based on machine vision according to claim 1, characterized in that: The rotating lifting frame (10) is provided with a grid structure formed by the cross-axis intersection of thin metal rods, and the projection lamp (11) is fixed on the intersection point; A clearance hole for evading the camera head of the industrial camera (4) is provided at the center of the rotating lifting frame (10).

4. The tobacco leaf flatness measuring device based on machine vision according to claim 1, characterized in that: The leveling device (13) comprises a hydraulic lifting rod, a mechanical moving groove is installed on the top of the hydraulic lifting rod, roller-type leveling rods (14) are installed in parallel in the horizontal direction in the mechanical moving groove, and a plurality of roller-type leveling rods (14) move in a diffuse manner in the horizontal direction, and the roller-type leveling rods (14) are sleeved with a soft roller cylinder.

5. A method for measuring tobacco leaf flatness based on machine vision, comprising the following steps: S1: Selection of tobacco leaves to be tested and setting of device parameters; According to the experimental requirements, select several tobacco leaves of different treatments, varieties, parts, etc., remove tobacco leaves with large damage, deformation, and infectious pathogens, and number the remaining tobacco leaves one by one in order; S2: Natural and flattened state image acquisition of cigar tobacco leaves; the operator places the cigar tobacco leaves on the measuring table and starts the experiment when the leaves are naturally stretched. Under the control of the programming system, the judgment point projection module moves to the vertical position directly above the cigar tobacco leaves, turns on the judgment point projection, and the industrial camera takes pictures of the tobacco leaves with the judgment points. Then the judgment point projection is turned off, the parallel fill light is turned on, and the industrial camera takes pictures of the tobacco leaves under different angles of illumination; S3: Measurement and analysis of the flatness of cigar tobacco leaves. After obtaining the tobacco leaf image, the built-in computer runs the machine vision system to identify the tobacco leaf image based on its features, and uses the MiDaS depth map estimation to predict the depth of the image to obtain the ridge images of various parts of the tobacco leaf. S4: Tobacco leaf depth image correction and improvement work; after the tobacco leaf image analysis is completed, the laser ranging system is turned on to measure the distance of the four corners, the center point and the randomly selected measurement points to correct the measurement data; S5: Analysis of tobacco leaf flatness characteristics and model training; after correcting the depth information of the tobacco leaf image, the built-in computer runs the convolutional neural network to analyze the multiple images obtained in this experiment, extract features of the height changes on the tobacco leaf surface and each measurement point, and conduct training based on actual data.

6. The method for measuring tobacco leaf flatness based on machine vision according to claim 5, characterized in that: In step S1, the device console is turned on, the number and number settings of the tobacco leaves to be measured are input into the console, the judgment point projection light is turned on, the "standard horizontal plane" calculation mode is selected, the parallel fill light is selectively turned on for depth information correction, the laser rangefinder is turned on, a number of judgment points are randomly selected for sampling and detection, and the cigar tobacco leaf flattening device is selectively turned on.

7. The method for measuring tobacco leaf flatness based on machine vision according to claim 5, characterized in that: In step S2, the tobacco leaf flattening device is turned on, and the tobacco leaf flattening device moves to the top of the cigar tobacco leaf, and then descends to press the tobacco leaf onto the measuring table. The flattening rods are spread out to both sides to press and flatten the cigar tobacco leaf. The shooting steps are the same as above. After the shooting is completed, the industrial camera transmits the image to the built-in computer.

8. The method for measuring tobacco leaf flatness based on machine vision according to claim 5, characterized in that: In step S3, the tobacco leaf image with the determination point is identified, and the ridge and wrinkle height and direction of the tobacco leaf at the determination point are calculated according to the size and shape of the determination point. Contour lines are drawn in combination with the image depth map to form a flatness image of the tobacco leaf.

9. The method for measuring tobacco leaf flatness based on machine vision according to claim 5, characterized in that: In step S4, the contour image of the tobacco leaf is identified. If the highest point and the lowest point are found to be within the measurement point interval and are completely enclosed by the contour lines without measurement data, the coordinates are calculated based on the position in the image, and the laser ranging module is moved to this point to measure the height of the tobacco leaf at the pole alone. After completing the measurement of this tobacco leaf, the experimenter takes it away and places the next numbered tobacco leaf, and then starts the experiment on the console.

10. The method for measuring tobacco leaf flatness based on machine vision according to claim 5, characterized in that: In step S5, after the training is completed, the measured images are repeatedly extracted. If overfitting or underfitting occurs, or errors occur in the data, the experimenter is reminded to change the training samples and parameters.

Images