A deep learning training method for flue-cured tobacco leaf grading

Abstract

The present invention provides a deep learning training method for flue-cured tobacco leaf grading. The method captures tobacco leaf images and performs de-impurity processing; obtains tobacco leaf position information, including a length variation graph, a width variation graph, an area graph, and a thickness variation graph; obtains tobacco leaf oil content information, including an oil distribution graph; obtains tobacco leaf integrity information, including a breakage coefficient graph; and uses the tobacco leaf position information, oil content information, and integrity information as training data to train a convolutional neural network. This invention utilizes a special imaging method combined with a unique feature operator to more accurately capture tobacco leaf position information, oil content information, and integrity information than existing image processing-based methods. By accurately capturing tobacco leaf position information, oil content information, integrity information, and impurity-free information, feature learning of the neural network is achieved using less tobacco leaf data, effectively reducing the development cycle of the deep learning algorithm and the cost of tobacco leaf data.

Description

Technical Field

[0001] The present invention relates to the field of intelligent tobacco leaf grading, and in particular to a deep learning training method for flue-cured tobacco leaf grading. Background Art

[0002] Tobacco leaf grading involves grouping tobacco leaves from the same production area and variety by growing part and color, and then grading them within each group based on quality factors such as maturity, leaf structure, and oil content. With the advancement of technology, intelligent grading of tobacco leaves is now increasingly being performed through technologies such as image recognition, saving labor and improving efficiency.

[0003] Currently, tobacco leaf grading using image processing relies on general RGB features or features derived from a combination of RGB channels, which cannot effectively represent leaf position, oil content, and integrity. Tobacco leaf grading using deep learning uses convolutional features learned autonomously by computers using large amounts of tobacco leaf data. Although convolutional features are robust to factors such as lighting, angle, posture, and scale, obtaining good convolutional features often requires hundreds of thousands or even millions of tobacco leaf data points. Since tobacco leaves are subject to state trade restrictions, obtaining them is extremely difficult. Consequently, data acquisition costs for neural network models used for tobacco leaf grading are extremely high. Furthermore, the sheer volume of data leads to long feature learning cycles and learning difficulties. Summary of the Invention

[0004] In order to solve the problems existing in the background technology, the present invention proposes a deep learning training method for flue-cured tobacco leaf grading.

[0005] A deep learning training method for flue-cured tobacco leaf grading comprises the following steps: obtaining tobacco leaf images and performing a de-cluttering process; obtaining tobacco leaf position information, including a length variation graph, a width variation graph, an area graph, and a thickness variation graph; obtaining tobacco leaf oil content information, including an oil distribution graph; obtaining tobacco leaf integrity information, including a tobacco leaf breakage coefficient graph; and using the tobacco leaf position information, oil content information, and integrity information as training data to train a convolutional neural network.

[0006] Based on the above, after obtaining the tobacco leaf image, the background image outside the tobacco leaf in the image is removed to remove impurity information in the tobacco leaf image.

[0007] Based on the above, the area map of the tobacco leaf is calculated according to the r channel binary map and b channel binary map of the tobacco leaf image.

[0008]

[0009] Among them, img 烟叶 Represents the tobacco leaf area map, img r_原图Represents the r channel binary image of the tobacco leaf image, img b_原图 Represents the b-channel binary image of the tobacco leaf image.

[0010] Based on the above, the center point of the tobacco leaf in the image is used as the center of the Gaussian distribution, and the length and width values ​​of the tobacco leaf are used as the Gaussian radius. An independent number is converted into a two-dimensional image to calculate the length change graph and width change graph of the tobacco leaf. The calculation formula is as follows

[0011]

[0012] Among them, σ represents the Gaussian kernel, the value here is the length value or width value, g is the Gaussian formula, x and y are the center points of the two-dimensional Gaussian distribution, and here are the coordinates of the center point of the tobacco leaf in the image.

[0013] Based on the above, the RGB channel segmentation processing is performed on the tobacco leaf image, and the binary image, median image, mode image, maximum image and minimum image of each channel image segmented are calculated as follows

[0014]

[0015]

[0016]

[0017]

[0018]

[0019] Among them, img represents the original tobacco leaf image, img r Represents the binary image of the r channel, img g Represents the binary image of the g channel, img b Represents the binary image of the b channel; img r_mid Represents the median image of the r channel, img r_mode Represents the mode map of the r channel, img r_min Represents the minimum value map of the r channel, img r_max The maximum value map of the r channel is represented; the median map, mode map, minimum map and maximum value map of the g channel and the b channel are calculated in the same way; the 15 acquired two-dimensional images are synthesized into one map to obtain a 15-dimensional image feature map, which is used as the thickness change map of the tobacco leaves.

[0020] Based on the above, an image acquisition device is deployed above and below the transparent image acquisition area of ​​the tobacco leaf, and a plane light source is deployed above the tobacco leaf to obtain the illuminated image of the tobacco leaf. The illuminated image of the tobacco leaf is segmented into RGB channels, and each segmented channel image is calculated to obtain the oil distribution map of the tobacco leaf, as shown below:

[0021] img s1 =img r ÷img b

[0022] img s2 =img b ÷img g

[0023] img s3 =img g ÷img r

[0024] Among them, img s1 Represents the red and blue feature map, img s2 Represents the blue-green feature map, img s3 Represents the green-red feature map, img r Represents the binary image of the r channel, img g Represents the binary image of the g channel, img b Represents a binary image of the b channel.

[0025] Based on the above, the damage coefficient of the tobacco leaf is calculated by calculating the area of ​​the holes inside the tobacco leaf and the area of ​​the entire tobacco leaf in the tobacco leaf image, and the damage coefficient is converted into a damage coefficient map of the tobacco leaf using Gaussian distribution.

[0026] Based on the above, the length change graph, width change graph, area graph, thickness change graph, oil distribution graph and breakage coefficient graph of tobacco leaves are sent to the convolutional neural network to train convolution features, as follows

[0027]

[0028] Among them, img all_result Indicates combining multiple graphs into one graph.

[0029] Compared with the existing technology, the present invention has outstanding substantial features and significant progress. Specifically, the present invention uses a special shooting method combined with a unique feature operator to more accurately obtain the position information, oil content information and integrity information of tobacco leaves than the existing image processing-based methods; through accurate tobacco leaf position information, oil content information, integrity information and impurity-free information, less tobacco leaf data is used to realize feature learning of neural networks, effectively reducing the development cycle of deep learning algorithms and the cost of tobacco leaf data. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 It is a schematic flow diagram of the present invention.

[0031] Figure 2 This is the tobacco leaf image originally obtained by the present invention and the tobacco leaf image after de-impurity processing.

[0032] Figure 3 This is the original lighting image of the tobacco leaf obtained by the present invention and the lighting image after de-impurity processing.

[0033] Figure 4 It is an area map of the tobacco leaf image of the present invention.

[0034] Figure 5 It is a length variation diagram of the tobacco leaf image of the present invention.

[0035] Figure 6 It is a width variation diagram of the tobacco leaf image of the present invention.

[0036] Figure 7-11 It is a thickness variation diagram of the tobacco leaf image of the present invention.

[0037] Figure 12 It is the oil distribution diagram of the tobacco leaf polishing image of the present invention.

[0038] Figure 13 is a damage coefficient diagram of the tobacco leaf image of the present invention.

[0039] Figure 14 It is a structural schematic diagram of the lighting and photographing device of the present invention.

[0040] Explanation of the accompanying drawings: 1. Camera; 2. Light-emitting panel; 3. Tobacco leaves; 4. Acrylic plate. DETAILED DESCRIPTION

[0041] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without creative work are within the scope of protection of the present invention.

[0042] like Figure 1 As shown, a deep learning training method for flue-cured tobacco leaf grading obtains tobacco leaf images and performs de-impurity processing; obtains tobacco leaf position information, including a length change graph, a width change graph, an area graph, and a thickness change graph; obtains tobacco leaf oil content information, including a tobacco leaf oil distribution graph; obtains tobacco leaf integrity information, including a tobacco leaf breakage coefficient graph; and uses the tobacco leaf position information, oil content information, and integrity information as training data to train a convolutional neural network.

[0043] Specifically, a conventional shooting method is used to obtain tobacco leaf images. An image acquisition device such as a camera 1 is respectively placed above and below the transparent image acquisition area of ​​the tobacco leaf 3, such as a transparent acrylic plate 4, and a planar light source such as a planar LED light panel 2 is placed above the tobacco leaf to obtain an image of the tobacco leaf illuminated by a high-brightness light source, such as Figure 14 As shown in the figure, the brightness of the LED light panel is 300cd / ㎡. The tobacco leaf image and the lighting image are respectively obtained, and the background image outside the tobacco leaf is removed through the segmentation algorithm, and the tobacco leaf background is processed into a pure color to remove the impurity information in the tobacco leaf image, as shown in the figure. Figure 2 and Figure 3 As shown, Figure 2 The upper middle image is the tobacco leaf image obtained by conventional shooting method, and the lower middle image is the image after de-impurity processing. Figure 3 The upper middle image is the original lighting image, and the lower middle image is the lighting image after the original lighting image is processed.

[0044] Image processing is used to obtain tobacco leaf location information, oil content, and integrity. Tobacco leaves can be divided into three parts based on their growing location: upper, middle, and lower. Upper leaves are generally thick, large, and wide. Middle leaves are generally moderately thick, large, and wide. Lower leaves are generally thin, small, and round. High-quality tobacco leaves have moderate oil content and high integrity.

[0045] Calculate the area map of the tobacco leaf based on the r channel binary map and b channel binary map of the tobacco leaf image

[0046]

[0047] Among them, img 烟叶 Represents the tobacco leaf area map, img r_原图 Represents the r channel binary image of the tobacco leaf image, img b_原图 The b channel binary image of the tobacco leaf image. In the original RGB three-channel binary image of the tobacco leaf image, the area where the red channel minus the blue channel is greater than 30 is 255. Since the area where rb is greater than 30 is the yellow spectrum range, it can be obtained after calculation. Figure 4 The tobacco leaf area area such as the middle white area, that is, the tobacco leaf area map, such as Figure 4 shown.

[0048] The center point of the tobacco leaf in the image is used as the center of the Gaussian distribution, and the length and width of the tobacco leaf are used as the Gaussian radius. An independent number is converted into a two-dimensional image, so as to calculate the length change graph and width change graph of the tobacco leaf, such as Figure 5 and Figure 6 As shown, the calculation formula is as follows

[0049]

[0050] Among them, σ represents the Gaussian kernel, the value here is the length value or width value, g is the Gaussian formula, x and y are the center points of the two-dimensional Gaussian distribution, and here are the coordinates of the center point of the tobacco leaf in the image.

[0051] Perform RGB channel segmentation on the tobacco leaf image to segment the channel images into three channels, and calculate the binary image of each segmented channel image as follows

[0052]

[0053] img represents the original tobacco leaf image, split represents the segmentation algorithm, img r Represents the binary image of the r channel, img g Represents the binary image of the g channel, img b Represents the binary image of the b channel; the image of the tobacco leaf obtained after calculation is as follows Figure 7 shown.

[0054] Calculate the median image of the binary image of each channel segmented as follows

[0055]

[0056] Among them, img r_mid The median image of the r channel is represented by , and the median images of the g channel and the b channel are calculated in the same way; the image of the tobacco leaf obtained after calculation is as follows Figure 8 shown.

[0057] Calculate the mode map of each channel binary image segmented as follows

[0058]

[0059] Among them, img r_mode The mode map of the r channel is represented by , and the mode maps of the g channel and the b channel are calculated in the same way. The image of the tobacco leaf obtained after calculation is as follows Figure 9 shown.

[0060] Calculate the minimum value map of each channel binary image segmented as follows

[0061]

[0062] Among them, img r_min The minimum value map of the r channel is represented by the same equation. The minimum value maps of the g channel and the b channel are calculated in the same way. The image of the tobacco leaf obtained after calculation is as follows Figure 10 shown.

[0063] Calculate the maximum value of each channel binary image segmented as follows

[0064]

[0065] Among them, img r_max The maximum value of the r channel is shown in Figure 2. The maximum values ​​of the g and b channels are calculated in the same way. The image of the tobacco leaf obtained after calculation is shown in Figure 2. Figure 11 shown.

[0066] The 15 acquired two-dimensional images are synthesized into one image to obtain a 15-dimensional image feature map, which is used as the thickness variation map of the tobacco leaves. That is, the dimension of the thickness variation map is fifteen dimensions.

[0067] The lighting image of the tobacco leaf is segmented into RGB channels, and each segmented channel image is calculated to obtain the oil distribution map of the tobacco leaf, such as Figure 12 As shown below

[0068] img s1 =img r ÷img b

[0069] img s2 =img b ÷img g

[0070] img s3 =img g ÷img r

[0071] Among them, img s1 Represents the red and blue feature map, img s2 Represents the blue-green feature map, img s3 Represents the green-red feature map, img r Represents the binary image of the r channel, img g Represents the binary image of the g channel, img b This represents a binary image of the b channel. The dimensions of the red-blue feature map, the blue-green feature map, and the green-red feature map are each two-dimensional. Combining these three images into one gives the oil distribution map, which has a three-dimensional dimension.

[0072] By calculating the area of ​​the holes inside the tobacco leaf and the area of ​​the entire tobacco leaf in the tobacco leaf image, the damage coefficient of the tobacco leaf is calculated, that is, the ratio of the hole area to the tobacco leaf area, and the damage coefficient is converted into a tobacco leaf damage coefficient map using Gaussian distribution. That is, with the center point of the tobacco leaf in the image as the center of the Gaussian distribution and the damage coefficient as the Gaussian radius, an independent number is converted into a two-dimensional image, thereby calculating the tobacco leaf damage coefficient map. The calculation formula is as follows

[0073]

[0074] Among them, σ represents the Gaussian kernel, the value here is the breakage coefficient, g is the Gaussian formula, x and y are the center points of the two-dimensional Gaussian distribution, and here are the coordinates of the center point of the tobacco leaf in the image. The image obtained is as follows Figure 13 shown.

[0075] Finally, the length change graph, width change graph, area graph, thickness change graph, oil distribution graph and breakage coefficient graph of tobacco leaves are fed into the convolutional neural network to train convolution features, as follows

[0076]

[0077] Among them, img all_result This method combines multiple images into a single image, which is then fed into a neural network for training. Using this method, a single tobacco leaf can provide multi-dimensional image features, allowing a small amount of tobacco leaves to provide a large amount of training data. This significantly reduces the amount of tobacco leaves required to provide training data, enabling neural network feature learning with less tobacco leaf data, reducing the development cycle of deep learning algorithms and tobacco leaf costs.

[0078] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above and that the invention can be embodied in other specific forms without departing from the spirit or essential characteristics of the invention. Therefore, the embodiments should be considered in all respects as illustrative and non-restrictive, and the scope of the invention is defined by the appended claims, not the foregoing description, and all variations within the meaning and range of equivalents of the claims are intended to be included therein. Any reference sign in a claim should not be construed as limiting the claim to which it relates.

Claims

1. A deep learning training method for flue-cured tobacco leaf grading, characterized by: Acquire tobacco leaf images and perform de-impurity processing; Obtaining tobacco leaf part information, including length change graph, width change graph, area graph, and thickness change graph; Obtaining oil content information of tobacco leaves, including oil distribution maps of tobacco leaves; Obtaining tobacco leaf integrity information, including a tobacco leaf breakage coefficient graph; The tobacco leaf’s position information, oil content information, and integrity information are used as training data to train the convolutional neural network. Perform RGB channel segmentation on the tobacco leaf image and calculate the binary image, median image, mode image, maximum image and minimum image of each channel image segmented as follows Among them, img represents the original tobacco leaf image, img r Represents the binary image of the r channel, img g Represents the binary image of the g channel, img b Represents the binary image of the b channel; img r_mid Represents the median image of the r channel, img r_mode Represents the mode map of the r channel, img r_min Represents the minimum value map of the r channel, img r_max The maximum value map of the r channel is represented; the median map, mode map, minimum map and maximum value map of the g channel and the b channel are calculated in the same way; the 15 acquired two-dimensional images are synthesized into one map to obtain a 15-dimensional image feature map, which is used as the thickness change map of the tobacco leaves.

2. The deep learning training method for flue-cured tobacco leaf grading according to claim 1, characterized in that: After obtaining the tobacco leaf image, the background image outside the tobacco leaf in the image is removed to remove impurity information in the tobacco leaf image.

3. The deep learning training method for flue-cured tobacco leaf grading according to claim 1, characterized in that: Calculate the area map of the tobacco leaf based on the r channel binary map and b channel binary map of the tobacco leaf image Among them, img 烟叶 Represents the tobacco leaf area map, img r_原图 Represents the r channel binary image of the tobacco leaf image, img b_原图 Represents the b-channel binary image of the tobacco leaf image.

4. The deep learning training method for flue-cured tobacco leaf grading according to claim 1, characterized in that: Taking the center point of the tobacco leaf in the image as the center of the Gaussian distribution and the length and width of the tobacco leaf as the Gaussian radius, an independent number is converted into a two-dimensional image to calculate the length change graph and width change graph of the tobacco leaf. The calculation formula is as follows Among them, σ represents the Gaussian kernel, the value here is the length value or width value, g is the Gaussian formula, x and y are the center points of the two-dimensional Gaussian distribution, and here are the coordinates of the center point of the tobacco leaf in the image.

5. The deep learning training method for flue-cured tobacco leaf grading according to claim 1, characterized in that: An image acquisition device is deployed above and below the transparent image acquisition area of ​​the tobacco leaf, and a plane light source is deployed above the tobacco leaf to obtain the illuminated image of the tobacco leaf. The illuminated image of the tobacco leaf is segmented into RGB channels, and each segmented channel image is calculated to obtain the oil distribution map of the tobacco leaf, as shown below. img s1 =img r ÷img b img s2 =img b ÷img g img s3 =img g ÷img r Among them, img s1 Represents the red and blue feature map, img s2 Represents the blue-green feature map, img s3 Represents the green-red feature map, img r Represents the binary image of the r channel, img g Represents the binary image of the g channel, img b Represents a binary image of the b channel.

6. The deep learning training method for flue-cured tobacco leaf grading according to claim 1, characterized in that: The damage coefficient of the tobacco leaf is calculated by calculating the area of ​​the holes inside the tobacco leaf and the area of ​​the entire tobacco leaf in the tobacco leaf image, and the damage coefficient is converted into a tobacco leaf damage coefficient map using Gaussian distribution.

7. The deep learning training method for flue-cured tobacco leaf grading according to claim 1, characterized in that: The length change graph, width change graph, area graph, thickness change graph, oil distribution graph and breakage coefficient graph of tobacco leaves are fed into the convolutional neural network to train convolution features, as follows img all_result =[img r_mid ,img g_mid ...img s1 ,img s2 ,img s3 ] Among them, img all_result Indicates combining multiple graphs into one graph.

Images