Method for removing tobacco stems from tobacco leaves based on image recognition

Abstract

The application discloses a method for removing tobacco stems in tobacco leaves based on image recognition, which comprises the steps of tobacco leaf conveying, obtaining tobacco leaf image data, image preprocessing, tobacco stem detection, tobacco stem classification and tobacco stem removal, etc. Through the steps of image preprocessing, tobacco stem detection, tobacco stem classification and removal, etc., the method can efficiently and accurately remove the tobacco stems in the tobacco leaves, has the advantages of simple operation, high precision, high efficiency and the like, can quickly and completely remove all the tobacco stems, realizes the removal treatment of different varieties, different growth environments and different harvesting periods of the tobacco, meanwhile, the integrity and quality of the tobacco leaves are maintained, and the method has important significance for improving the quality of the tobacco products.

Description

Technical Field

[0001] This invention relates to the field of cigarette production, and more specifically to a method for removing tobacco stems from tobacco leaves based on image recognition. Background Technology

[0002] Tobacco is one of the world's most important economic crops, and its processed and manufactured tobacco products are among the most important consumer goods on the global market. However, due to differences in tobacco field management and harvesting processes, tobacco leaves often contain a large amount of tobacco stems. Figure 7 The diagram shows the structure of a tobacco leaf, with the central black section representing the stem and the leaves on either side. These stems reduce the aroma and flavor of the tobacco, affecting its quality. Therefore, the stems must be removed from the tobacco leaves during processing and manufacturing.

[0003] Currently, several methods have been used for the detection and identification of tobacco stems. However, these methods have some problems. For example, traditional visual inspection methods require a lot of manpower, resources, and time, and are not only inaccurate, but also increase tobacco leaf breakage and cause serious material loss due to manual stem removal. Hyperspectral imaging methods are susceptible to interference from noise and image changes, and are generally used to detect the stem content in tobacco leaves, but cannot effectively remove tobacco stems from tobacco leaves.

[0004] Therefore, developing a method that can efficiently, accurately, and intelligently remove tobacco stems from tobacco leaves is of great significance for improving the quality of tobacco products. Summary of the Invention

[0005] To overcome the above problems, this invention provides a method for removing tobacco stems from tobacco leaves based on image recognition. This method effectively identifies and removes tobacco stems by analyzing images of tobacco leaves.

[0006] The technical solution adopted by this invention to solve its technical problem is as follows:

[0007] Methods for removing tobacco stems from tobacco leaves based on image recognition, including

[0008] S1 Tobacco Leaf Conveying

[0009] The tobacco leaves to be tested are conveyed into the discretization device via a conveyor belt. The discretization device spreads the tobacco leaves evenly and thinly, and then the tobacco leaves enter the single leaf selection channel.

[0010] The conveyor belt speed is reduced to 5 cm / s, and then the channel is narrowed so that a single tobacco leaf is laid flat on the conveyor belt.

[0011] S2 acquires tobacco leaf image data

[0012] An image acquisition device is installed above the conveyor belt to capture images of individual tobacco leaves passing through the shooting area, and the image data of the tobacco leaves is transmitted to a computer.

[0013] S3 Image Preprocessing

[0014] The tobacco leaf image is preprocessed by noise removal, grayscale conversion, and binarization to remove the background area and obtain the target tobacco leaf area.

[0015] S4 tobacco stem detection

[0016] A deep learning-based convolutional neural network algorithm is used to detect tobacco stems in tobacco leaf images. Combined with the features of tobacco stems, tobacco leaves and tobacco stems are classified and identified. The contours of tobacco leaves and tobacco stems are automatically detected in tobacco leaf images, the tobacco stems are identified, and their position and morphological parameters are extracted.

[0017] This step uses a deep residual network as the basic network structure;

[0018] First, multi-scale convolutional layers and pooling layers are designed to adapt to tobacco stems and leaves of different sizes and shapes, taking into account the characteristics of tobacco leaf images.

[0019] Example: After obtaining the tobacco leaf image preprocessed by S3, image segmentation is performed; to ensure that the image size is 500px, the edge parts need to be padded with 0, where W is the input size, O is the output size, K is the filter size, P is the padding, and S is the stride.

[0020] According to the formula, S = (W - K + 2P) / O - 1 = 2, so the movement step of the filter is 2.

[0021] To accommodate tobacco leaf images with varying sizes and shapes, a double pooling method is designed. The resulting images are then compared with tobacco leaf and stem feature maps using the REU activation function. The matching degree is determined based on the calculated values.

[0022] If the segmented image is matched with the tobacco leaf feature map and the value exceeds 70%, the segmented image is considered to be a tobacco leaf and is retained; otherwise, the image is removed. Finally, an image with only the tobacco leaf and no tobacco stem is obtained. If the segmented image is matched with the tobacco stem feature map and the value exceeds 70%, the segmented image is considered to be a tobacco stem; otherwise, the image is removed. Finally, an image with only the tobacco stem and no tobacco leaf is obtained.

[0023] Secondly, a local perception mechanism and backpropagation algorithm are used to train and optimize the network to improve the classification accuracy of tobacco stems;

[0024] S5 tobacco stem classification

[0025] The tobacco leaf and stem images obtained in step S4 are used to calculate the outline side lengths of the tobacco leaves and stems by summing the outer contour pixels. The proportion of the stem in the tobacco leaves is determined by dividing the stem perimeter by the tobacco leaf perimeter, and a stem proportion threshold is set.

[0026] The threshold includes two criteria: criterion 1 is the outline length of the tobacco stem, and criterion 2 is the proportion of the tobacco stem in the tobacco leaf.

[0027] If condition 1 is not met, i.e. the outline side length of the tobacco stem exceeds the set threshold a, the tobacco stem in the tobacco leaf is determined to be removed.

[0028] If condition 2 is not met, that is, the proportion of tobacco stems in tobacco leaves exceeds the set threshold b, it is determined that the tobacco stems in the tobacco leaves need to be removed.

[0029] If conditions 1 and 2 are not met, that is, the outline side length of the tobacco stem exceeds the set threshold a and the proportion of the tobacco stem in the tobacco leaf exceeds the set threshold b, it is determined that the tobacco stem in the tobacco leaf needs to be removed.

[0030] If conditions 1 and 2 are both met, that is, the outline side length of the tobacco stem does not exceed the set threshold a and the proportion of the tobacco stem in the tobacco leaf does not exceed the set threshold b, it is determined that the tobacco stem in the tobacco leaf does not need to be removed.

[0031] Step 6: Remove tobacco stems

[0032] If it is determined that the tobacco leaves do not need to have their stems removed, they will enter the cabinet feeding conveyor belt when passing through the sorting channel;

[0033] If it is determined that the tobacco leaves need to have their stems removed, they will enter the stem separation conveyor belt as they pass through the sorting channel.

[0034] A three-pronged device with a camera is installed above the tobacco stem separating conveyor belt. The three-pronged device rotates to adjust the direction of the tobacco stem toward the forward direction of the belt. The saw teeth of the three-pronged device vibrate to cut and separate the tobacco leaves on both sides of the tobacco stem, thereby removing the tobacco stem.

[0035] The cut tobacco leaves enter the cabinet feeding conveyor belt, and the cut tobacco stems enter the tobacco stem collection cabinet.

[0036] The beneficial effects of this invention are as follows:

[0037] This invention can be adjusted and optimized according to different tobacco varieties and growing environments to achieve better removal results.

[0038] It has the following advantages:

[0039] 1. High efficiency: Utilizing automated image recognition technology, it can quickly and accurately remove tobacco stems from tobacco leaves, improving work efficiency.

[0040] 2. High accuracy: The classification model is built using deep learning algorithms, which can effectively distinguish the subtle differences between tobacco stems and tobacco leaves, improving the accuracy of rejection.

[0041] 3. Easy to operate: The system is easy to operate; the rejection process can be completed simply by taking pictures and processing them with a digital camera and a computer. Attached Figure Description

[0042] The present invention will be further described below with reference to the accompanying drawings and specific embodiments:

[0043] Figure 1 This is a schematic diagram of the process of the present invention;

[0044] Figure 2 This is a schematic diagram of the tobacco stem recognition process based on a convolutional neural network according to the present invention;

[0045] Figure 3 This is a simplified structural diagram of the discrete device in Example 2;

[0046] Figure 4 This is a simplified diagram of the single-chip selection channel and narrowing channel structure in Example 2;

[0047] Figure 5 , Figure 6 This is a simplified structural diagram of the three-pronged device in Example 2;

[0048] Figure 7 A simplified schematic diagram of the structure of tobacco leaves and stems. Detailed Implementation

[0049] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0050] Example 1

[0051] Reference Figure 1 Methods for removing tobacco stems from tobacco leaves based on image recognition include

[0052] S1 Tobacco Leaf Conveying

[0053] The tobacco leaves to be tested are conveyed into the discretization device via a conveyor belt. The discretization device spreads the tobacco leaves evenly and thinly, and then the tobacco leaves enter the single leaf selection channel.

[0054] The conveyor belt speed is reduced to 5 cm / s, and then the channel is narrowed so that a single tobacco leaf is laid flat on the conveyor belt.

[0055] S2 acquires tobacco leaf image data

[0056] An image acquisition device is installed above the conveyor belt to photograph individual tobacco leaves passing through the shooting area, and the image data of the tobacco leaves is transmitted to a computer; the image acquisition device uses a digital camera with 2 million pixels to ensure image resolution;

[0057] S3 Image Preprocessing

[0058] The tobacco leaf image is preprocessed by noise removal, grayscale conversion, and binarization to remove the background area and obtain the target tobacco leaf area.

[0059] This embodiment uses the deep learning framework TensorFlow for image preprocessing, tobacco stem detection, and classification.

[0060] S4 tobacco stem detection

[0061] Reference Figure 2 The algorithm uses a deep learning-based convolutional neural network to detect tobacco stems in tobacco leaf images. Combined with the features of tobacco stems, it classifies and identifies tobacco leaves and stems. It automatically detects the contours of tobacco leaves and stems in tobacco leaf images, finds the stems, and extracts their position and morphological parameters.

[0062] This step uses a deep residual network as the basic network structure;

[0063] First, multi-scale convolutional layers and pooling layers are designed to adapt to tobacco stems and leaves of different sizes and shapes, taking into account the characteristics of tobacco leaf images.

[0064] Example: After obtaining the tobacco leaf image preprocessed by S3, image segmentation is performed; to ensure that the image size is 500px, the edge parts need to be padded with 0, where W is the input size, O is the output size, K is the filter size, P is the padding, and S is the stride.

[0065] According to the formula, S = (W - K + 2P) / O - 1 = 2, so the movement step of the filter is 2.

[0066] To accommodate tobacco leaf images with varying sizes and shapes, a double pooling method is designed. The resulting images are then compared with tobacco leaf and stem feature maps using the REU activation function. The matching degree is determined based on the calculated values.

[0067] If the segmented image is matched with the tobacco leaf feature map and the value exceeds 70%, the segmented image is considered to be a tobacco leaf and is retained; otherwise, the image is removed. Finally, an image with only the tobacco leaf and no tobacco stem is obtained. If the segmented image is matched with the tobacco stem feature map and the value exceeds 70%, the segmented image is considered to be a tobacco stem; otherwise, the image is removed. Finally, an image with only the tobacco stem and no tobacco leaf is obtained.

[0068] Secondly, a local perception mechanism and backpropagation algorithm are used to train and optimize the network to improve the classification accuracy of tobacco stems;

[0069] S5 tobacco stem classification

[0070] The tobacco leaf and stem images obtained in step S4 are used to calculate the outline side lengths of the tobacco leaves and stems by summing the outer contour pixels. The proportion of the stem in the tobacco leaves is determined by dividing the stem perimeter by the tobacco leaf perimeter, and a stem proportion threshold is set.

[0071] The threshold includes two criteria: criterion 1 is the outline length of the tobacco stem, and criterion 2 is the proportion of the tobacco stem in the tobacco leaf.

[0072] If condition 1 is not met, i.e. the outline side length of the tobacco stem exceeds the set threshold a, the tobacco stem in the tobacco leaf is determined to be removed.

[0073] If condition 2 is not met, that is, the proportion of tobacco stems in tobacco leaves exceeds the set threshold b, it is determined that the tobacco stems in the tobacco leaves need to be removed.

[0074] If conditions 1 and 2 are not met, that is, the outline side length of the tobacco stem exceeds the set threshold a and the proportion of the tobacco stem in the tobacco leaf exceeds the set threshold b, it is determined that the tobacco stem in the tobacco leaf needs to be removed.

[0075] If conditions 1 and 2 are both met, that is, the outline side length of the tobacco stem does not exceed the set threshold a and the proportion of the tobacco stem in the tobacco leaf does not exceed the set threshold b, it is determined that the tobacco stem in the tobacco leaf does not need to be removed.

[0076] Step 6: Remove tobacco stems

[0077] If it is determined that the tobacco leaves do not need to have their stems removed, they will enter the cabinet feeding conveyor belt when passing through the sorting channel;

[0078] If it is determined that the tobacco leaves need to have their stems removed, they will enter the stem separation conveyor belt as they pass through the sorting channel.

[0079] A three-pronged device with a camera is installed above the tobacco stem separating conveyor belt. The three-pronged device rotates to adjust the direction of the tobacco stem toward the forward direction of the belt. The saw teeth of the three-pronged device vibrate to cut and separate the tobacco leaves on both sides of the tobacco stem, thereby removing the tobacco stem.

[0080] The cut tobacco leaves enter the cabinet feeding conveyor belt, and the cut tobacco stems enter the tobacco stem collection cabinet.

[0081] Example 2

[0082] A system for removing tobacco stems from tobacco leaves based on image recognition includes a tobacco leaf conveying device, an image acquisition device, an image processing device, and a tobacco stem cutting device.

[0083] The tobacco leaf conveying device includes a discrete device, a single leaf sorting channel, and a narrowing channel arranged sequentially on the conveyor belt;

[0084] Reference Figure 3 , Figure 4 (Simplified for illustrating the structure and position of the device, not the actual structure or scale) The tobacco leaves to be tested are centrally stored in the leaf storage cabinet and transported out by the outlet conveyor belt to the discretization device. The discretization device includes a vibrating arm 11 located below the conveyor belt, baffles 12 on both sides above the conveyor belt, and a square roller 13. The vibration and roller of the discretization device are used to spread the tobacco leaves evenly and thinly. Then, the tobacco leaves enter the single leaf selection channel, which includes a rotating fan blade 21 located above the conveyor belt and a narrowing channel 22 located at its end. First, the speed of the conveyor belt drive motor is reduced, so that the belt conveying speed is reduced to 5 cm / s. Then, the single tobacco leaf is laid flat on the conveyor belt by passing through the narrowing channel 22.

[0085] An image acquisition device is installed above the conveyor belt to photograph individual tobacco leaves, transmitting the image data to an image processing device. The image acquisition device uses a 2-megapixel digital camera to ensure image resolution.

[0086] The image processing device preprocesses the image to remove light, shadow, noise and other defects, and extracts the position and morphological parameters of the tobacco stem.

[0087] Specifically, the image processing device includes an image preprocessing module, a neural network analysis module, and a tobacco stem location estimation module. The image processing module preprocesses the acquired images; the neural network analysis module identifies tobacco leaves and stems from the preprocessed images, then identifies the stems and determines their size and location; finally, a stem cutting device trims the tobacco leaves.

[0088] The image processing device uses the deep learning framework TensorFlow for image preprocessing, tobacco stem detection, and classification, while using deep learning convolutional neural network algorithms for image training.

[0089] Finally, the tobacco stems are removed from the tobacco leaves using a stem cutting device. A three-pronged device with a camera is installed above the stem separating conveyor belt. (See reference) Figure 5 , Figure 6 The three-pronged device includes three rotatable metal spikes 31 positioned above the conveyor belt for fixing / adjusting the direction of the tobacco leaves, a camera 32 located at the front end of the metal spikes, and serrations 33 located on both sides of the metal spikes. By rotating the metal spikes 31, the direction of the tobacco stems on the tobacco leaves is adjusted to face the forward direction of the belt. By vibrating the serrations 33 of the three-pronged device, the tobacco leaves on both sides of the tobacco stems are cut and separated, thereby removing the tobacco stems.

[0090] It should be noted that the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for removing tobacco stems from tobacco leaves based on image recognition, characterized in that: include S1 Tobacco Leaf Conveying The tobacco leaves to be tested are conveyed into the discretization device via a conveyor belt. The discretization device spreads the tobacco leaves evenly and thinly, and then they enter the tobacco leaf single leaf selection channel. By narrowing the channel, the single tobacco leaf is laid flat on the conveyor belt. S2 acquires tobacco leaf image data An image acquisition device is installed above the conveyor belt to capture images of individual tobacco leaves passing through the shooting area, and the image data of the tobacco leaves is transmitted to a computer. S3 Image Preprocessing The tobacco leaf image is preprocessed by noise removal, grayscale conversion, and binarization to remove the background area and obtain the target tobacco leaf area. S4 tobacco stem detection A deep learning-based convolutional neural network algorithm is used to detect tobacco stems in tobacco leaf images. Combined with the features of tobacco stems, tobacco leaves and tobacco stems are classified and identified. The contours of tobacco leaves and tobacco stems are automatically detected in tobacco leaf images, the tobacco stems are identified, and their position and morphological parameters are extracted. S5 tobacco stem classification The tobacco leaf and stem images obtained in step S4 are used to calculate the outline side lengths of the tobacco leaves and stems by summing the outer contour pixels. The proportion of the stem in the tobacco leaves is determined by dividing the stem perimeter by the tobacco leaf perimeter, and a stem proportion threshold is set. The threshold includes two criteria: criterion 1 is the outline length of the tobacco stem, and criterion 2 is the proportion of the tobacco stem in the tobacco leaf. If condition 1 is not met, i.e. the outline side length of the tobacco stem exceeds the set threshold a, the tobacco stem in the tobacco leaf is determined to be removed. If condition 2 is not met, that is, the proportion of tobacco stems in tobacco leaves exceeds the set threshold b, it is determined that the tobacco stems in the tobacco leaves need to be removed. If conditions 1 and 2 are not met, that is, the outline side length of the tobacco stem exceeds the set threshold a and the proportion of the tobacco stem in the tobacco leaf exceeds the set threshold b, it is determined that the tobacco stem in the tobacco leaf needs to be removed. If conditions 1 and 2 are both met, that is, the outline side length of the tobacco stem does not exceed the set threshold a and the proportion of the tobacco stem in the tobacco leaf does not exceed the set threshold b, it is determined that the tobacco stem in the tobacco leaf does not need to be removed. Step 6: Remove tobacco stems If it is determined that the tobacco leaves do not need to have their stems removed, they will enter the cabinet feeding conveyor belt when passing through the sorting channel; If it is determined that the tobacco leaves need to have their stems removed, they will enter the stem separation conveyor belt as they pass through the sorting channel. A three-pronged device with a camera is installed above the tobacco stem separating conveyor belt. The three-pronged device rotates to adjust the direction of the tobacco stems toward the forward direction of the belt. The saw teeth of the three-pronged device vibrate to cut and separate the tobacco leaves on both sides of the tobacco stems, thereby removing the tobacco stems.

2. The method for removing tobacco stems from tobacco leaves based on image recognition according to claim 1, characterized in that: In S1, the conveyor belt speed of the tobacco leaves to be tested is reduced to 5 cm / s when passing through the narrow channel.

3. The method for removing tobacco stems from tobacco leaves based on image recognition according to claim 1, characterized in that: In step S4, a deep residual network is used as the basic network structure. First, multi-scale convolutional and pooling layers are designed to accommodate tobacco stems and leaves of different sizes and shapes, taking into account the characteristics of tobacco leaf images. Second, a local perception mechanism and backpropagation algorithm are used to train and optimize the network to improve the classification accuracy of tobacco stems.

Images