A method and device for detecting and positioning a leaf with a stem
By combining image and thermal radiation data preprocessing and feature fusion with multiple laser point positioning technology, the accuracy problem of leaf stem detection was solved, achieving efficient and accurate leaf stem positioning and removal, thus improving the quality stability of tobacco products.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- HONGTA TOBACCO (GROUP) CO LTD
- Filing Date
- 2026-03-26
- Publication Date
- 2026-06-23
AI Technical Summary
Existing technologies for detecting stems in leaves suffer from low efficiency, poor positioning accuracy, and low rejection precision, which affect the stability of tobacco product quality.
Preprocessing was performed by combining images and thermal radiation data of the smoke sheet. Features were extracted using ResNet50 and Sobel operators. Cross-modal fusion was performed using a Cross-Modal Transformer structure. Dynamic thresholding algorithm was used to determine areas where the leaf stem exceeded the standard. Automatic displacement positioning and tracking were performed using multiple laser points.
This improved the positioning and rejection accuracy of leaf stem detection, reduced false judgments, and ensured the stability of tobacco product quality.
Smart Images
Figure CN122259564A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of tobacco leaf detection technology, specifically to a method and apparatus for detecting and locating the presence of stems in leaves. Background Technology
[0002] In the tobacco manufacturing process, the stem-to-leaf ratio plays a decisive role in tobacco quality. Currently, traditional methods for detecting and removing stems from tobacco leaves have many drawbacks. Manual inspection is not only extremely inefficient but also prone to significant errors due to human factors, and its heavy reliance on manual operation leads to high labor costs. While some existing automated equipment has improved detection efficiency, its positioning accuracy is poor, and its removal precision is low, making it difficult to accurately control the stem content in tobacco leaves, seriously affecting the stability of tobacco product quality. Therefore, to improve the detection and positioning accuracy of stems in tobacco leaves, this application provides a method and apparatus for detecting and positioning stems in tobacco leaves. Summary of the Invention
[0003] The purpose of this invention is to provide a method and apparatus for detecting and locating leaves containing stems, in order to improve the detection and location accuracy of leaves containing stems, thereby improving the rejection accuracy.
[0004] To solve the above-mentioned technical problems, the present invention adopts the following technical solution:
[0005] A method for detecting and locating leaf stems, comprising:
[0006] Acquire images and thermal radiation data of the smoke sheet;
[0007] Preprocessing of smoke sheet images and thermal radiation data;
[0008] By combining image data and thermal radiation data, it can be determined whether the tobacco leaves contain excessive stems, and the location and extent of the excessive tobacco leaves can be identified.
[0009] Based on the location information of excessive stems in the leaves, multiple laser points are emitted to automatically locate and track the displacement of the excessive tobacco flakes.
[0010] Furthermore, the preprocessing of the smoke image includes image grayscale conversion, filtering and denoising, and edge enhancement operations, while the preprocessing of the thermal radiation data includes wavelet denoising and thermal radiation temperature difference calculation.
[0011] Furthermore, the smoke image is grayscaled using the Otsu dynamic threshold segmentation algorithm, with an initial inter-class variance threshold σ0 = 0.18.
[0012] In wavelet denoising of thermal radiation data, the high-frequency component energy is retained at a rate of ≥75%.
[0013] Furthermore, the method of determining whether tobacco leaves contain excessive stems by combining image data and thermal radiation data includes:
[0014] ResNet50 was used for image feature extraction, and the Sobel operator was used to extract the temperature gradient in the X / Y directions.
[0015] A Cross-Modal Transformer structure is used to perform cross-modal fusion of image features and temperature gradient features to generate a joint feature vector.
[0016] Based on the joint feature vector, a dynamic threshold algorithm is used to determine the region where the leaf stalk exceeds the standard. The formula for calculating the dynamic threshold is as follows:
[0017]
[0018] This represents the average width of the leaf veins. Here is the standard deviation, and α is the adjustment coefficient. For local temperature differences, This represents the average temperature of the tobacco sheet.
[0019] The region exceeding the limit is determined and its three-dimensional coordinates are marked by comparing the joint feature vector with the dynamic threshold thres.
[0020] Furthermore, after determining the location and range of the substandard tobacco flakes, an alarm signal is generated to trigger an alarm.
[0021] In addition, this application also provides a leaf stem detection and positioning device, the detection and positioning device comprising:
[0022] A CCD high-definition camera is used to acquire images of the smoke sheets;
[0023] Thermal infrared imager is used to acquire thermal radiation data of smoke sheets;
[0024] The processing module is used to perform preprocessing of tobacco leaf images and thermal radiation data, and to combine image data and thermal radiation data to determine whether the tobacco leaves contain excessive stems, and to determine the location and range of tobacco leaves that exceed the standard;
[0025] Multi-line lidar is used to automatically locate and track the excessive smoke flakes by emitting multiple laser points based on the location information of the excessive stem content in the leaves.
[0026] The alarm module is used to generate an alarm signal after determining the location and range of the excessive smoke flakes.
[0027] Compared with the prior art, the beneficial effects of the present invention are:
[0028] This application combines images and thermal radiation data of tobacco leaves to determine whether the leaves contain stems. Compared with traditional methods that rely solely on images, this method has higher accuracy and reduces the likelihood of misjudgment. Furthermore, this application uses multiple laser points to automatically locate and track the displacement of tobacco leaves that exceed the standard. Compared with single laser point positioning, multiple laser points can more accurately delineate the location of tobacco leaves that exceed the standard, effectively avoiding the deviation problems that may be caused by single-point positioning. This method can effectively improve the detection and positioning accuracy of leaves containing stems. Attached Figure Description
[0029] Figure 1 This is a flowchart illustrating the execution of the method described in this invention.
[0030] Figure 2 for Figure 1 A flowchart of one implementation method for step S03.
[0031] Figure 3 This is a structural block diagram of the leaf stem detection and positioning device provided by the present invention. Detailed Implementation
[0032] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
[0033] refer to Figure 1 As shown, this application provides a method for detecting and locating leaf stems, including:
[0034] S01. Acquire images and thermal radiation data of the smoke sheet;
[0035] Specifically, existing technologies for detecting leaf stems typically rely on image analysis. However, this approach has limitations due to image resolution constraints, the relatively small differences in color or texture between leaves and stems, the influence of lighting conditions, and deficiencies in image processing algorithms. Furthermore, environmental factors such as dust and humidity can also affect image quality, thus impacting detection accuracy. This application combines image data and thermal radiation data. Since the thermal radiation characteristics of leaves and stems differ due to material variations, combining these two data points further improves detection accuracy.
[0036] S02. Preprocess the smoke image and thermal radiation data;
[0037] Specifically, the preprocessing of the smoke image includes image grayscale conversion, filtering and denoising, and edge enhancement operations. The preprocessing of the thermal radiation data includes wavelet denoising and thermal radiation temperature difference calculation. More specifically, in order to perform noise reduction processing on the data, the smoke image is grayscale converted using the Otsu dynamic threshold segmentation algorithm, with an initial inter-class variance threshold σ0 = 0.18.
[0038] In wavelet denoising of thermal radiation data, the high-frequency component energy is retained at a rate of ≥75%. This denoising process avoids the influence of extreme low-frequency data.
[0039] S03. Combine image data and thermal radiation data to determine whether the tobacco leaves contain excessive stems, and determine the location and range of the excessive tobacco leaves;
[0040] A further step in this process is,
[0041] S031. ResNet50 is used to extract image features, and the Sobel operator is used to extract the temperature gradient in the X / Y direction.
[0042] Replacing standard convolutions with ResNet50 reduces computation by 40%, thereby improving response speed. Specifically, for temperature gradients, the Sobel operator convolution kernel is:
[0043] ,
[0044] and These are the Sobel operators for the X and Y directions, respectively.
[0045] S032. A Cross-Modal Transformer structure is used to perform cross-modal fusion of image features and temperature gradient features to generate a joint feature vector. The specific structure of the cross-modal fusion is as follows:
[0046] Attention(Q,K,V)=Softmax( V;
[0047] Among them, Q comes from visual features, and K and V come from thermal features;
[0048] S033. Based on the joint feature vector, a dynamic threshold algorithm is used to determine the region where the leaf stalk exceeds the standard. The formula for calculating the dynamic threshold is:
[0049]
[0050] This represents the average width of the leaf veins. Here is the standard deviation, and α is the adjustment coefficient. For local temperature differences, This represents the average temperature of the tobacco sheet.
[0051] S034. By comparing the joint feature vector with the dynamic threshold thres, the out-of-limit region is determined and the three-dimensional coordinates are marked. If it exceeds the dynamic threshold thres, it is determined to be an out-of-limit region.
[0052] S04. Based on the location information of excessive stems in the leaves, multiple laser points are emitted to automatically locate and track the excessive tobacco leaves.
[0053] In addition, during step S04, the steps performed simultaneously include: after determining the location and range of the substandard tobacco flakes, generating an alarm signal to ensure that the operator can receive the alarm signal in a timely and accurate manner, thereby quickly carrying out subsequent selection and removal work.
[0054] On the other hand, based on the above method, this application also provides a leaf stem detection and positioning device, the detection and positioning device comprising:
[0055] CCD high-definition camera 1 is used to acquire images of tobacco leaves. Preferably, the CCD high-definition camera 1 has a resolution of ≥5000×5000 pixels, a frame rate of ≥30fps, and RGB+near-infrared dual channels, so as to clearly acquire the surface texture of tobacco leaves.
[0056] Thermal infrared imager 2 is used to acquire thermal radiation data of smoke sheets. Preferably, the thermal infrared imager 2 has a thermal sensitivity ≤0.03℃@30Hz and a temperature measurement range of -20~550℃.
[0057] During implementation, the CCD high-definition camera 1 and the thermal infrared imager 2 were installed at the optimal positions in the smoke sheet conveying area of the air separator to ensure comprehensive and clear acquisition of images and thermal radiation temperature difference data of the smoke sheet material. The shooting angle and focal length of the CCD high-definition camera 1, as well as the detection range and sensitivity parameters of the thermal infrared imager 2, were carefully adjusted to enable it to accurately detect the characteristic information of the smoke sheet and stem.
[0058] Processing module 3 is used to perform preprocessing of tobacco leaf images and thermal radiation data, and to combine image data and thermal radiation data to determine whether the tobacco leaves contain excessive stems, and to determine the location and range of tobacco leaves that exceed the standard. Processing module 3 can be integrated into an industrial control system.
[0059] The multi-line lidar 4 is used to automatically locate and track the excessive smoke flakes by emitting multiple laser points based on the location information of the excessive stem content in the leaves.
[0060] Similarly, when installing the multi-line lidar 4, ensure that the laser points it emits can completely cover the smoke sheet conveying area of the wind separator, and establish a stable and reliable data connection with the processing module 3.
[0061] Alarm module 5 is used to generate an alarm signal after determining the location and range of the excessive smoke flakes.
[0062] The working process of this application:
[0063] During tobacco production, CCD high-definition cameras and thermal infrared imagers collect images and thermal radiation temperature difference data of the tobacco leaves in real time and synchronously, and quickly transmit this data to the processing module. The processing module uses built-in algorithms for preprocessing and real-time analysis to accurately determine whether the tobacco leaves contain excessive amounts of stems. Once excessive stems are detected, the processing module immediately sends a command to a multi-line lidar system, which then emits multiple laser points to automatically locate and track the excessive stems. Simultaneously, the alarm module issues a clear and unambiguous alarm signal to alert operators to take timely action. Based on the location indication of the laser points, operators accurately select and remove the excessive stems, completing the stem removal process.
[0064] In this specification, terms such as "one embodiment," "another embodiment," "embodiment," and "preferred embodiment" refer to specific features, structures, or characteristics described in connection with that embodiment, which are included in at least one embodiment described in the general description of this application. The appearance of the same term in multiple places in the specification does not necessarily refer to the same embodiment. Furthermore, when a specific feature, structure, or characteristic is described in connection with any embodiment, the intention is to suggest that implementing such a feature, structure, or characteristic in conjunction with other embodiments also falls within the scope of this invention.
[0065] Although the invention has been described herein with reference to several illustrative embodiments, it should be understood that many other modifications and implementations can be devised by those skilled in the art, which will fall within the scope and spirit of the principles disclosed herein. More specifically, various modifications and improvements can be made to the components or layout of the subject matter arrangement within the scope of the disclosure, drawings, and claims. Besides modifications and improvements to the components or layout, other uses will be apparent to those skilled in the art.
Claims
1. A method for detecting and locating leaf stems, characterized in that, include: Acquire images and thermal radiation data of the smoke sheet; Preprocessing of smoke sheet images and thermal radiation data; By combining image data and thermal radiation data, it can be determined whether the tobacco leaves contain excessive stems, and the location and extent of the excessive tobacco leaves can be identified. Based on the location information of excessive stems in the leaves, multiple laser points are emitted to automatically locate and track the displacement of the excessive tobacco flakes.
2. The leaf-stalk detection and localization method according to claim 1, characterized in that: The preprocessing of the smoke image includes image grayscale conversion, filtering and denoising, and edge enhancement operations. The preprocessing of the thermal radiation data includes wavelet denoising and thermal radiation temperature difference calculation.
3. The leaf stalk detection and localization method according to claim 2, characterized in that: The smoke image is converted to grayscale using the Otsu dynamic threshold segmentation algorithm, with an initial inter-class variance threshold σ0 = 0.
18. In wavelet denoising of thermal radiation data, the high-frequency component energy is retained at a rate of ≥75%.
4. The leaf-stalk detection and localization method according to claim 1, characterized in that: The method of determining whether tobacco leaves contain excessive stems by combining image data and thermal radiation data includes: ResNet50 was used for image feature extraction, and the Sobel operator was used to extract the temperature gradient in the X / Y directions. A Cross-Modal Transformer structure is used to perform cross-modal fusion of image features and temperature gradient features to generate a joint feature vector. Based on the joint feature vector, a dynamic threshold algorithm is used to determine the region where the leaf stalk exceeds the standard. The formula for calculating the dynamic threshold is as follows: ； This represents the average width of the leaf veins. Here is the standard deviation, and α is the adjustment coefficient. For local temperature differences, This represents the average temperature of the tobacco sheet. The system compares the joint feature vector with the dynamic threshold thres to determine the out-of-range region and marks its three-dimensional coordinates.
5. The leaf-stalk detection and localization method according to claim 1, characterized in that: After determining the location and range of the substandard tobacco flakes, an alarm signal is generated to trigger an alarm.
6. The leaf stem detection and positioning device according to claim 1, characterized in that: The detection and positioning device includes: A CCD high-definition camera is used to acquire images of the smoke sheets; Thermal infrared imager is used to acquire thermal radiation data of smoke sheets; The processing module is used to perform preprocessing of tobacco leaf images and thermal radiation data, and to combine image data and thermal radiation data to determine whether the tobacco leaves contain excessive stems, and to determine the location and range of tobacco leaves that exceed the standard; Multi-line lidar is used to automatically locate and track the excessive smoke flakes by emitting multiple laser points based on the location information of the excessive stem content in the leaves. The alarm module is used to generate an alarm signal after determining the location and range of the excessive smoke flakes.