A tobacco leaf grading method and system based on machine vision
Through the integrated learning method based on capsule neural network, the problems of strong subjectivity, low efficiency and insufficient accuracy in the grading of curled tobacco leaves are solved, and the efficient and accurate level of curled tobacco leaves is achieved, which is suitable for complex morphology and occlusion characteristics. Identification has improved the production efficiency and automation level of the tobacco industry.
Patent Information
- Application Number
- CN202411094180.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-09
- Publication Date
- 2024-12-06
- Estimated Expiration
- 2044-08-09
AI Technical Summary
The existing curled tobacco leaf rating methods have problems such as strong subjectivity, low efficiency and insufficient accuracy. Especially in dealing with the diversity and complexity of curled tobacco leaf images, traditional convolutional neural networks are difficult to accurately capture complex morphology and occlusion features.
The integrated learning method based on capsule neural network is adopted to build an integrated capsule network model, combining image preprocessing and dynamic routing mechanisms to efficient feature extraction and grading of curled tobacco leaves, reducing manual operation errors, and improving grading efficiency and accuracy.
Improve the accuracy and robustness of curled tobacco leaf image classification, better understand complex morphology and occlusion characteristics, reduce dependence on manual annotation, adapt to tobacco leaf images of different curling degrees and morphology, and significantly improve the production efficiency of the tobacco industry. and automation level.
Smart Images

Figure CN118628482B_ABST
Abstract
Description
Technical Field
[0001] The invention belongs to the technical field of curled tobacco leaf grade recognition, and in particular relates to a curled tobacco leaf grading method and system based on machine vision. Background Art
[0002] As an important raw material for tobacco products, the quality of tobacco leaves directly affects the flavor and quality of the final product. Therefore, accurate and reliable tobacco leaf grading is of great significance to the tobacco industry. Traditional tobacco leaf grading methods rely on manual visual judgment, which is inefficient and highly subjective. With the development of computer vision technology, automated tobacco leaf grading methods have gradually attracted attention. However, image feature extraction of curled tobacco leaves faces challenges because curling may make some feature information invisible, which brings additional challenges to image classification.
[0003] With the development of computer vision and machine learning technologies, methods based on image processing and neural networks have begun to be applied to tobacco leaf grading. As a common deep learning model, convolutional neural network (CNN) has been widely used for image classification and feature extraction. However, the irregularity of curly tobacco leaf images and the possibility that some features may be blocked bring additional challenges to tobacco leaf image classification. Traditional convolutional neural networks may find it difficult to accurately capture these complex visual changes.
[0004] As a new type of neural network architecture, Capsule Network can provide richer feature expressions than traditional convolutional neural networks. Capsule Network captures the spatial relationship and posture changes of objects in the image in the form of vectors, which enables the network to more accurately identify and understand the complex morphology of curly tobacco leaves.
[0005] Ensemble learning is a powerful machine learning method that improves the overall classification performance by combining the prediction results of multiple models. Ensemble learning can effectively reduce the overfitting of the model and improve the accuracy and robustness of classification. Although existing automation technology has made progress in some aspects, it still has shortcomings in dealing with the diversity and complexity of curly tobacco leaf images.
[0006] The present invention proposes a curled tobacco leaf grading method and system based on capsule neural network. Through efficient image processing and feature extraction technology, a capsule neural network model based on ensemble learning is adopted to realize automatic evaluation of the quality of curled tobacco leaves, reduce errors caused by manual operation, improve grading efficiency and accuracy, and provide a new technical means for quality control in the tobacco industry. Summary of the invention
[0007] The present invention focuses on the problems existing in the above-mentioned prior art and provides a tobacco grading method, system and equipment based on machine vision, aiming to solve the problems of strong subjectivity, low efficiency and insufficient accuracy in the existing curled tobacco grading methods, and realize the automation and intelligentization of curled tobacco quality evaluation.
[0008] A first aspect of the present invention provides a tobacco leaf grading method based on machine vision, comprising:
[0009] Step S100: using an image acquisition device to take a high-definition photo of a natural state of a curled tobacco leaf to obtain an image of the curled tobacco leaf;
[0010] Step S200: preprocessing the acquired curled tobacco leaf image, wherein the preprocessing process is to perform the following operations after receiving the image: denoising; image enhancement; cropping and normalization;
[0011] Step S300: constructing an integrated capsule network model, wherein the integrated capsule network model includes constructing a sub-capsule network and performing integrated learning using the sub-capsule network;
[0012] Step S400: training and optimizing the integrated capsule network model;
[0013] Step S500: inputting the acquired curled tobacco leaf image into the trained integrated capsule neural network for analysis, and outputting the grading result of the curled tobacco leaf;
[0014] Step S600: Display the grading results.
[0015] Preferably, the step of using an image acquisition device to perform high-definition photography on the curled tobacco leaves to obtain high-definition images of the curled tobacco leaves comprises:
[0016] Step S110: the picture acquisition process includes: 1) trigger mechanism; 2) continuous shooting; 3) image transmission.
[0017] Step S120: The image quality control process includes: 1) real-time monitoring; 2) automatic focus; 3) exposure adjustment.
[0018] Preferably, the image preprocessing in step S200 further includes:
[0019] Step S210: Denoising, removing random noise in the image to improve the quality and clarity of the image, specifically including: Median filtering: using a median filter to process the image.
[0020] Step S220: Image enhancement processing is intended to improve the contrast and details of the image to facilitate subsequent feature extraction. The specific steps include:
[0021] Contrast stretching: stretch the grayscale value of the image to the full range (0-255) to enhance the image contrast.
[0022] Histogram equalization: By adjusting the grayscale histogram distribution of the image, the contrast of the image is enhanced.
[0023] Step S230: The purpose of cropping is to remove irrelevant background parts in the image and retain the effective area of the tobacco leaves. The specific steps include:
[0024] Automatic edge detection: Use the Canny edge detection algorithm to find the edge of the tobacco leaf. The parameters should select appropriate high and low thresholds, usually 100 and 200.
[0025] Contour extraction: According to the edge detection results, the contour of the tobacco leaf is extracted. The method is: use the findContours function in OpenCV to extract the contour.
[0026] Determine the cropping area: Determine the cropping area based on the extracted contour. The method is: calculate the circumscribed rectangle of the contour and crop out the minimum rectangular area containing the tobacco leaves.
[0027] Step S240: Normalization processing aims to normalize the image pixel values to the same range to facilitate model processing. The specific steps include:
[0028] Pixel value normalization: Normalize the image pixel value to the range of 0-1. The method is: divide each pixel value by 255;
[0029] Preferably, the step S300 of constructing a tobacco leaf grading prediction model based on a capsule neural network further comprises:
[0030] Step S310: The sub-capsule network model comprises an input layer, a convolutional layer, a main capsule layer, a dynamic routing, a capsule-like layer and a decoder:
[0031] 1) Input layer: receiving the curled tobacco leaf image, the curled tobacco leaf image is preprocessed and downsampled from the original 1080*1080 pixels to 64*64 pixels to obtain a downsampled image I;
[0032] 2) Convolution layer: Use 64 9*9 convolution kernels with a step size of 1 to generate 64 56*56 feature maps, and use the convolution kernel W to perform a convolution operation on the downsampled image I to obtain a feature map F. The feature map F is a mapping of the input image, indicating the response strength of the downsampled image I in different feature dimensions;
[0033] 3) Main capsule layer: It consists of 256 9*9 convolution kernels with a step size of 2, forming 32 capsules of size 8. Each capsule i receives the feature map F from the convolution layer and generates a set of feature vectors u i , and through the weight matrix W ij Perform affine transformation to generate prediction vector , where u i is the output of component capsule i, is the prediction of component capsule i for class capsule j;
[0034] 4) Dynamic routing: This process is the key mechanism in CapsNet, which allows the output of the main capsule to be routed to the appropriate class capsule. The dynamic routing is performed in an iterative manner. The output vector of the main capsule is used as a prediction and sent to all possible class capsules. Each class capsule adjusts the coupling weight with the main capsule according to the received prediction. The specific process of the dynamic routing algorithm is as follows:
[0035] Calculate the coupling coefficient using the softmax function ;
[0036] Among them, b ij is the log probability between component capsule i and class capsule j, b ik is the log probability between component capsule i and class capsule k;
[0037] Calculate the input vector of class capsule j ,
[0038] where c ij is the coupling coefficient between component capsule i and class capsule j, is the prediction of component capsule i for class capsule j;
[0039] 5) Capsule layer: Contains 3 capsules with a capsule size of 16. Each capsule corresponds to a type of curly tobacco leaf. The purpose of the capsules is to detect whether the specific categories they represent exist in the input image.
[0040] Apply a nonlinear activation function to obtain the final output vector v j :
[0041] ;
[0042] where s j is the input vector of class capsule j; the final output vector v of each class capsule j The length of represents the probability of the corresponding category in the input image.
[0043] 6) Decoder: It consists of three fully connected layers, containing 512, 1024 and 4096 neurons respectively. The number of neurons in the last fully connected layer is the same as the number of pixels in the input image to minimize the squared difference between the input image and the reconstructed image. The fully connected layer is used to extract the output vector v from the capsule layer. j Reconstruct the input image;
[0044] Step S320: Using the capsule network to perform ensemble learning, specifically including:
[0045] Diverse sampling of the dataset: sampling images of curly tobacco leaves;
[0046] Multi-model parallel training: Design and implement a series of sub-capsule network models, each with unique initialization parameters. Each sub-capsule network selects and optimizes the classification of different features of curly tobacco leaves:
[0047] 3) Integrated prediction mechanism: The prediction results of sub-capsule network models trained on different data sets are integrated to achieve more accurate prediction of tobacco leaf grading.
[0048] Preferably, the step S400 performs training and optimization of the integrated capsule network model, specifically including:
[0049] Step S410: Initial weight setting: In the early stage of model training, initial weights are set for the parameters in the integrated capsule neural network model. These weights are randomly generated and will be continuously adjusted during the training process;
[0050] Step S420: forward propagation process: input the preprocessed curled tobacco leaf image into the sub-capsule neural network model, calculate the final output vector of the class capsule, and use the adaboost algorithm to obtain the final category according to the output vector of each sub-capsule neural network model;
[0051] Step S430: Loss function calculation: calculating the reconstruction loss between the sub-capsule neural network model input image and the reconstructed image, and the classification loss between the predicted category of the integrated capsule network model and the real category of the tobacco leaf;
[0052] Step S440: back propagation: using the back propagation algorithm, the weights in the integrated capsule network model are updated according to the gradient information of the loss function;
[0053] Step S450: iterative training: repeat the process of forward propagation, loss function calculation and back propagation, and perform multiple rounds of iterative training;
[0054] Step S460: After the accuracy of the validation set is no longer significantly improved, the training is terminated and the final model parameters are saved.
[0055] Preferably, step S500 performs system training and optimization, and continuously adjusts weights through a dynamic routing algorithm, and the feature transfer optimization process further includes:
[0056] The step S500 inputs the acquired curled tobacco leaf image into the trained integrated capsule neural network model for analysis, and outputs the grading result of the curled tobacco leaf, specifically including:
[0057] Step S510: inputting the preprocessed curled tobacco leaf image into different sub-capsule neural network models;
[0058] Step S520: performing preliminary feature extraction on the curled tobacco leaf image through a convolution layer to generate a feature map;
[0059] Step S530: converting the feature map into component capsules, each component capsule contains vectors of multiple dimensions, which are used to represent the attributes and spatial positions of the features;
[0060] Step S540: At the capsule-like layer, the outputs of the component capsules are aggregated by dynamic routing to form a high-level feature vector;
[0061] Step S550: Analyze multiple high-level feature vectors obtained by the sub-capsule neural network model and classify the curly tobacco leaves into different grades using the adaboost algorithm.
[0062] Preferably, step S600 of displaying the grading result further includes:
[0063] According to the output results of the capsule network model, the curly tobacco leaves are classified into different grades and the grading results are displayed on the display. Users can view the grading report through the user interaction interface. The report includes the grade, feature analysis and recommended storage conditions of the curly tobacco leaves.
[0064] A second aspect of the present invention provides a tobacco grading system based on machine vision, comprising:
[0065] Image acquisition module: using an image acquisition device to take high-definition photos of the natural state of the curled tobacco leaves to obtain images of the curled tobacco leaves;
[0066] Image preprocessing module: preprocess the acquired curled tobacco leaf image. The preprocessing process is to perform the following operations after receiving the image: denoising; image enhancement; cropping and normalization;
[0067] Capsule neural network model construction module: constructing a curly tobacco leaf grading prediction model based on a capsule neural network, wherein the grading prediction model includes capsule network construction and ensemble learning using the capsule network;
[0068] Grading module: input the preprocessed curled tobacco leaf image into the trained capsule neural network for analysis, and output the grading result of the curled tobacco leaf;
[0069] Dynamic routing optimization module: performs system training and optimization, continuously adjusts weights through dynamic routing algorithms, and optimizes the feature transfer process;
[0070] Result display module: displays the grading results.
[0071] Preferably, the system further comprises a cloud computing evaluation module for grading the curled tobacco leaves;
[0072] A user interaction terminal, used for receiving and displaying the grading results of the curled tobacco leaves;
[0073] A cloud storage module, used for receiving and storing the grading results of the curled tobacco leaves;
[0074] The cloud storage module includes:
[0075] A first communication unit, used for performing data transmission with the user interaction terminal and the cloud computing evaluation module;
[0076] A distributed storage unit, used for performing distributed storage on the received data;
[0077] The data allocation management unit is used to monitor the storage status of the distributed storage unit, and to decompose the received data into tasks and allocate them to the storage devices in the distributed storage unit.
[0078] According to a third aspect of the present invention, a computer device is provided, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program implements the steps of any of the aforementioned methods when executed by the processor.
[0079] Compared with the prior art, the present invention has the following beneficial effects:
[0080] 1) Enhanced feature expression capabilities
[0081] Using capsule neural networks to classify curly tobacco leaf images can provide richer feature expressions than traditional convolutional neural networks. Capsule networks capture the spatial relationship and posture changes of objects in the image in the form of vectors, which enables the network to more accurately identify and understand the complex morphology of curly tobacco leaves.
[0082] 2) Improve classification accuracy
[0083] The dynamic routing mechanism and equivariance characteristics of the capsule network enable the model to maintain high classification accuracy when facing deformation and occlusion of curled tobacco leaf images, which is crucial to improving the accuracy of tobacco leaf grading.
[0084] 3) Improved robustness
[0085] Since the capsule network can better understand image information, it shows stronger robustness when processing curly tobacco leaf images, and can maintain stable classification performance even in the presence of poor image quality or noise.
[0086] 4) Reduce reliance on manual annotation
[0087] The adaptive nature of capsule networks allows the model to rely less on fine-grained manually labeled data during training. This can reduce the workload of data preparation and, to a certain extent, reduce the need for large amounts of labeled data.
[0088] 5) Strong adaptability
[0089] The capsule network can adapt to tobacco leaf images with different curling degrees and shapes, providing a more flexible and general method for tobacco leaf classification. This enables the model to be applied to a wider range of tobacco leaf samples, including leaves of different varieties and growing conditions.
[0090] 6) Improve the level of automation
[0091] Through the automated tobacco leaf image classification system, the production efficiency of the tobacco industry can be significantly improved, labor costs can be reduced, and 24-hour continuous operation can be achieved, which is of great significance for improving the automation level of the entire industry.
[0092] 7) Promote the intelligent development of the tobacco industry
[0093] Using capsule neural networks to classify curled tobacco leaf images is an important step in the tobacco industry's transformation towards intelligence and informatization. This can not only improve production efficiency, but also provide data support for the quality and market competitiveness of tobacco products.
[0094] 8) Environmentally friendly
[0095] The automated curled tobacco leaf grading system reduces manual operations, thereby reducing the impact on the environment, which is in line with the current trend of sustainable development and green environmental protection. The method and system of the present invention can significantly improve the efficiency and quality of tobacco leaf grading, provide an efficient and reliable quality assessment tool for the tobacco industry, and have broad application prospects.
[0096] Through the above technical solution, the integrated capsule network model of the present invention can learn and characterize the characteristics of curled tobacco leaves from multiple perspectives, improving the generalization ability and robustness of the model. The application of integrated learning enables the capsule network to provide more stable and reliable prediction results when facing complex and changeable tobacco leaf grading tasks. BRIEF DESCRIPTION OF THE DRAWINGS
[0097] Figure 1 This is a flow chart of the curled tobacco leaf grading method based on capsule neural network of the present invention;
[0098] Figure 2 This is a diagram of the capsule neural network architecture of the present invention;
[0099] Figure 3 This is a graph showing the change of capsule neural network loss and decoding loss with training rounds;
[0100] Figure 4 This is a performance comparison chart of the capsule neural network of the present invention and the convolutional neural network. DETAILED DESCRIPTION
[0101] The technology described below can be transformed in many ways and can have many embodiments, which are described in detail with reference to the accompanying drawings in specific embodiments. However, this does not mean that the technology described below is limited to specific embodiments. It should be understood that the present invention includes all similar modifications, equivalents and substitutions without departing from the spirit and technical scope of the technology described below.
[0102] Example 1
[0103] like Figure 1 As shown, the present invention provides a tobacco leaf grading method based on machine vision, comprising:
[0104] In step S100, a natural state image of the curled tobacco leaf is photographed in high definition using an image acquisition device. The steps of acquiring the curled tobacco leaf image specifically include:
[0105] Step S110: using the image acquisition device to collect the curled tobacco leaves is specifically as follows: 1) trigger mechanism: the sensor detects that the curled tobacco leaves arrive at the shooting area, triggering the camera to shoot, and the sensor is a photoelectric sensor or a laser sensor, which sends a signal to the camera after detecting the curled tobacco leaves; 2) continuous shooting: the camera is set to a continuous shooting mode, capturing the curled tobacco leaf image at a frequency of 2-5 frames per second, and the shooting frequency is adjusted according to the speed of the conveyor belt and the number of tobacco leaves to ensure that each tobacco leaf can be clearly captured; 3) image transmission: the captured image is transmitted to the processing unit through a high-speed interface. The processing unit is an industrial computer or server installed next to the production line, which is responsible for receiving and storing the curled tobacco leaf image data;
[0106] Step S120: The quality control process of the acquired tobacco leaf image is as follows: 1) Real-time monitoring: The image acquisition quality is monitored in real time through dedicated software to ensure that the clarity and exposure of each image meet the requirements. If the image is blurred or poorly exposed, the system will issue an alarm to prompt the operator to make adjustments; 2) Autofocus: The camera is equipped with an autofocus function, which can dynamically adjust the focal length according to the position of the curled tobacco leaf to ensure that each image is in the best focus. The autofocus mechanism can be achieved through a preset focus area and focus algorithm; 3) Exposure adjustment: According to changes in ambient light, the camera's exposure parameters are automatically adjusted to ensure that the image brightness is moderate. The automatic exposure function is combined with an ambient light sensor to achieve real-time adjustment.
[0107] Preferably, the image preprocessing step S200 further includes:
[0108] Step S210: Denoising, removing random noise in the image to improve the quality and clarity of the image, specifically including: median filtering: using a median filter to process the image, the median filter can effectively remove noise by sorting the pixel values in the local area and selecting the middle value as the new pixel value, window size: selecting a window size of 3*3 or 5*5;
[0109] Step S220: performing image enhancement processing. The specific steps include:
[0110] Contrast stretching: stretch the grayscale value of the image to the full range (0-255) to enhance the image contrast, including: linear stretching, mapping the minimum grayscale value of the original image to 0, the maximum grayscale value to 255, and linear interpolation of the remaining grayscale values.
[0111] Histogram equalization: The contrast of the image is enhanced by adjusting the grayscale histogram distribution of the image, calculating the grayscale histogram cumulative distribution function (CDF), and mapping the grayscale values of the original image according to the grayscale histogram cumulative distribution function;
[0112] Step S230: The specific steps of cutting include:
[0113] Automatic edge detection: Use the Canny edge detection algorithm to find the edge of the curled tobacco leaf. The parameters should select appropriate high and low thresholds.
[0114] Contour extraction: According to the edge detection results, the contour of the curled tobacco leaf is extracted using the findContours function in OpenCV;
[0115] Determine the cropping area: Determine the cropping area based on the extracted contour, calculate the circumscribed rectangle of the contour, and crop the minimum rectangular area containing the curled tobacco leaves;
[0116] Step S240: Normalization processing includes:
[0117] Pixel value normalization: Normalize the image pixel values to the range of 0-1, specifically by dividing each pixel value by 255.
[0118] Preferably, the step S300 further comprises:
[0119] like Figure 2 As shown, the step S300 constructs an integrated capsule network model, and the integrated capsule network model includes sub-capsule network construction and integrated learning using the sub-capsule network, specifically including:
[0120] Step S310: The sub-capsule network model comprises an input layer, a convolutional layer, a main capsule layer, a dynamic routing, a capsule-like layer and a decoder:
[0121] 1) Input layer: receiving the curled tobacco leaf image, the curled tobacco leaf image is preprocessed and downsampled from the original 1080*1080 pixels to 64*64 pixels to obtain a downsampled image I;
[0122] 2) Convolution layer: Use 64 9*9 convolution kernels with a step size of 1 to generate 64 56*56 feature maps, and use the convolution kernel W to perform a convolution operation on the downsampled image I to obtain a feature map F. The feature map F is a mapping of the input image, indicating the response strength of the downsampled image I in different feature dimensions;
[0123] 3) Main capsule layer: It consists of 256 9*9 convolution kernels with a step size of 2, forming 32 capsules of size 8. Each capsule i receives the feature map F from the convolution layer and generates a set of feature vectors u i , and through the weight matrix W ij Perform affine transformation to generate prediction vector , where u i is the output of component capsule i, is the prediction of component capsule i for class capsule j;
[0124] 4) Dynamic routing: This process is the key mechanism in CapsNet, which allows the output of the main capsule to be routed to the appropriate class capsule. The dynamic routing is performed in an iterative manner. The output vector of the main capsule is used as a prediction and sent to all possible class capsules. Each class capsule adjusts the coupling weight with the main capsule according to the received prediction. The specific process of the dynamic routing algorithm is as follows:
[0125] Calculate the coupling coefficient using the softmax function ;
[0126] Among them, b ijis the log probability between component capsule i and class capsule j, b ik is the log probability between component capsule i and class capsule k;
[0127] Calculate the input vector of class capsule j ,
[0128] where c ij is the coupling coefficient between component capsule i and class capsule j, is the prediction of component capsule i for class capsule j;
[0129] 5) Capsule layer: Contains 3 capsules with a capsule size of 16. Each capsule corresponds to a type of curly tobacco leaf. The purpose of the capsules is to detect whether the specific categories they represent exist in the input image.
[0130] Apply a nonlinear activation function to obtain the final output vector v j :
[0131] ;
[0132] where s j is the input vector of class capsule j; the final output vector v of each class capsule j The length of represents the probability of the corresponding category in the input image.
[0133] 6) Decoder: It consists of three fully connected layers, containing 512, 1024 and 4096 neurons respectively. The number of neurons in the last fully connected layer is the same as the number of pixels in the input image to minimize the squared difference between the input image and the reconstructed image. The fully connected layer is used to extract the output vector v from the capsule layer. j Reconstruct the input image;
[0134] Step S320: Using the capsule network to perform ensemble learning, specifically including:
[0135] 1) Diversified sampling of data sets: Sample images of curled tobacco leaves, and set different control group data to reflect the impact of different tobacco leaf features on tobacco leaf grade, such as vein color control group data, leaf texture control group data, leaf uniformity feature control group data, etc. Then, train the capsule network on different data sets to learn the impact of different features on tobacco leaf grade.
[0136] 2) Multi-model parallel training: Design and implement a series of sub-capsule network models, each with unique initialization parameters. Each sub-capsule network selects and optimizes the classification of different features of curly tobacco leaves: Sub-capsule network 1: mainly identifies the impact of the vein color of curly tobacco leaves on the grade of tobacco leaves; Sub-capsule network 2: mainly identifies the impact of the leaf thickness characteristics of curly tobacco leaves on the grade of tobacco leaves, and the leaf thickness affects the burning rate and taste of tobacco leaves; Sub-capsule network 3: mainly identifies the impact of the leaf texture of curly tobacco leaves on the grade of tobacco leaves, and the fineness of the texture and uniformity can affect the processing of tobacco leaves and the taste of the final product; sub-capsule network 4: mainly identifies the impact of the physical structure of curled tobacco leaves on the grade of tobacco leaves, such as the porosity and fiber structure of the leaves. These characteristics affect the burning characteristics of tobacco leaves; sub-capsule network 5: mainly identifies the impact of the uniformity characteristics of curled tobacco leaves on the grade of tobacco leaves. The uniformity of tobacco leaves includes the consistency of color, size and shape; sub-capsule network 6: mainly identifies the impact of the color of curled tobacco leaves on the grade of tobacco leaves. The color of tobacco leaves can reflect its maturity and quality, and the color usually ranges from light green to dark brown.
[0137] Through the setting of a series of sub-capsule network models, the integrated capsule network model can realize the recognition and extraction of multi-faceted features of curly tobacco leaves. Feature extraction can standardize the tobacco leaf evaluation process, reduce the interference of human factors, and ensure the consistency of evaluation results. In addition, the characteristics of tobacco leaves include color, texture, shape, size, moisture content, uniformity, etc. Multi-dimensional analysis can provide a more comprehensive understanding of the quality of tobacco leaves, learn and characterize the characteristics of curly tobacco leaves from multiple angles, and improve the generalization ability and robustness of the model.
[0138] In addition, because the shape and size of curly tobacco leaves may vary greatly, compared with traditional convolutional neural networks (CNNs), capsule networks show better generalization ability when facing image transformations such as rotation and scaling, which is particularly important for the grading of curly tobacco leaves. Capsule networks capture the spatial relationship and posture changes of objects in images in the form of vectors, which enables the network to more accurately identify and understand the complex morphology of curly tobacco leaves. The dynamic routing mechanism and equivariance characteristics of capsule networks enable the model to maintain high classification accuracy when facing deformation and occlusion of curly tobacco leaf images. Using capsule neural networks combined with ensemble learning to form an integrated capsule network, integrating multiple models can reduce the errors that may exist in a single model, and improve the overall grading accuracy by combining the advantages of different models. In addition, by combining multiple models, ensemble learning can reduce the risk of overfitting of a single model to training data. Different models may perform better on different data subsets. Ensemble learning can make the entire system perform well on a wider range of data and enhance generalization capabilities. Models in ensemble learning can be trained independently, which makes the training process parallelized, thereby speeding up the training process. Through the above technical solution, the integrated capsule network model of the present invention can learn and characterize the characteristics of curly tobacco leaves from multiple perspectives, improving the generalization ability and robustness of the model. The application of ensemble learning enables the capsule network to provide more stable and reliable prediction results when facing complex and changeable tobacco leaf grading tasks.
[0139] As another optional implementation, different sub-capsule network models are used to respectively identify different areas of the same curled tobacco leaf image. For example, sub-capsule network 1 is used to identify the first flat part in the curled tobacco leaf image, sub-capsule network 2 is used to identify the curled part in the curled tobacco leaf image, and sub-capsule network 3 is used to identify the second flat part in the curled tobacco leaf image. Among them, the above-mentioned curled part is located between the first flat part and the second flat part and connects the first flat part and the second flat part. Due to the spatial characteristics of the curled tobacco leaf, the spatial position, angle, and shape of different parts of it are different. Using different sub-capsule network models with unique initialization parameters to process them separately and then integrate them can optimize the recognition results to a greater extent.
[0140] Furthermore, a part of the sub-capsule network can be selected to identify and analyze the vein color, texture, thickness, etc. of the curled tobacco leaves, for example, sub-capsule networks 1, 2, and 3; another part of the sub-capsule network can be used to identify and analyze different parts of the curled tobacco leaves, such as the first flat part, the curled part, the second flat part, etc., for example, sub-capsule networks 4, 5, and 6; and the results can be integrated.
[0141] 3) Integrated prediction mechanism: The prediction results of sub-capsule network models trained on different data sets are integrated to achieve more accurate prediction of tobacco leaf grading.
[0142] Preferably, step S400 performs training and optimization of the integrated capsule network model, specifically including:
[0143] Step S410: Initial weight setting: In the early stage of model training, initial weights are set for the parameters in the integrated capsule neural network model. These weights are randomly generated and will be continuously adjusted during the training process;
[0144] Step S420: forward propagation process: input the preprocessed curled tobacco leaf image into the sub-capsule neural network model, calculate the final output vector of the class capsule, and use the adaboost algorithm to obtain the final category according to the output vector of each sub-capsule neural network model;
[0145] Step S430: Loss function calculation: calculating the reconstruction loss between the sub-capsule neural network model input image and the reconstructed image, and the classification loss between the predicted category of the integrated capsule network model and the real category of the tobacco leaf;
[0146] Step S440: back propagation: using the back propagation algorithm, the weights in the integrated capsule network model are updated according to the gradient information of the loss function;
[0147] Step S450: iterative training: repeat the process of forward propagation, loss function calculation and back propagation, and perform multiple rounds of iterative training;
[0148] Step S460: After the accuracy of the validation set is no longer significantly improved, the training is terminated and the final model parameters are saved.
[0149] Preferably, the step S500 inputs the acquired curled tobacco leaf image into the trained integrated capsule neural network model for analysis, and outputs the grading result of the curled tobacco leaf, specifically including:
[0150] Step S510: inputting the preprocessed curled tobacco leaf image into different sub-capsule neural network models;
[0151] Step S520: performing preliminary feature extraction on the curled tobacco leaf image through a convolution layer to generate a feature map;
[0152] Step S530: converting the feature map into component capsules, each component capsule contains vectors of multiple dimensions, which are used to represent the attributes and spatial positions of the features;
[0153] Step S540: At the capsule-like layer, the outputs of the component capsules are aggregated by dynamic routing to form a high-level feature vector;
[0154] Step S550: Analyze multiple high-level feature vectors obtained by the sub-capsule neural network model and classify the curly tobacco leaves into different grades using the adaboost algorithm.
[0155] Through the above steps, the system continuously optimizes the capsule neural network model to ensure the accuracy and stability of the feature transfer process, thereby improving the accuracy and reliability of tobacco leaf grading.
[0156] Preferably, step S600 of displaying the grading result further includes:
[0157] According to the output results of the capsule network model, the tobacco leaves are classified into different grades and the grading results are displayed on the display. Users can view the grading report through the user interaction interface. The report includes the grade of the tobacco leaves, feature analysis and recommended storage conditions.
[0158] Specifically, CapsNet is used for tobacco leaf grading tasks. CapsNet Loss, Decoder Loss and Total Loss are calculated after each Epoch. For example, this training is performed for 50 Epochs. Definition of loss function: CapsNet Loss: measures the classification error; Decoder Loss: measures the difference between the input image and the reconstructed image; Total Loss: the weighted sum of CapsNet Loss and Decoder Loss.
[0159] The specific analysis of the results of this experiment is as follows:
[0160] like Figure 3 As shown in the figure, as the number of Epochs increases, both the capsule network loss and the decoding loss gradually decrease, indicating that the performance of the model on the training data gradually improves;
[0161] The total loss also decreases with the increase of Epoch, which is due to the reduction of the two component losses; the reduction of CapsNet loss shows that the model has become more accurate in the classification task;
[0162] The reduction in decoding loss indicates that the model has also become more accurate in reconstructing the image, which helps the model learn more robust features.
[0163] Control experiment setup:
[0164] The performance of capsule network (CapsNet) and traditional convolutional neural network (CNN) in tobacco leaf grading task was compared. The performance indicators included training loss, validation loss, training accuracy and validation accuracy. The training was conducted for 50 epochs.
[0165] The experimental results are analyzed as follows:
[0166] like Figure 4As shown in the figure, during the training process, the losses of CapsNet and CNN are gradually decreasing, and the accuracy is gradually improving.
[0167] CapsNet has lower losses than CNN on both the training and validation sets, indicating that CapsNet performs better in fitting the training data and is more effective in generalizing to the validation data.
[0168] Both the training accuracy and validation accuracy of CapsNet are higher than those of CNN, which indicates that CapsNet has higher classification accuracy on this task.
[0169] CNN performed poorly in the early stage, but as the number of epochs increased, its performance gradually improved, but it still failed to surpass CapsNet in the end.
[0170] These results may indicate that CapsNet has advantages on visual recognition tasks due to its dynamic routing and sensitivity to spatial relations.
[0171] Example 2
[0172] The present invention also provides a tobacco leaf grading system based on machine vision, comprising:
[0173] Image acquisition module: using an image acquisition device to take high-definition photos of the natural state of the curled tobacco leaves to obtain images of the curled tobacco leaves;
[0174] Image preprocessing module: preprocess the acquired curled tobacco leaf image. The preprocessing process is to perform the following operations after receiving the image: denoising; image enhancement; cropping and normalization;
[0175] Capsule neural network model construction module: constructing a curly tobacco leaf grading prediction model based on a capsule neural network, wherein the grading prediction model includes capsule network construction and ensemble learning using the capsule network;
[0176] Grading module: input the preprocessed curled tobacco leaf image into the trained capsule neural network for analysis, and output the grading result of the curled tobacco leaf;
[0177] Dynamic routing optimization module: performs system training and optimization, continuously adjusts weights through dynamic routing algorithms, and optimizes the feature transfer process;
[0178] Result display module: displays the grading results.
[0179] Preferably, the system further comprises a cloud computing evaluation module for grading the curled tobacco leaves;
[0180] A user interaction terminal, used for receiving and displaying the grading results of the curled tobacco leaves;
[0181] A cloud storage module, used for receiving and storing the grading results of the curled tobacco leaves;
[0182] The cloud storage module includes: a first communication unit for performing data transmission with the user interaction terminal and the cloud computing evaluation module;
[0183] A distributed storage unit, used for performing distributed storage on the received data;
[0184] The data allocation management unit is used to monitor the storage status of the distributed storage unit, and to decompose the received data into tasks and allocate them to the storage devices in the distributed storage unit.
[0185] Example 3
[0186] The present invention provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the computer program implements the steps of any of the aforementioned methods when executed by the processor;
[0187] The memory may include a volatile memory (English: volatile memory), such as a random access memory (English: random-access memory, abbreviated: RAM), and the memory may also include a non-volatile memory (English: non-volatile memory), such as a flash memory (English: flash memory), a hard disk drive (English: hard disk drive, abbreviated: HDD) or a solid-state drive (English: solid-state drive, abbreviated: SSD). The memory may also include a combination of the above-mentioned types of memory;
[0188] The processor may be a central processing unit (CPU), a network processor (NP), or a combination of a CPU and a NP.
[0189] The processor may further include a hardware chip, the hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD) or a combination thereof, the PLD may be a complex programmable logic device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL) or any combination thereof;
[0190] It will be appreciated by those skilled in the art that all or some of the steps and systems in the disclosed method above may be implemented as software, firmware, hardware and appropriate combinations thereof. Some physical components or all physical components may be implemented as software executed by a processor, such as a central processing unit, a digital signal processor or a microprocessor, or may be implemented as hardware, or may be implemented as an integrated circuit, such as an application specific integrated circuit. Such software may be distributed on a computer-readable medium, which may include a computer storage medium (or a non-transitory medium) and a communication medium (or a temporary medium). As known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, disk storage or other magnetic storage devices, or any other medium that may be used to store desired information and may be accessed by a computer. Furthermore, it is well known to those skilled in the art that communication media typically embodies computer readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.
[0191] Although the present invention has been described in detail above with general description and specific embodiments, some modifications or improvements can be made on the basis of the present invention. The above description is only a preferred embodiment of the present invention and is not limited to the scope of the present invention. Other changes and modifications made by those skilled in the art without departing from the spirit and scope of protection of the present invention are still included in the scope of protection of the present invention.
Claims
1. A tobacco leaf grading method based on machine vision, characterized in that: The method comprises the following steps: Step S100: using an image acquisition device to take a high-definition photo of the natural state of the curled tobacco leaf to obtain the image of the curled tobacco leaf, specifically including: Step S110: using the image acquisition device to collect the curled tobacco leaves is specifically as follows: 1) trigger mechanism: the sensor detects that the curled tobacco leaves arrive at the shooting area, triggering the camera to shoot, and the sensor is a photoelectric sensor, which sends a signal to the camera after detecting the curled tobacco leaves; 2) continuous shooting: the camera is set to a continuous shooting mode, capturing the curled tobacco leaf image at a frequency of 2-5 frames per second, and the shooting frequency is adjusted according to the speed of the conveyor belt and the number of tobacco leaves to ensure that each curled tobacco leaf can be clearly captured; 3) image transmission: the captured image is transmitted to the processing unit through a high-speed interface. The processing unit is a server installed next to the production line, which is responsible for receiving and storing the curled tobacco leaf image data; Step S120: The quality control process of the acquired tobacco leaf image is as follows: 1) Real-time monitoring: The image acquisition quality is monitored in real time through dedicated software to ensure that the clarity and exposure of each image meet the requirements. If the image is found to be blurred or poorly exposed, the system will issue an alarm to prompt the operator to make adjustments; 2) Autofocus: The camera is equipped with an autofocus function, which can dynamically adjust the focal length according to the position of the curled tobacco leaf to ensure that each image is in the best focus. The autofocus mechanism can be achieved through a preset focus area and focus algorithm; 3) Exposure adjustment: According to changes in ambient light, the exposure parameters of the camera are automatically adjusted to ensure that the brightness of the image is moderate. The automatic exposure function is combined with an ambient light sensor to achieve real-time adjustment; Step S200: preprocessing the acquired curled tobacco leaf image, wherein the preprocessing process is to perform the following operations after receiving the image: denoising, image enhancement, cropping and normalization; Step S300: constructing an integrated capsule network model, wherein the integrated capsule network model includes constructing a sub-capsule network and performing integrated learning using the sub-capsule network, specifically including: Step S310: Construct a sub-capsule network model, including an input layer, a convolutional layer, a main capsule layer, a dynamic routing, a capsule-like layer, and a decoder: 1) Input layer: receiving the curled tobacco leaf image, the curled tobacco leaf image is pre-processed and downsampled from the original 1080x1080 pixels to 64x64 pixels to obtain a downsampled image I; 2) Convolution layer: Use 64 9x9 convolution kernels with a step size of 1 to generate 64 56x56 feature maps, and use the convolution kernel W to perform a convolution operation on the downsampled image I to obtain a feature map F. The feature map F is a mapping of the input image, indicating the response strength of the downsampled image I in different feature dimensions; 3) Main capsule layer: It consists of 256 9x9 convolution kernels with a step size of 2, forming 32 component capsules of size 8. Each component capsule i receives the feature map F from the convolution layer and generates a set of feature vectors , and through the weight matrix Perform affine transformation to generate prediction vector ,in, is the output of component capsule i, is the prediction of component capsule i for class capsule j; 4) Dynamic routing: This is done iteratively. The output vector of the main capsule is used as prediction data and sent to all class capsules. Each class capsule adjusts the coupling weight with the main capsule based on the received prediction data. The specific process is as follows: Calculate the coupling coefficient using the softmax function ; in, is the log probability between component capsule i and class capsule j, is the log probability between component capsule i and class capsule k; is the coupling coefficient between component capsule i and class capsule j; Calculate the input vector of class capsule j ; 5) Capsule layer: contains 3 capsules, the size of the capsule is 16, each capsule corresponds to a type of curly tobacco leaf, and a nonlinear activation function is applied to obtain the final output vector : ; in is the input vector of class capsule j; the final output vector of each class capsule The length of represents the probability of the corresponding category in the input image; 6) Decoder: It consists of three fully connected layers, containing 512, 1024 and 4096 neurons respectively. The number of neurons in the last fully connected layer is the same as the number of pixels in the input image to minimize the squared difference between the input image and the reconstructed image. The output vector of the capsule layer is obtained from the fully connected layer. Reconstruct the input image; Step S320: using the sub-capsule network model to perform ensemble learning, specifically including performing ensemble learning on multiple sub-capsule network models: 1) Diversified sampling of the data set: Sample images of curled tobacco leaves and set different control group data to reflect the impact of different characteristics of tobacco leaves on tobacco grade; 2) Multi-model parallel training: Design and implement a series of sub-capsule network models, each with unique initialization parameters. Use different sub-capsule network models to identify different areas of the same curled tobacco leaf image. Sub-capsule network 1 is used to identify the first flat part in the curled tobacco leaf image, sub-capsule network 2 is used to identify the curled part in the curled tobacco leaf image, and sub-capsule network 3 is used to identify the second flat part in the curled tobacco leaf image. The curled part is located between the first flat part and the second flat part and connects the first flat part and the second flat part. The spatial position, angle, and shape of different parts of the curled tobacco leaf are different. Using different sub-capsule network models with unique initialization parameters to process them separately and then integrate them can optimize the recognition results to a greater extent. 3) Integrated prediction mechanism: Integrate the prediction results of sub-capsule network models trained on different data sets; Step S400: training and optimizing the integrated capsule network model, specifically including: Step S410: Initial weight setting: In the early stage of model training, initial weights are set for the parameters in the integrated capsule network model. The initial weights are randomly generated and will be continuously adjusted during the training process; Step S420: forward propagation process: input the preprocessed curled tobacco leaf image into each sub-capsule network model, calculate the final output vector of the class capsule, and use the adaboost algorithm to obtain the final curled tobacco leaf category according to the output vector of each sub-capsule network model; Step S430: Loss function calculation: calculating the reconstruction loss between the sub-capsule network model input image and the reconstructed image, and the classification loss between the predicted category of the integrated capsule network model and the real category of the curled tobacco leaf; Step S440: back propagation: using the back propagation algorithm, the weights in the integrated capsule network model are updated according to the gradient information of the loss function; Step S450: iterative training: repeat the process of forward propagation, loss function calculation and back propagation, and perform multiple rounds of iterative training; Step S460: After the accuracy of the validation set is no longer significantly improved, the training is terminated and the final model parameters are saved; Step S500: inputting the acquired curled tobacco leaf image into the trained and optimized integrated capsule network model for analysis, and outputting the grading result of the curled tobacco leaf; Step S600: Display the grading results.
2. The method according to claim 1, characterized in that The step S200 of preprocessing the acquired curled tobacco leaf image specifically includes: Step S210: denoising, removing random noise in the curled tobacco leaf image to improve the quality and clarity of the curled tobacco leaf image, specifically, using a median filter to process the image; Step S220: image enhancement, the specific steps include: Contrast stretching: stretch the grayscale value of the curled tobacco leaf image to the full range to enhance the contrast of the curled tobacco leaf image. Specifically, the minimum grayscale value of the original curled tobacco leaf image is mapped to 0 and the maximum grayscale value is mapped to 255 through linear stretching, and the remaining grayscale values are linearly interpolated. Histogram equalization: by adjusting the grayscale histogram distribution of the curled tobacco leaf image, the contrast of the curled tobacco leaf image is enhanced, the grayscale histogram cumulative distribution function is calculated, and the grayscale value of the original curled tobacco leaf image is mapped according to the grayscale histogram cumulative distribution function; Step S230: cutting, the specific steps include: Automatic edge detection: Use the Canny edge detection algorithm to find the edges of the curled tobacco leaf image; Contour extraction: According to the edge detection results, the contour of the curled tobacco leaf is extracted using the findContours function in OpenCV; Determine the cropping area: Determine the cropping area based on the extracted contour, calculate the circumscribed rectangle of the contour, and crop the minimum rectangular area containing the curled tobacco leaves; Step S240: Normalization, the specific steps include: Pixel value normalization: Normalize the image pixel values to the range of 0-1, specifically by dividing each pixel value by 255.
3. The method according to claim 1, characterized in that The step S500 inputs the acquired curled tobacco leaf image into the trained integrated capsule network model for analysis, and outputs the grading result of the curled tobacco leaf, specifically including: Step S510: inputting the preprocessed curled tobacco leaf image into different sub-capsule network models; Step S520: performing preliminary feature extraction on the curled tobacco leaf image to generate a feature map; Step S530: converting the feature map into component capsules, each component capsule contains vectors of multiple dimensions, which are used to represent the attributes and spatial positions of the features; Step S540: Aggregate the outputs of the component capsules through dynamic routing to form a high-level feature vector; Step S550: Analyze multiple high-level feature vectors obtained from multiple sub-capsule network models, and use the adaboost algorithm to classify the curly tobacco leaves into different grades.
4. The method according to claim 1, characterized in that: The step S600 of displaying the grading result specifically includes: According to the output results of the capsule network model, the curled tobacco leaves are classified into different grades and the grading results are displayed on the monitor. The user can view the grading report through the user interaction interface. The report includes the grade, feature analysis and recommended storage conditions of the curled tobacco leaves.
5. A tobacco leaf grading system based on machine vision for implementing the method according to any one of claims 1 to 4, characterized in that: The system comprises: Image acquisition module: using image acquisition equipment to take high-definition photos of the natural state of the curled tobacco leaves to obtain images of the curled tobacco leaves; Image preprocessing module: preprocess the acquired curled tobacco leaf image. The preprocessing process is to perform the following operations after receiving the image: denoising; image enhancement; cropping and normalization; Integrated capsule network model construction module: constructing an integrated capsule network model, wherein the integrated capsule network model includes sub-capsule network construction and integrated learning using the sub-capsule network; Integrated capsule network model optimization module: training and optimization of integrated capsule network model; Grading module: input the preprocessed curled tobacco leaf image into the trained capsule network for analysis, and output the grading result of the curled tobacco leaf; Result display module: displays the grading results.
6. The system according to claim 5, characterized in that Also includes: A cloud-based computing assessment module for grading curled tobacco leaves; A user interaction terminal, used for receiving and displaying the grading results of the curled tobacco leaves; A cloud storage module, used for receiving and storing the grading results of the curled tobacco leaves; The cloud storage module includes: A first communication unit, used for performing data transmission with the user interaction terminal and the cloud computing evaluation module; A distributed storage unit, used for performing distributed storage on the received data; The data allocation management unit is used to monitor the storage status of the distributed storage unit, and to decompose the received data into tasks and allocate them to the storage devices in the distributed storage unit.
7. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 4 are implemented.
