Intelligent grading method for appearance quality of baked tobacco leaves, medium and system

Abstract

The invention relates to the technical field of quality detection of flue-cured tobacco leaves, in particular to an intelligent grading method, medium and system for appearance quality of flue-cured tobacco leaves, and the method comprises the steps: obtaining high-resolution multi-angle flue-cured tobacco leaf image data; noise in the image is removed through Gaussian filtering, and image quality is enhanced through histogram equalization; using a convolutional neural network to extract multi-level appearance features such as color, vein and integrity of the flue-cured tobacco leaves; carrying out binarization segmentation on the image, and extracting an accurate edge contour and leaf meridian features of the flue-cured tobacco leaves in combination with morphological operation; a deep convolutional neural network is utilized to fuse the multi-dimensional features, grading evaluation is carried out on the appearance quality of the cured tobacco leaves, and scores are output; the tobacco leaf local abnormal area identification method can be widely applied to quality detection in tobacco production, and has the advantages of high efficiency, intelligence and accuracy.

Description

Technical Field

[0001] The present invention relates to the technical field of quality detection of cured tobacco leaves, and in particular to an intelligent grading method, medium and system for the appearance quality of cured tobacco leaves. Background Art

[0002] During the production and processing of flue-cured tobacco leaves, the grading of the appearance quality of the flue-cured tobacco leaves has an important impact on their subsequent use value and economic benefits.

[0003] Prior art CN1 16524224A discloses a method and system for detecting the type of flue-cured tobacco leaves based on machine vision, the method comprising the following steps: (1) collecting tobacco leaf images; (2) preprocessing images; (3) extracting tobacco leaf image colors; (4) extracting tobacco leaf texture change information; (5) acquiring parameters; (6) constructing a tobacco leaf image discrimination model; and (7) entering the model into a mobile terminal-based system for discriminating the type of flue-cured tobacco leaves, discriminating the type of flue-cured tobacco leaves based on the extracted tobacco leaf image information, and proposing corresponding cause analysis and measures.

[0004] However, with the rapid development of image processing technology and artificial intelligence, intelligent grading methods based on image data have gradually become an important direction in the field of quality inspection of cured tobacco leaves. Some existing methods based on deep learning and image processing can realize automatic grading of the appearance quality of cured tobacco leaves to a certain extent, but are usually limited to simple feature extraction and cannot fully consider the multi-dimensional characteristics of cured tobacco leaves such as color, veins, and shape. It is difficult to carefully identify local abnormal areas of cured tobacco leaves. In addition, the current intelligent grading system performs poorly in dealing with complex backgrounds and image noise, which affects the accuracy of grading. Summary of the invention

[0005] The purpose of the present invention is to solve the problems in the background technology and to propose an intelligent grading method, medium and system for the appearance quality of cured tobacco leaves.

[0006] In order to achieve the above object, the present invention adopts the following technical solutions:

[0007] A method for intelligently grading the appearance quality of flue-cured tobacco leaves, the method specifically comprising the following steps:

[0008] Step 1: Obtain high-resolution multi-angle image data of cured tobacco leaves;

[0009] Step 2: Remove noise from the image through Gaussian filtering and enhance image quality through histogram equalization;

[0010] Step 3: Use a convolutional neural network to extract multi-level appearance features and feature values ​​of the cured tobacco leaves, where the appearance features include color features, vein features, and shape features;

[0011] Step 4: Binarize and segment the image, and extract the precise edge contour and meridian features of the tobacco leaves after baking by combining morphological operations;

[0012] Step 5: Use a deep convolutional neural network to fuse multi-dimensional features, grade and evaluate the appearance quality of the tobacco leaves after baking, and output a comprehensive quality score;

[0013] Step 6: Identify the local abnormal areas of the tobacco leaves after curing and perform independent scoring based on the area characteristics;

[0014] Step 7: Output intelligent grading results and generate a visual quality report.

[0015] As a further solution of the present invention, when obtaining high-resolution multi-angle image data of cured tobacco leaves, the tobacco leaf image data is collected multiple times under multi-angle and multi-light source conditions, wherein the multi-angle includes the top, the oblique side and the bottom, and the multi-light source includes a standard light source and a darkroom environment.

[0016] As a further solution of the present invention, a method for removing image noise includes:

[0017] Based on the filtering formula Remove random noise from the image, where (x, y) represents the position coordinates in the two-dimensional space, σ represents the standard deviation of the Gaussian distribution in the filter parameters, and σ is set to 1.5;

[0018] Then use the formula Perform histogram equalization on the image to enhance the brightness and contrast of the image, where h eq is the equalized image, CDF represents the cumulative distribution function, and L is the grayscale level of the image.

[0019] As a further embodiment of the present invention, a method for extracting multi-level appearance features and feature values ​​of the smoked tobacco leaves includes:

[0020] The preprocessed image is marked as input image I. Using the convolutional neural network algorithm, the input image I is first passed through the first convolution layer, and low-level features are extracted based on the formula F1=ReLU(W1*I+b1), where W1 is the convolution kernel of the first convolution layer, b1 is the bias of the first convolution layer, * represents convolution operation, ReLU is the activation function, and ReLU(a)=max(0,a), where low-level features include color and edge information;

[0021] Again based on the formula f k =ReLU(Wk *I k-1 +b k ) to obtain the eigenvalues ​​of the middle layer, where f k Indicates the specific values ​​of different features, k represents different convolutional layers, k = 1, 2, ..., p, indicating that there are p convolutional layers in total. Furthermore, each convolutional layer has specific features. When k is 1, I0 represents the preprocessed image.

[0022] Through multi-layer convolution operations, the numerical values ​​of each feature are combined into a multi-dimensional vector f = {f k |k∈[1, p]}, the feature map f at this time represents the multi-level appearance features of the tobacco leaves after baking.

[0023] As a further embodiment of the present invention, a method for extracting precise edge contours and meridian features of tobacco leaves after baking includes:

[0024] The preprocessed image is binarized using an adaptive threshold segmentation algorithm, where the threshold is determined using the maximum inter-class variance method Otsu algorithm, and the expression formula of the maximum inter-class variance method Otsu algorithm is: Among them, Var(t) is the inter-class variance, t represents the threshold, P1(t) and P2(t) represent the pixel probabilities of the two pixel levels after being segmented by the threshold t. and are the intra-class variances of the two pixel levels respectively;

[0025] After the image segmentation is completed, the binary segmented image is optimized using morphological operations, isolated noise is removed using corrosion operations, and then the complete outline of the cured tobacco leaves is restored using dilation operations. The edge outline of the cured tobacco leaves is then extracted based on the Canny edge detector. The detection formula is: E(x, y) represents the edge strength of the image, Represents the partial derivative of the grayscale function of image I in the x-axis direction, Represents the partial derivative of the grayscale function of image I in the y-axis direction.

[0026] As a further solution of the present invention, the multi-dimensional features fused by the deep convolutional neural network include:

[0027] Color characteristics: evaluate yellowing degree and color difference area;

[0028] Vein characteristics: detect the roughness and tightness of the leaf surface;

[0029] Shape characteristics: analyze the tearing and defect area of ​​the leaves;

[0030] Local abnormal features: Identify mildew spots, discolored areas, and diseased patches.

[0031] As a further solution of the present invention, a method for obtaining a comprehensive quality score and an independent score includes:

[0032] Based on the deep convolutional neural network model, the images of the cured tobacco leaves are graded and evaluated. The extracted multidimensional features f are input into the deep convolutional neural network model to evaluate the quality of the cured tobacco leaves. The model fuses the input multidimensional features through pre-trained weights and calculates the comprehensive quality score Q, where ω k represents the weight of feature k;

[0033] Based on the local convolutional network, the local features of the image are analyzed to identify the local abnormal area A = {a j |j∈[1,m]}, m means there are m local abnormal regions in the image;

[0034] Based on the formula Get the area of ​​the abnormal area Area(a j ),in, Indicates area a j The sum of all pixels (x, y) is calculated, and the corresponding value of each pixel is 1;

[0035] Then use the formula Calculate the abnormal area a j Independent rating of Qa j , Total Area represents the total area of ​​the image.

[0036] As a further solution of the present invention, after the comprehensive quality score and the independent score are obtained, multiple similarity comparisons and voting mechanisms are used, combined with feature visualization analysis from different angles, to make multiple classification decisions on the test samples, and the final tobacco leaf grading judgment is obtained through the comprehensive results of multiple voting, wherein the tobacco leaf grading includes B grade, C grade and X grade, and B grade>C grade>X grade.

[0037] A computer-readable storage medium stores program instructions, which are used to implement the above-mentioned intelligent grading method for the appearance quality of post-baked tobacco leaves when the program instructions are executed.

[0038] An intelligent grading system for the appearance quality of flue-cured tobacco leaves comprises the above-mentioned computer-readable storage medium.

[0039] Compared with the prior art, the advantages of the present invention are:

[0040] The present invention obtains multi-angle high-resolution image data of flue-cured tobacco leaves and uses a deep learning model to extract multi-dimensional features of the images, thereby realizing intelligent grading and evaluation of the appearance quality of flue-cured tobacco leaves after curing. The system combines efficient image preprocessing, precise edge detection and a comprehensive quality evaluation algorithm, and can accurately detect local anomalies on the surface of flue-cured tobacco leaves and output a detailed visual grading report. This method can significantly improve the efficiency and accuracy of flue-cured tobacco leaf quality detection, reduce subjective errors in manual operations, and is suitable for quality control and management in large-scale flue-cured tobacco leaf production. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 It is a schematic diagram of the method flow of the present invention;

[0042] Figure 2 The model's classification results for abnormal appearance features of tobacco leaves after baking;

[0043] Figure 3 Voting results for the model's predictions for the sample;

[0044] Figure 4 is the cumulative probability distribution of the sample;

[0045] Figure 5 Results of actual model runs for random samples. DETAILED DESCRIPTION

[0046] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments.

[0047] Reference Figure 1 , a method for intelligently grading the appearance quality of flue-cured tobacco leaves, the method specifically comprising the following steps:

[0048] Step 1: Use a high-precision camera device to collect images of flue-cured tobacco leaves under multiple angles and multiple light sources to obtain an image data set {I1, I2, ..., I n}, I represents an image, and n represents an image number at different angles, wherein the high-precision camera device is fixed by a fixed bracket to ensure that the camera shoots the surface of the cured tobacco leaves from different directions, thereby covering all the appearance features of the cured tobacco leaves. At the same time, in order to ensure the image quality, the resolution of the camera device in this embodiment is set to 4096×2160 pixels;

[0049] Furthermore, different orientations include top, oblique side and bottom, multiple light sources include standard light source and darkroom environment, and appearance features include main vein, branch vein, leaf edge and potential surface defects;

[0050] Step 2: Preprocess the image;

[0051] Use Gaussian filter to filter the image {I1, I2, ..., I n} to process the random noise introduced in the process. The specific processing method is the acquisition filtering formula: Wherein, (x, y) represents the position coordinates in the two-dimensional space, σ represents the standard deviation of the Gaussian distribution in the filter parameters, and in this embodiment, the value of σ is set to 1.5;

[0052] Using Gaussian filter to process the image can effectively reduce the noise interference in the image, while maintaining the clarity of the edge of the tobacco leaf after baking, thereby retaining the detailed features of the leaf edge;

[0053] Then use the formula Perform histogram equalization on the image to enhance the brightness and contrast of the image, making the color of the tobacco leaves more prominent after baking. eq is the equalized image, CDF represents the cumulative distribution function, and L is the gray level of the image;

[0054] Step 3: Extract appearance features of the preprocessed image:

[0055] The preprocessed image is marked as input image I. Using the convolutional neural network algorithm, the input image I is first passed through the first convolution layer, and low-level features are extracted based on the formula F1=ReLU(W1*I+b1), where W1 is the convolution kernel of the first convolution layer, b1 is the bias of the first convolution layer, * represents convolution operation, ReLU is the activation function, and ReLU(a)=max(0,a), where low-level features include color features and edge information;

[0056] Based on the previous convolutional layer, again based on the formula f k =ReLU(W k *I k-1 +b k ) to obtain the eigenvalues ​​of the middle layer, where f k Indicates the specific values ​​of different features, k represents different convolutional layers, k = 1, 2, ..., p, indicating that there are p convolutional layers in total. Furthermore, each convolutional layer has specific features. When k is 1, I0 represents the preprocessed image.

[0057] The middle layer of the convolutional network continues to extract higher-level features, such as the distribution of leaf veins, surface cracks, wrinkles, and spots, etc. Figure 2 Through multi-layer convolution operations, the numerical values ​​of each feature are combined into a multi-dimensional vector f = {f k |k∈[1, p]}, the feature map f at this time represents the multi-level appearance features of the tobacco leaves after baking;

[0058] In another embodiment of the present invention, different structures such as deep neural networks or twin neural networks can also be applied to comprehensively cover the diversity of appearance characteristics of tobacco leaves after baking, ensuring the refined expression of various characteristics;

[0059] Step 4: Use the adaptive threshold segmentation algorithm to perform binary segmentation on the preprocessed image. In the adaptive threshold segmentation algorithm, the threshold is determined by the maximum inter-class variance method 0tsu algorithm, and the expression formula of the maximum inter-class variance method 0tsu algorithm is: Among them, Var(t) is the inter-class variance, t represents the threshold, P1(t) and P2(t) represent the pixel probabilities of the two pixel levels after being segmented by the threshold t. and are the intra-class variances of the two pixel levels respectively;

[0060] After the image segmentation is completed, the binary segmented image is optimized using morphological operations to accurately extract the edge and vein features of the cured tobacco leaves and avoid complex background interference in the grading results. The specific optimization methods include:

[0061] First, the corrosion operation is used to remove the isolated noise, and then the expansion operation is used to restore the complete outline of the tobacco leaves after baking. The Canny edge detector is used to accurately extract the edge outline of the tobacco leaves after baking. The detection formula is: E(x, y) represents the edge strength of the image, Represents the partial derivative of the grayscale function of image I in the x-axis direction, Represents the partial derivative of the grayscale function of image I in the y-axis direction;

[0062] Step 5: Then, based on the deep convolutional neural network model, the images of the cured tobacco leaves are graded and evaluated. The extracted multidimensional features f are input into the deep convolutional neural network model to evaluate the quality of the cured tobacco leaves. The model fuses the input multidimensional features through pre-trained weights and calculates the comprehensive quality score Q, where: ω k represents the weight of feature k;

[0063] Among them, in this embodiment, the triplet loss function is cited when performing the grading evaluation to further optimize the discrimination of different category features, so that the model can be more sensitive and accurate in identifying the quality of the tobacco leaves after baking. At the same time, when performing the grading evaluation to obtain the comprehensive quality score Q, the scoring criteria cover the color characteristics, vein characteristics, shape characteristics and vein clarity, wherein the color characteristics include the yellowing degree and color difference area of ​​the tobacco leaves, the vein characteristics include the roughness and compactness of the leaf surface, and the shape characteristics include the tearing and defect area of ​​the leaf.

[0064] Step 6: After the comprehensive quality score Q is calculated, the local features of the image are analyzed based on the local convolutional network to identify the local abnormal area A = {a j |j∈[1,m]}, m means there are m local abnormal regions in the image;

[0065] Based on the formula Get the area of ​​the abnormal area Area(a j ),in, Indicates area a j The sum of all pixels (x, y) is calculated, and the corresponding value of each pixel is 1;

[0066] Then use the formula Calculate the abnormal area a j Independent rating of Qa j , Total Area represents the total area of ​​the image;

[0067] In another embodiment of the present invention, in order to ensure the accuracy of the classification results, multiple similarity comparisons and voting mechanisms are used in this embodiment, combined with feature visualization analysis from different angles, to make multiple classification decisions on the test samples, such as Figure 3 , Figure 4 and Figure 5 As shown, the system makes a final grading judgment through multiple voting results to improve the overall judgment stability and objectivity of the results. The tobacco leaf grading includes B grade, C grade and X grade, and B grade>C grade>X grade;

[0068] Step 7: Based on the comprehensive quality score Q and the independent score Qa of the abnormal area j , generate the final grading report, and transmit the grading report to the terminal display device, so that the user can intuitively view the scoring results and grading status of each piece of tobacco leaf after baking, wherein the report includes the color distribution map of the tobacco leaf after baking, texture analysis, shape integrity report and local anomaly detection results, further, the local anomaly detection results include abnormalities such as mold spots, discolored areas and diseased patches;

[0069] Example 2: Based on Example 1, this example uses the processing method in Example 1 to score and grade the cured tobacco leaves. The results are shown in the following table:

[0070] Example color Completeness Local abnormality grade Example 1 0.87 0.87 0.76 C Example 2 0.92 0.90 0.84 B Example 3 0.88 0.72 0.56 X Example 4 0.81 0.92 0.89 B Example 5 0.77 0.88 0.68 X

[0071] A second aspect of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores program instructions, and when the program instructions are executed, the program instructions are used to implement the above-mentioned intelligent grading method for the appearance quality of flue-cured tobacco leaves;

[0072] A third aspect of the present invention provides an intelligent grading system for the appearance quality of flue-cured tobacco leaves, which comprises the above-mentioned computer-readable storage medium.

[0073] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. An intelligent grading method for the appearance quality of flue-cured tobacco leaves, characterized in that: The method specifically comprises the following steps: Step 1: Obtain high-resolution multi-angle image data of cured tobacco leaves; Step 2: Remove noise from the image through Gaussian filtering and enhance image quality through histogram equalization; Step 3: Use a convolutional neural network to extract multi-level appearance features and feature values ​​of the cured tobacco leaves, where the appearance features include color features, vein features, and shape features; Step 4: Binarize and segment the image, and extract the precise edge contour and meridian features of the tobacco leaves after baking by combining morphological operations; Step 5: Use a deep convolutional neural network to fuse multi-dimensional features, grade and evaluate the appearance quality of the tobacco leaves after baking, and output a comprehensive quality score; Step 6: Identify the local abnormal areas of the tobacco leaves after curing and perform independent scoring based on the area characteristics; Step 7: Output intelligent grading results and generate a visual quality report.

2. The method for intelligent grading of the appearance quality of flue-cured tobacco leaves according to claim 1, characterized in that: When obtaining high-resolution multi-angle image data of cured tobacco leaves, the tobacco leaf image data is collected multiple times under multi-angle and multi-light source conditions, where the multi-angles include the top, the oblique side and the bottom, and the multi-light source includes a standard light source and a darkroom environment.

3. The method for intelligent grading of the appearance quality of flue-cured tobacco leaves according to claim 1, characterized in that: Methods for removing image noise include: Based on the filtering formula Remove random noise from the image, where (x, y) represents the position coordinates in the two-dimensional space, σ represents the standard deviation of the Gaussian distribution in the filter parameters, and σ is set to 1.5; Then use the formula Perform histogram equalization on the image to enhance the brightness and contrast of the image, where h eq is the equalized image, CDF represents the cumulative distribution function, and L is the grayscale level of the image.

4. The method for intelligent grading of the appearance quality of flue-cured tobacco leaves according to claim 1, characterized in that: The multi-level appearance features and feature value extraction methods of the cured tobacco leaves include: The preprocessed image is marked as input image I. Using the convolutional neural network algorithm, the input image I is first passed through the first convolution layer, and low-level features are extracted based on the formula F1=ReLU(W1*I+b1), where W1 is the convolution kernel of the first convolution layer, b1 is the bias of the first convolution layer, * represents convolution operation, ReLU is the activation function, and ReLU(a)=max(0,a), where low-level features include color and edge information; Again based on the formula f k =ReLU(W k *I k-1 +b k ) to obtain the eigenvalues ​​of the middle layer, where f k Indicates the specific values ​​of different features. k represents different convolutional layers. k = 1, 2, ..., p, indicating that there are p convolutional layers in total. Furthermore, each convolutional layer has specific features. When k is 1, I0 represents the preprocessed image. Through multi-layer convolution operations, the numerical values ​​of each feature are combined into a multi-dimensional vector f = {f k |k∈[1, p]}, the feature map f at this time represents the multi-level appearance features of the tobacco leaves after baking.

5. The method for intelligent grading of the appearance quality of flue-cured tobacco leaves according to claim 1, characterized in that: The method for extracting the precise edge contour and meridian features of the tobacco leaves after curing includes: The preprocessed image is binarized using an adaptive threshold segmentation algorithm. In the adaptive threshold segmentation algorithm, the threshold is determined using the maximum inter-class variance method Otsu algorithm, and the expression formula of the maximum inter-class variance method Otsu algorithm is: Among them, Var(t) is the inter-class variance, t represents the threshold, P1(t) and P2(t) represent the pixel probabilities of the two pixel levels after being segmented by the threshold t. and are the intra-class variances of the two pixel levels respectively; After the image segmentation is completed, the binary segmented image is optimized using morphological operations, isolated noise is removed using corrosion operations, and then the complete outline of the cured tobacco leaves is restored using dilation operations. The edge outline of the cured tobacco leaves is then extracted based on the Canny edge detector. The detection formula is: E(x, y) represents the edge strength of the image. Represents the partial derivative of the grayscale function of image I in the x-axis direction, Represents the partial derivative of the grayscale function of image I in the y-axis direction.

6. The method for intelligent grading of the appearance quality of flue-cured tobacco leaves according to claim 1, characterized in that: The multi-dimensional features of deep convolutional neural network fusion include: Color characteristics: evaluate yellowing degree and color difference area; Vein characteristics: detect the roughness and tightness of the leaf surface; Shape characteristics: analyze the tearing and defect area of ​​the leaves; Local abnormal features: Identify mildew spots, discolored areas, and diseased patches.

7. The method for intelligently grading the appearance quality of flue-cured tobacco leaves according to claim 4, characterized in that: The methods for obtaining the comprehensive quality score and independent score include: Based on the deep convolutional neural network model, the images of the cured tobacco leaves are graded and evaluated. The extracted multidimensional features f are input into the deep convolutional neural network model to evaluate the quality of the cured tobacco leaves. The model fuses the input multidimensional features through pre-trained weights and calculates the comprehensive quality score Q, where ω k represents the weight of feature k; Based on the local convolutional network, the local features of the image are analyzed to identify the local abnormal area A = {a j |j∈[1,m]}, m means there are m local abnormal regions in the image; Based on the formula Get the area of ​​the abnormal area Area(a j ),in, Indicates area a j The sum of all pixels (x, y) is calculated, and the corresponding value of each pixel is 1; Then use the formula Calculate the abnormal area a j Independent rating of μa j , Total Area represents the total area of ​​the image.

8. The method for intelligently grading the appearance quality of flue-cured tobacco leaves according to claim 1, characterized in that: After the comprehensive quality score and independent score are obtained, multiple similarity comparisons and voting mechanisms are used, combined with feature visualization analysis from different angles, to make multiple classification decisions on the test samples. The final tobacco leaf grading judgment is obtained through the comprehensive results of multiple voting. The tobacco leaf grading includes B grade, C grade and X grade, and B grade>C grade>X grade.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores program instructions, which are used to implement the intelligent grading method for the appearance quality of flue-cured tobacco leaves as described in any one of claims 1 to 8 when the program instructions are executed.

10. An intelligent grading system for the appearance quality of flue-cured tobacco leaves, characterized in that: A computer-readable storage medium comprising the computer-readable storage medium of claim 9.

Images