A method for detecting tobacco shag impurities based on machine vision

Abstract

The application discloses a tobacco impurity detection method based on machine vision, which comprises seven steps: 1, establishing an impurity detection data set and a table module; 2, dividing the detection data set into batches; 3, color preprocessing through the table module; 4, after the image processed by the table module is transmitted into the main network for feature extraction; 5, the m feature maps in each batch are further extracted through step-by-step stacking residual blocks; 6, the feature maps are added with attention mechanisms for multiple times, and the feature maps with the same number of channels are obtained through convolution block processing for multiple times; 7, the model loss is calculated to help the model to carry out back propagation, so that the model parameters are updated, and the model most suitable for the tobacco impurity detection task is trained. The method can better extract the feature information in the image and improve the recognition ability of the model to the tobacco impurity; the channel attention mechanism is added, the network can selectively focus on the key information in the image, and the recognition ability of the model to the tobacco impurity is further improved; and the table is constructed, which provides a usable research basis for researchers in the field.

Description

Technical Field

[0001] This invention relates to a machine vision-based method for detecting impurities in tobacco shreds. Background Technology

[0002] The tobacco processing stage is a crucial part of cigarette manufacturing. During this process, various impurities often enter the tobacco mixture through various pathways, significantly impacting the quality of the tobacco product. These impurities are classified into three categories: Category A impurities include rubber, plastic fibers, animal feathers, and metal objects. Their combustion produces unpleasant odors, causing discomfort to smokers and even significant harm, severely affecting the quality of the tobacco product. Category B impurities include paper scraps, rope ends, burlap shavings, and stones, which have a significant impact on tobacco quality. Category C impurities refer to plant and non-tobacco plant fibers, which have a minor impact on tobacco products.

[0003] Traditional methods of impurity removal involve a large workforce on the production line for sorting. While simple, this method is time-consuming, labor-intensive, has low accuracy, and high industrial costs. Most existing tobacco impurity removal systems rely on color information for foreign object detection, which is ineffective at identifying impurities that are the same or similar in color to the tobacco. Therefore, there is an urgent need for a robust tobacco impurity detection model to upgrade and replace existing tobacco impurity removal systems. Summary of the Invention

[0004] To address the shortcomings of existing technologies, the present invention aims to calculate the loss function of the entire network (i.e., the error between the predicted and true values) by outputting predicted values ​​through forward propagation, and then performing backpropagation to automatically update the massive parameters in the network, thereby making the predicted values ​​closer to the true values. This enables accurate identification of impurities in the tobacco production process with a faster detection speed.

[0005] This invention proposes a machine vision-based method for detecting impurities in tobacco shreds, comprising the following steps:

[0006] Step 1: Establish an impurity detection dataset and a lookup table module. The impurity detection dataset is divided into a training set, a validation set, and a test set, with a data volume ratio of 8:1:1. The lookup table module maps the R, G, and B values ​​of each pixel in the image using a lookup table method and saves the mapped values ​​in a table. The lookup table mapping formula is as follows:

[0007] Value = f(R, G, B) = 255 2 R+255G+B;

[0008] f(·) is the mapped value of the pixel obtained after mapping using the lookup table method; a color image has three channels, R, G, and B, which represent the values ​​of each pixel in the R, G, and B channels, respectively.

[0009] Step 2, N in the training set t The data is randomly and evenly divided into t batches, and each batch contains m data. Similarly, the data in the validation set and the test set are also evenly divided into t batches.

[0010] Step 3: When a batch of data is input, it first undergoes color preprocessing through the lookup table module. After processing, the color information of impurities in the image will be significantly different from normal tobacco and the background, making it easier for the backbone network to extract the color features of impurities.

[0011] When the mapping value of a certain pixel in the image cannot be found in the lookup table, the pixel is considered to be an impurity pixel. At the same time, the values ​​of R, G, and B are changed to 0, 0, and 255 respectively, which means that the color of the pixel is changed to blue (b), making the impurity pixel more obvious.

[0012] Step 4: The image processed by the lookup table module is then fed into the backbone network for feature extraction.

[0013] Step 5: Extract feature information from m feature maps contained in each batch by stacking residual blocks in stages.

[0014] Step 6: Add SENet attention mechanisms to the feature maps obtained in Step 5. Name the feature maps after adding attention mechanisms 3, 4, and 5 times as C3, C4, and C5, respectively, and output them from the backbone network. Process C5 through convolutional blocks to further extract information, and then process it in three ways: ① Directly use the feature map P5 formed by processing C5 through convolutional blocks to output the detection result using the detector head; ② Perform downsampling to deepen the network, and then extract features through convolutional blocks to output feature map P6. On the one hand, input P6 into the detector head to output the detection result, and on the other hand, continue downsampling. After feature extraction and channel number adjustment via convolutional blocks, feature map P7 is obtained, and the detection head outputs the detection result; ③ Feature fusion is performed through a three-dimensional convolution operation, then an upsampling layer doubles the size of the feature map, and it is concatenated with C4 to achieve feature fusion. Then, feature extraction and channel number adjustment via convolutional blocks yield feature map P4, which is input into the detection head to output the detection result; P4 undergoes another three-dimensional convolution operation, upsampling, and concatenation with C3. Then, feature fusion and channel number adjustment via convolutional blocks yield P3, which is input into the detection head for detection;

[0015] At this point, the dimensions of P3, P4, P5, P6, and P7 are respectively... and Together they form the Feature Pyramid (FPN) structure;

[0016] Step 7: Calculating the model loss helps the model perform backpropagation, thereby updating the model parameters and training the model best suited for the tobacco impurity detection task. Backpropagation uses gradient descent to calculate the gradient of parameter changes and finally calculates the updated weights.

[0017] Preferably, Step 4 involves inputting an image of size H×W×3 into the Focus module for downsampling, thus changing the image dimensions. Where W represents the number of pixels in the image length, H represents the number of pixels in the image width, and 3 represents the number of channels in the image. After processing by the Focus module, the number of channels becomes 64. This process involves three steps:

[0018] Step 4-1: The convolution kernel slides across the image until it has traversed the entire image, forming a feature map. The convolution operation can be represented as:

[0019]

[0020]

[0021] Where X represents the feature map output after the convolution operation, and W... in H represents the number of pixels in the length of the input image. in C represents the number of pixels in width. in k is the number of input channels. i This represents the i-th convolutional kernel of size k. This represents the value at position (x, y) of the j-th channel of the i-th convolutional kernel, where s and p are the stride and padding number, respectively, and a... j，x，y This represents the value at position (x, y) in the j-th channel of the input feature map. conv represents the convolution operation, and concat represents the concatenation operation.

[0022] Step 4-2, Batch standardization, the specific formula is:

[0023] Input: B={x1, x2,...x m};γ,β

[0024] Ouput:{y i =BN γ，β (x i )}

[0025]

[0026]

[0027]

[0028]

[0029] Where B represents the input batch; x1, x2...x m The corresponding m feature maps in the batch; γ and β are parameters that can be obtained through training; y i This represents the feature map output after batch normalization; BN represents the batch normalization process, ε is a very small parameter designed to avoid a denominator of 0; γ and β are the parameters obtained during training;

[0030] Step 4-3: Add a non-linear factor to the feature map using the SiLU activation function. The expression is:

[0031]

[0032] Where x represents the value of a pixel in each channel, and e is the base of the natural logarithm function.

[0033] Preferably, Step 5 includes two steps:

[0034] Step 5-1: Each batch contains m feature maps that enter the residual module. First, a one-dimensional convolution operation is performed to reduce the number of channels to 32 to reduce the computational cost of subsequent three-dimensional convolution operations. Then, batch normalization and SiLU activation function are applied. A three-dimensional convolution kernel is used for convolution operation. To keep the size of the feature map constant, a padding ring is added around the outside of the feature map. After the convolution operation, the number of channels is expanded to 64. Then, the feature map that has been processed by batch normalization and SiLU activation function is added to the input feature map to achieve feature fusion.

[0035] Step 5-2: The feature map after residual block fusion is processed by the downsampling module to reduce the size of the feature map to half of the input size, while doubling the number of channels. This includes two steps:

[0036] Step 5-2-1: The feature map is processed by the attention mechanism, and different weights are assigned to each channel so that the network can focus on and extract key information.

[0037] Step 5-2-2: Perform a depthwise separable convolution operation on the feature map using a 3D convolution kernel with a stride of 2 and padding of 1, making the feature map dimension... The depthwise separable convolution consists of two steps: the first step is to perform convolution operations on different channels of the feature map using single-channel convolution kernels with the same number of channels as the input feature map; the second step is to perform one-dimensional convolution operations on the feature map to achieve point-by-point operations and change the number of channels of the feature map.

[0038] Preferably, Step 6 includes two steps:

[0039] Step 6-1: The convolutional block consists of 5 convolution operations superimposed on the input feature map to further extract feature information. After the feature map is processed by the convolutional block, the size of the feature map remains unchanged, but the number of channels changes. After the first one-dimensional convolution operation, the number of channels is reduced to half of the input. After the three-dimensional convolution operation, the number of channels is restored to the same as the input. The next two convolution operations have the same effect as the previous two. The last convolution operation changes the number of channels to 256 to share the detection head.

[0040] Step 6-2: Detection is performed using an anchorless bounding box method. When the feature map enters the detection head, it will be divided into two directions: one is the classification branch, which performs classification operations; the other includes a regression branch and a centrality branch.

[0041] Preferably, in Step 7, after each batch of data completes one forward propagation, the loss is calculated, and then the parameters are updated via back propagation. The formula for calculating the model loss (Loss) is as follows:

[0042]

[0043] Where m is the number of feature maps in each batch, and y i This represents a binary label of 1 or 0. In this task, 1 is considered impurity, and 0 is considered non-impurity; p(y i ) indicates that the prediction belongs to y i The score of the label; generally speaking, for a label of 1, if the predicted value is close to 1, then the loss function value is close to 0, and vice versa.

[0044]

[0045] Where θ represents the network parameters, h θ x i x represents i The score output by the sigmoid function;

[0046] The expression for Loss should be:

[0047]

[0048]

[0049] Find the derivative of the parameter θ:

[0050]

[0051] The beneficial effects of this invention are:

[0052] (1) The color information-based preprocessing module is effectively combined with the deep learning target detection algorithm. First, the color information preprocessing module is used to preprocess the data, and the processed image data is used to train the model. The backbone network of the deep learning target detection algorithm can better extract the feature information in the image and improve the model's ability to identify tobacco impurities.

[0053] (2) Add a channel attention mechanism before the region generation network of the traditional two-stage target detection algorithm. The network can selectively focus on key information in the image, thereby improving the model's ability to identify tobacco impurities.

[0054] (3) A dataset of tobacco impurity removal and a reference table required in the color information preprocessing module were collected and constructed, providing a usable research foundation for researchers in this field. Attached image description:

[0055] Figure 1 Here is a schematic diagram of the process of this invention;

[0056] Figure 2 Here is a flowchart illustrating the implementation process of the reference table module.

[0057] Figure 3 For example: Focus module diagram;

[0058] Figure 4 Here is the structure diagram of the residual module;

[0059] Figure 5 Here is a diagram of the downsampling layer structure.

[0060] Figure 6 Here is a diagram of the SEnet network structure.

[0061] Figure 7 For example: Convolutional block structure diagram;

[0062] Figure 8 For example: Schematic diagram of the detection head;

[0063] x, y, z, w, q, b represent the colors of the pixel;

[0064] The eye pattern in the downsampling layer represents the attention mechanism. Detailed Implementation

[0065] The present invention will now be described in further detail with reference to specific embodiments.

[0066] Step 1: Establish an impurity detection dataset and a reference table. The impurity detection dataset is divided into a training set, a validation set, and a test set, with a data volume ratio of 8:1:1. The training set is used to perform gradient descent on the training error to learn the weight parameters. The validation set is used to initially evaluate the model's capabilities and adjust the hyperparameters. The test set is used to evaluate the generalization ability of the final model.

[0067] Step 2, N in the training set t The data is randomly and evenly divided into t batches, with each batch containing m data items; similarly, the data in the validation set and test set are also evenly divided into t batches. Subsequently, the model will be trained and the parameters will be updated sequentially in batches. Training all batches is considered one iteration.

[0068] Step 3: When a batch of data is input, it first undergoes color preprocessing through the lookup table module. After processing, the color information of impurities in the image will be significantly different from normal tobacco and the background, making it easier for the backbone network to extract the color features of impurities.

[0069] The lookup table module first maps the R, G, and B values ​​of each pixel in the image using a lookup table method, and saves the mapped values ​​in a table. After deduplication and ascending sorting, a lookup table is formed.

[0070] The lookup table mapping formula is:

[0071] Value = f(R, G, B) = 255 2 R+255G+B

[0072] f(·) is the mapping value of the pixel obtained after mapping using the lookup table method; a color image has three channels, R, G, and B, which represent the values ​​of each pixel in the R, G, and B channels, respectively. When the mapping value of a certain pixel in the image cannot be found in the lookup table, the pixel is considered to be an impurity pixel. At the same time, the values ​​of R, G, and B are changed to 0, 0, and 255, respectively, that is, the color of the pixel is changed to blue b. In this way, the pixel classified as an impurity pixel becomes more obvious.

[0073] Step 4: The image processed by the lookup table module is fed into the backbone network for feature extraction; when an image of size H×W×3 enters the Focus module for downsampling, the image dimensions become...

[0074] W represents the number of pixels in the image length;

[0075] H represents the width of the image in pixels;

[0076] 3 represents the number of channels in the image. After processing by the Focus module, the number of channels becomes 64.

[0077] The data enters the Focus module and is first split into four parts using a slicing method. The feature maps are stitched together to become The feature map is then processed; then, a one-dimensional convolution operation is performed, that is, a convolution operation is performed with a convolution kernel of size 1×1 to expand the number of channels to 64. In this way, the spatial information of the image is transformed into the channel dimension, which achieves downsampling and reduces information loss. Next, a batch standardization operation is performed to make the data follow a standard normal distribution, the purpose of which is to solve the internal covariate shift (ICS) problem. Finally, a nonlinear transformation is performed through the SiLU activation function to increase the complexity of the model and improve the expressive power of the network.

[0078] 1) Convolution is an effective feature extraction method in image processing. A convolution kernel slides across the image and performs corresponding convolution operations at different positions, resulting in corresponding outputs, until the entire image is traversed to form a feature map. Figure 3 The convolution operation in the code can be represented as:

[0079]

[0080] Where X represents the feature map output after the convolution operation, and W... in H represents the number of pixels in the length of the input image. in C represents the number of pixels in width. in k is the number of input channels. i This represents the i-th convolutional kernel of size k. This represents the value at position (x, y) of the j-th channel of the i-th convolutional kernel, where s and p are the stride and padding number, respectively, and a... j，x，y This represents the value at position (x, y) in the j-th channel of the input feature map. conv represents the convolution operation, and concat represents the concatenation operation.

[0081] Convolution operations essentially multiply and sum the values ​​of the input data at corresponding positions on different channels using the values ​​on the convolution kernel. This summation is achieved by sliding the kernel across the feature map, generating a new one-dimensional feature map. The concatenation operation combines the multiple feature maps generated by the convolution operation into a multi-channel feature map. Here, a convolution operation with a kernel size of 1 does not change the size of the feature map; its effect is to change the number of channels. The output... Feature map;

[0082] 2) Batch normalization is often applied before activation layers in deep neural networks. Its functions include: accelerating convergence during model training, making the training process more stable, preventing gradient explosion or vanishing, and thus providing a degree of regularization. Its main calculation formula is:

[0083] Input: B={x1, x2,...x m};γ,β

[0084] Ouput:{y i =BN γ，β (x i )}

[0085]

[0086]

[0087]

[0088]

[0089] Where B represents the input batch; x1, x2...x m The corresponding m feature maps in the batch; γ and β are parameters that can be obtained through training; y i This represents the feature map output after batch normalization; BN represents the batch normalization process, which can be specifically represented as: ① First, input a batch containing m feature maps, then calculate the mean μ of the m feature maps for each channel. B and variance ① Ensure that the data from each channel follows a standard normal distribution; ② Perform data standardization to obtain... It is worth noting that a very small parameter ε is added to the denominator to avoid the denominator being 0; ③ Combining the parameters γ and β obtained from training, the final y is calculated. i ;

[0090] 3) After batch standardization, the feature maps follow a standard normal distribution in each channel. Then, the SiLU activation function is used to add a non-linear factor to the feature maps, improving the algorithm's expressive power. The expression for the SiLU activation function is:

[0091]

[0092] Where x represents the value of a pixel in each channel. After passing through the activation function, the value of each pixel in each channel will undergo a corresponding nonlinear transformation. The SiLU activation function has no upper bound, which can avoid overfitting. It has a lower bound, which can produce a better regularization effect. Its smoothness makes it differentiable everywhere. x < 0 makes it non-monotonic, which is of great significance to the distribution.

[0093] Step 5: After processing by the Focus module, each batch contains m images. The feature map is further extracted by stacking residual blocks in steps. The number of residual blocks stacked in each step is 1, 2, 8, 8, 4 respectively. After the first 4 steps of stacking residual blocks, a downsampling layer is used to reduce the feature map size and increase the number of channels.

[0094] 1) When m sheets The feature map enters the residual module. First, it undergoes a one-dimensional convolution operation to reduce the number of channels to 32 to reduce the computational cost of subsequent three-dimensional convolution operations. Then, it is processed by batch normalization and SiLU activation function. A three-dimensional convolution kernel is used for convolution operation. In order to keep the size of the feature map unchanged, a ring needs to be padded around the outside of the feature map. After the convolution operation, the number of channels is expanded to 64. Then, the feature map after batch normalization and SiLU activation function processing is added to the input feature map to achieve feature fusion.

[0095] 2) The feature map after residual block fusion is processed by a downsampling module, reducing the feature map size to half the input size and doubling the number of channels. The implementation process is as follows:

[0096] ① The eye pattern in the downsampling layer represents the attention mechanism. The input feature map is processed by the Squeeze-and-Excitation Networks (SENet) attention mechanism, where different weights are assigned to each channel, allowing the network to focus on and extract key information. Its network structure is as follows: Figure 6 As shown:

[0097] For a feature map of dimension w×h×c, it is first compressed into a 1×1×c feature map using global average pooling. The compressed one-dimensional parameters then become the global receptive field. Next, two fully connected layers are applied consecutively to fit the complex correlations between channels and output weights with the same number of channels as the input features. Then, the weights are normalized using a sigmoid activation function to obtain new weights. Finally, a reconstruction operation applies the new weights to the corresponding channels of the feature map, updating the feature map and focusing on key information. It is worth noting that the input feature map here has a dimension of w×h×c.

[0098] ② Apply a 3D convolution kernel to the feature map after the attention mechanism is applied, performing a depthwise separable convolution operation with a stride of 2 and padding of 1, thus making the feature map dimension […]. Depthwise separable convolution consists of two steps: First, single-channel convolution kernels with the same number of channels as the input feature map are used to perform convolution operations on different channels of the feature map. Here, 64 different single-channel convolution kernels are used. By traversing the entire image, 64 single-channel feature maps are obtained. Then, these 64 feature maps are concatenated to form a 64-channel feature map. Second, a one-dimensional convolution operation is performed on the feature map to achieve pointwise computation and change the number of channels. The operation process of depthwise separable convolution can be represented as:

[0099]

[0100] X represents the feature map output after the convolution operation, W in H represents the number of pixels in the length of the input image. in C represents the number of pixels in width. in k is the number of input channels. i This represents the i-th convolutional kernel of size k. This represents the value at position (x, y) of the i-th channel of the i-th convolutional kernel, where s and p are the stride and padding number, respectively, and a... i，x，y This represents the value at position (x, y) in the i-th channel of the input feature map. `conv` represents the convolution operation, and `concat` represents the concatenation operation. Compared to ordinary convolution, depthwise separable convolution can reduce the number of parameters and computational cost. The feature map after the depthwise separable convolution operation is batch normalized and activated by the SiLU activation function, and then fed into the subsequent network. By continuously stacking residual blocks and downsampling operations, the network is deepened, enabling it to extract more useful information.

[0101] Step 6: After the backbone network finishes extracting information, the feature maps downsampled 3, 4, and 5 times are output respectively. SENet attention mechanisms are then added to each of these maps to allow subsequent networks to better focus on key information. For ease of explanation, the feature maps after adding the attention mechanism are named C3, C4, and C5 respectively. At this point, the dimension of C3 is... The dimension of C4 is C5's dimensions become C5 is processed through a convolutional block to further extract information, and then further processed in three directions: ① The output feature map P5 is directly used by the detection head to output the detection result; ② Downsampling is performed to deepen the network, and then features are extracted again through a convolutional block to output P6. P6 is then input into the detection head to output the detection result, and further downsampling is performed, followed by feature extraction and channel number adjustment through a convolutional block before outputting the detection result; ③ Feature fusion is performed through a 3D convolution operation, followed by an upsampling layer to double the feature map size, and then concatenated with C4 to achieve feature fusion. Features are then extracted and the channel number is adjusted through a convolutional block to obtain P4, which is input into the detection head to output the detection result. P4 undergoes another 3D convolution operation, upsampling is performed, and it is concatenated with C3. Features are then fused and the channel number is adjusted through a convolutional block to obtain P3, which is then input into the detection head for detection. It is worth noting that the dimensions of P3, P4, P5, P6, and P7 are respectively... and Together, they form the Feature Pyramid (FPN) structure, with the aim of sharing detection heads for multi-scale detection. The specific implementation process is as follows:

[0102] 1) A convolutional block is actually five convolution operations superimposed on the input feature map to further extract feature information.

[0103] After the feature map is processed by the convolutional block, the size of the feature map remains unchanged, but the number of channels changes. First, after the one-dimensional convolution operation, the number of channels is reduced to half of the input, the purpose of which is to reduce the amount of computation; then, after the three-dimensional convolution operation, the number of channels is restored to the same as the input; the next two convolution operations have the same effect as the previous two; the last convolution operation changes the number of channels to 256, so as to share the detection head.

[0104] 2) This invention employs an anchorless detection method. When the feature map enters the detection head, it splits into two directions: a classification branch for classification and a regression and centrality branch. First, both directions undergo feature fusion through four sets of convolutional layers. These convolutional layers include 3D convolution, group normalization, and activation functions, without altering the feature map's dimensions. Then, the classification branch performs a 3D convolution operation, outputting a w×h×1 classification result, where w and h represent the length and width of the input feature map, respectively, and 1 indicates a score for a point on the feature map being detected as an impurity. The regression and centrality branches are similar to the classification branch, outputting detection results with dimensions w×h×4 and w×h×1, respectively. In the regression branch, 4 represents the distance from the detection point to the left (l), right (r), top (t), and bottom (b) of the target. The centrality branch outputs the centrality value, which measures the distance of the detection point from the center.

[0105] Suppose that the coordinates of a point on the feature map mapped back to the original image are (c x c y If ), then the coordinates of the corresponding target box are:

[0106] x min =c x -l·s,y min =c y -t·s

[0107] x max =c x -r·s,y max =c y -b·s

[0108] Where s represents the scaling ratio relative to the original image, for example, the scaling ratio of P5 is 32. l, r, t, and b represent the distances of the detection point to the left, right, top, and bottom of the feature map, respectively, and x min x max y min y max The four values ​​represent the distances of the detection point to the left, right, top, and bottom in the original image dimensions. These four values ​​can form a predicted detection box. Centrality reflects the distance of a point on the feature map from the center of the target; its value is between 0 and 1, with a larger value indicating a closer distance to the center. The formula for calculating centrality is:

[0109]

[0110] Among them, l * r * , t * and b * These represent the distances from the detected point to the ground truth bounding box, with `min` representing the minimum value and `max` representing the maximum value. Before calculating the loss, positive and negative samples need to be determined. A point on the feature map that falls within the center region of a ground truth bounding box is considered a positive sample, and vice versa. When a point on the feature map falls into the intersection of multiple ground truth bounding boxes, it is assigned to the ground truth bounding box with the smallest area by default. However, this does not completely solve the problem of disputed samples. The FPN structure can predict targets of different scales based on the feature map, separating overlapping targets. Combined with the center region, this further reduces the problem of disputed samples.

[0111]

[0112] For the point at (x, y) in the feature map, p x，y t x，y and s x，yThese represent the predicted class scores, bounding box information, and centrality parameters calculated by the centrality branch, respectively. and These represent the actual category label, bounding box information, and centrality parameter, respectively. This indicates the classification of a sample as positive or negative; a value of 1 indicates a positive sample, and a value of 0 indicates a negative sample. The classification and centrality branches use the binary cross-entropy loss function, while the regression branch uses giou_loss.

[0113] Step 7: Calculating the model loss helps the model perform backpropagation, thereby updating the model parameters and training the model best suited for the tobacco impurity detection task. Backpropagation uses gradient descent to calculate the gradient of parameter changes, ultimately calculating the updated weights. In the proposed algorithm, the initial parameters are initialized. After each batch of data completes one forward propagation, the loss is calculated, followed by backpropagation to update the parameters. Taking the binary cross-entropy loss function as an example:

[0114]

[0115] Where m is the number of samples, y i This represents a binary label of 1 or 0. In this task, 1 is considered impurity, and 0 is considered non-impurity; p(y i ) indicates that the prediction belongs to y i The score of the label. Generally speaking, for a label of 1, if the predicted value is close to 1, then the loss function value is close to 0, and vice versa.

[0116]

[0117] Where θ represents the network parameters, h θ x i x represents i The score output by the sigmoid function. The expression for Loss should be:

[0118]

[0119] Find the derivative of the parameter θ:

[0120]

[0121] Anchor-free object detection differs from anchor-bound methods. Since it doesn't require pre-defined detection boxes, it reduces the number of parameters and avoids the computational burden of Intersection over Union (IOU), thus improving detection speed. The introduction of a feature pyramid structure increases the number of predicted boxes and reduces box overlap, enabling the model to perform multi-scale detection. Furthermore, the introduction of centrality effectively suppresses low-quality detection boxes, thereby improving the model's detection accuracy. Experiments show that the proposed algorithm achieves an accuracy of over 99.50% in detecting tobacco impurities, demonstrating its ability to accurately identify impurities in the tobacco production process.

[0122] The above description is only a part of the specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the protection scope of the present invention.

Claims

1. A method for detecting impurities in tobacco shreds based on machine vision, characterized in that... It includes the following 7 steps: Step 1: Establish the impurity detection dataset and lookup table module. The impurity detection dataset is divided into a training set, a validation set, and a test set, with a data volume ratio of 8:1:

1. The lookup table module is based on each pixel of the image. R,G,B The values ​​are mapped using a lookup table method, and the mapped values ​​are saved in a table. The formula for the lookup table mapping method is: ; The mapped value of this pixel is obtained after mapping using a lookup table method; a color image has three channels. R,G,B Each pixel represents the position of the pixel. R,G,B The values ​​on the three channels; Step 2, Training Set N t The data is randomly divided into t batches, and each batch contains... m Similarly, the data in the validation set and the test set are also divided into t batches; Step 3: When a batch of data is input, it first undergoes color preprocessing via the lookup table module. During color preprocessing, if the mapping value of a pixel in the image cannot be found in the lookup table, the pixel is considered an impurity pixel, and the R, G, and B values ​​are changed to 0, 0, and 255 respectively, effectively changing the color of that pixel to blue. b， This makes the color of the impurities clearly distinguishable from the normal tobacco and the background color; Step 4: The image processed by the lookup table module is then fed into the backbone network for feature extraction. Step 5: Each batch contains m feature maps. Further feature information is extracted by stacking residual blocks in stages. Step 6: Add SENet attention mechanisms to the feature maps obtained in Step 5. Name the feature maps after adding attention mechanisms 3, 4, and 5 times as C3, C4, and C5, respectively, and output them from the backbone network. After C5 is processed by a convolutional block, it is further processed in three directions: ① The feature map P5 formed by C5 after convolutional block processing is directly used as the detection head to output the detection result; ② Downsampling is performed to deepen the network, and then features are extracted again through a convolutional block to output feature map P6. On the one hand, P6 is input into the detection head to output the detection result, and on the other hand, downsampling continues. After feature extraction and channel number adjustment via convolutional blocks, feature map P7 is obtained, and the detection head outputs the detection result; ③ Feature fusion is performed through a three-dimensional convolution operation, then an upsampling layer doubles the size of the feature map, and it is concatenated with C4 to achieve feature fusion. Then, feature extraction and channel number adjustment via convolutional blocks yield feature map P4, which is input into the detection head to output the detection result; P4 undergoes another three-dimensional convolution operation, upsampling, and concatenation with C3. Then, feature fusion and channel number adjustment via convolutional blocks yield P3, which is input into the detection head for detection; At this point, the dimensions of P3, P4, P5, P6, and P7 are respectively: , , , and Together, they form the Feature Pyramid (FPN) structure; Step 7: Calculate the model loss to help the model perform backpropagation, thereby updating the model parameters and training a model suitable for the tobacco impurity detection task; backpropagation is to use gradient descent to calculate the gradient of parameter changes and finally calculate the updated weights. In Step 6, the convolutional block processing involves five convolutional operations superimposed on the input feature map to further extract feature information. After the convolutional block processing, the size of the feature map remains unchanged, but the number of channels changes. First, after the one-dimensional convolutional operation, the number of channels is reduced to half of the input number. After the three-dimensional convolutional operation, the number of channels is restored to the same as the input number. The next two convolutional operations have the same effect as the previous two. The last convolutional operation changes the number of channels to 256 to share the detection head. In Step 6, the detection is performed using an anchorless method. When the feature map enters the detection head, it is divided into two directions: one is a classification branch, which performs classification operations; the other includes a regression branch and a centrality branch.

2. The method for detecting tobacco impurities based on machine vision according to claim 1, characterized in that, Step 4 involves adjusting the size to... H × W The ×3 image is then fed into the Focus module for downsampling, and the image dimensions become: ;in, W The number of pixels representing the length of the image. H The number of pixels represents the image width, and 3 represents the number of image channels. After processing by the Focus module, the number of channels becomes 64. This process involves three steps: Step 4-1: The convolution kernel slides across the image until it has traversed the entire image, forming a feature map. The convolution operation is represented as follows: ； in X This represents the feature map output after the convolution operation. W in This indicates the number of pixels in the input image. H in Indicates the width in pixels. C in Input the number of channels. Indicates the first i The size is k convolution kernel, Indicates the first i The first convolutional kernel j One channel ( x , y The value at the position, s , p Separate step size and padding number, This indicates that the input feature map is in the j-th channel ( x , y The value at the position, conv This represents the convolution operation. concat For splicing operations; Step 4-2, Batch standardization, the specific formula is: ； in, B Indicates input batch ; correspond batch In m Each feature map; and These are parameters obtained through training; This represents the feature map output after batch standardization. BN This indicates the process of batch standardization. The parameter is extremely small, designed to avoid the denominator being... 0 ; and These are the parameters obtained during training; Step4-3, use SiLU The activation function adds a non-linear factor to the feature map, and its expression is: ； in x This represents the value of a pixel in each channel, where e is the base of the natural logarithm function.

3. The method for detecting tobacco impurities based on machine vision according to claim 1, characterized in that, Step 5 includes two steps: Step 5-1: Each batch contains m feature maps that enter the residual module. First, a one-dimensional convolution operation reduces the number of channels to 32 to decrease the computational cost of subsequent three-dimensional convolution operations. Then, batch normalization and... SiLU Activation function processing; convolution operation is performed using a 3D convolution kernel. To maintain the size of the feature map, a padding ring is needed around the outside of the feature map. After the convolution operation, the number of channels is expanded to 64. Then, the features are sequentially batch-normalized and... SiLU The feature map processed by the activation function is added to the input feature map to achieve feature fusion; Step 5-2: The feature map after residual block fusion is processed by the downsampling module to reduce the size of the feature map to half of the input size, while doubling the number of channels. This includes two steps: Step 5-2-1: The feature map is processed by the attention mechanism, and different weights are assigned to each channel so that the network can focus on and extract key information. Step 5-2-2: Perform a depthwise separable convolution operation on the feature map using a 3D convolution kernel with a stride of 2 and padding of 1, making the feature map dimension... The depth-separable convolution consists of two steps: the first step is to perform convolution operations on different channels of the feature map using single-channel convolution kernels with the same number of channels as the input feature map; the second step is to perform one-dimensional convolution operations on the feature map to achieve point-by-point operations and change the number of channels of the feature map.

4. The method for detecting tobacco impurities based on machine vision according to claim 1, characterized in that, Each of the steps in Step 7 batch After the data completes one forward propagation, the loss is calculated, and then the parameters are updated via backpropagation, resulting in the model loss. Loss The calculation formula is as follows: ; in, m The number of feature maps included in each batch. This indicates a binary label of 1 or 0, where 1 indicates an impurity and 0 indicates a non-impurity. Indicates that the prediction belongs to The score of the label; for the case of label 1, if the predicted value is close to 1, then the loss function value is close to 0, and vice versa. ; in For network parameters, express through sigmoid The score output by the function; Loss The expression should be: ； Calculate the corresponding parameters The derivative: 。

Images