Model training method for tobacco impurity detection, detection method, storage medium and equipment

Abstract

The invention relates to a model training method for tobacco impurity detection, a detection method, a storage medium and equipment. According to the method, training optimization is carried out and an image layer-by-layer processing model is constructed based on a multi-layer self-improved Transform model, so that layer-by-layer processing can be effectively carried out on images in the tobacco production process, a region of interest is reduced layer by layer, optimal configuration of computing resources is realized, the requirement on hardware computing power is reduced, and a high-resolution tobacco impurity detection method is further developed; the impurity detection precision is improved, the requirement for computing resources is remarkably reduced, and the method is more suitable for the actual production environment.

Description

Technical Field

[0001] The present invention belongs to the field of artificial intelligence technology and relates to a model training method, a detection method, a storage medium and a device for tobacco impurity detection. Background Art

[0002] In the tobacco production process, tobacco leaves are often mixed with various impurities, such as rubber, plastic, nylon, foam and other non-plant impurities. If the impurities are not removed, they will be cut into shreds along with the tobacco leaves after passing through the leaf shredder, making it more difficult for operators to identify the impurities. The presence of impurities not only affects product quality, but also affects the brand's market competitiveness. Therefore, it is crucial to accurately and efficiently detect impurities in tobacco.

[0003] The current impurity sorting process includes manual identification of impurities and machine image recognition. Manual identification of impurities is inefficient and affects product quality. Machine image recognition is gradually being promoted and applied. For example, CN115393667A discloses a model training method, a target detection method, a device and a storage medium. The training method includes: constructing a target detection model for detecting tobacco and impurities mixed in tobacco; obtaining tobacco pictures and annotating the tobacco pictures to generate training samples; creating a model training task and converting the training samples into a model file; using the model file to train the target detection model to be trained; determining the detection loss of the target detection model, adjusting the target detection model based on the detection loss, and being able to intelligently identify and locate impurities, thereby improving the efficiency and accuracy of detecting impurities in tobacco and the stability of identifying impurities. CN112189877A provides an online detection method for tobacco impurities in a tobacco production line, which includes two parts: data dimension reduction and tobacco impurity detection. The specific implementation steps are: data dimension reduction is to select the band of tobacco spectral image data through an improved optimal index method to achieve data dimension reduction, and the tobacco spectral image data is de-noised, sampled and grouped three times in sequence before calculating the optimal index, so as to select a band subset with a large amount of information, a small correlation between bands and a small number; tobacco impurity detection combines a deep convolutional neural network with a neighborhood de-scattering algorithm, firstly performs preliminary recognition of each point of the spectral image to be recognized through a trained convolutional neural network model to obtain a classified image; then, the classified impurities are re-recognized through the neighborhood de-scattering algorithm to eliminate the misclassified scatter points in the impurities in the classified image.

[0004] With the development of image acquisition equipment, high-resolution images provide a basis for more accurate impurity detection, but also put forward higher requirements on the computing power of detection equipment. Traditional target detection algorithms often find it difficult to balance computing efficiency and detection accuracy when faced with high-resolution images. Therefore, how to develop high-resolution image processing methods with high efficiency and low computing power requirements to achieve accurate impurity detection is one of the problems that urgently need to be solved in the field of tobacco impurity detection. Summary of the invention

[0005] In view of the deficiencies in the prior art and actual needs, the present invention provides a model training method, a detection method, a storage medium and a device for tobacco debris detection, and develops a high-precision and high-efficiency tobacco debris detection method based on a deep learning target detection algorithm, which improves the detection accuracy while reducing the demand for computing resources.

[0006] To achieve this object, the present invention adopts the following technical solutions:

[0007] In a first aspect, the present invention provides a method for training a layer-by-layer image processing model, the method comprising:

[0008] Obtain a tobacco image dataset with impurity annotations, wherein each image in the tobacco image dataset contains an annotated impurity region and a corresponding impurity category label, and use the tobacco image dataset as a training set; perform standardization on the images in the training set, train an improved Transformer model, and obtain an image layer-by-layer processing model, wherein the training includes single-layer training and multi-layer joint training;

[0009] The single-layer training includes:

[0010] First-layer training: input the images in the training set and the corresponding impurity region annotations for training, and use the loss function to calculate the first-layer region proposal loss L RPN and classification loss L cls , calculate the first layer comprehensive loss L L1 , L L1 =L RPN +α1·L cls , where α1 is a coefficient used to weigh the two parts of the loss; Second layer training: input the image obtained from the first layer training and the corresponding impurity area annotation for training, and use the loss function to calculate the second layer refinement loss L refine , region proposal loss L RPN and classification loss L cls , calculate the second layer comprehensive loss L L2 , L L2 =L RPN +α2·L cls +β2·L refine, where α2 and β2 are weight coefficients; Third layer training: input the image obtained from the second layer training and the corresponding impurity area annotation for training, and use the loss function to calculate the final detection loss L of the third layer final , classification loss L cls and the refinement loss L refine , calculate the third layer comprehensive loss L L3 , L L3 =L final +α3·L cls +β3·L refine , where α3 and β3 are trade-off coefficients; the parameters of the model are adjusted according to the comprehensive loss of each layer until the comprehensive loss is lower than the preset threshold;

[0011] The multi-layer joint training includes calculating the feedback consistency loss L feedback and the global optimization loss L global , calculate the total loss L total , L total =γ1·L L1 +γ2·L L2 +γ3·L L3 +λ·L feedback +δ·L global , where γ1, γ2, and γ3 represent the weights of the losses of each layer, λ and δ represent the weights of feedback consistency and global optimization, respectively. The parameters of the model are adjusted according to the total loss until the total loss is lower than the preset threshold.

[0012] In the present invention, based on the multi-layer self-improving transformer model, training optimization is carried out to construct a layer-by-layer image processing model, which can effectively process the tobacco production process image layer by layer, reduce the focus area layer by layer, achieve optimal configuration of computing resources, and reduce the requirements for hardware computing power.

[0013] Preferably, the loss function for training each layer includes any one of a cross entropy loss function, an IoU loss function, a Focal Loss loss function or a Dice loss function.

[0014] Preferably, the training also includes a model fine-tuning step, specifically including calculating the fine-tuning loss L finetune , L finetune =L final +δ·L global , adjust the parameters of the model according to the fine-tuning loss until the fine-tuning loss is lower than the preset threshold.

[0015] In the present invention, the impurities include: plant impurities: such as grass, wood blocks, etc.; man-made materials: such as plastic, waste paper, thread, cloth, etc.; animal-derived impurities: such as feathers, animal hair, etc.; production process residues: such as hemp rope for bundling, cardboard or wrapping paper for packaging materials, etc.; others: such as impurities generated by molds or other foreign matter in the industrial process.

[0016] In a second aspect, the present invention provides an image layer-by-layer processing model, which is trained by the image layer-by-layer processing model training method described in the first aspect.

[0017] In a third aspect, the present invention provides a method for layer-by-layer processing of tobacco production process images, the method for layer-by-layer processing of tobacco production process images comprising using the layer-by-layer image processing model described in the second aspect to perform layer-by-layer processing on the tobacco production process images to obtain a target image area;

[0018] The layer-by-layer processing includes:

[0019] The first level of processing: macro-region screening, input tobacco production process image I0, image resolution R x ×R y , downsample the image by the scaling factor s1 to obtain the reduced image I1, whose resolution is Impurity detection is performed on I1 to locate the target area that may contain impurities. According to the formula I1=Downsample(I0,s1), the area is marked with coordinate information and passed as input to the second layer of processing; the second layer of processing: mesoscopic region refinement is performed, the reduced image I1 and the target area obtained from the first layer of processing are input, the target area marked in I1 is enlarged with a magnification factor of r2, and a medium-resolution image I2 is obtained. According to the formula I2=ZoomIn(I1,r2), feature extraction and impurity detection are performed on the enlarged area, and the target area is marked; the third layer of processing: microscopic region depth detection is performed, the image I2 and the target area obtained from the second layer of processing are input, the target area is enlarged with a magnification factor of r3, a high-resolution regional image is obtained, and impurity detection is performed on this basis. According to the formula I3=ZoomIn(I2,r3), the precise position and category of the detected impurities are output to generate the final detection result.

[0020] In the present invention, a layer-by-layer processing method is designed, and the task of each layer is to gradually narrow the image area that needs to be focused on, so as to reduce the consumption of computing resources and improve the detection accuracy.

[0021] Preferably, the method for layer-by-layer processing of tobacco production process images includes a step of feedback optimization, and the formula is: in, is the image representation processed by the kth layer, σ is the feedback weight coefficient, is the feedback information from the k+1th layer processing.

[0022] In the present invention, in each layer, the model optimizes the detection results of the previous layer through a backward feedback mechanism. Specifically, the detection results of the high-resolution layer will be fed back to the previous layer to improve the selection of its region of interest, thereby improving the detection accuracy layer by layer.

[0023] In a fourth aspect, the present invention provides a method for detecting foreign matter in tobacco, the method for detecting foreign matter in tobacco comprising the following steps:

[0024] (1) Acquire tobacco images from an image acquisition device of a tobacco production line and perform preprocessing;

[0025] (2) processing the image obtained in step (1) using the method for layer-by-layer processing of tobacco production process images described in the third aspect;

[0026] (3) Use a machine learning algorithm to perform impurity detection on the image processed in step (2) and output the detection result.

[0027] Preferably, the machine learning algorithm comprises an algorithm based on a convolutional neural network (CNN) or a transformer, such as YOLO or Faster R-CNN.

[0028] Preferably, the pretreatment in step (1) comprises:

[0029] (1-1) Denoising: Use a filter (such as Gaussian filter) to denoise the image and reduce the random noise in the image;

[0030] (1-2) Contrast enhancement: Enhance the contrast of the image through methods such as histogram equalization to improve the distinction between impurities and background;

[0031] (1-3) Image normalization: Normalize the pixel values ​​of the image so that the brightness and contrast of the image remain within a reasonable range, reducing the impact of lighting changes on detection.

[0032] Preferably, the impurity detection in step (3) comprises:

[0033] (3-1) Feature extraction: Extract multi-scale feature maps from the layered image to detect impurities of different sizes;

[0034] (3-2) Classification and localization: Classify and localize impurities through target detection heads (such as YOLO or Faster R-CNN);

[0035] (3-3) Output: The test results include the category c of the impurity i and location information (xi ,y i ,w i ,h i ); Target detection formula: Output = {(c i ,(x i ,y i ,w i ,h i ))}, where c i Indicates the impurity category, (x i ,y i ,w i ,h i ) represents the coordinates and size of the detection box.

[0036] In a fifth aspect, the present invention provides a device for detecting foreign matter in tobacco, the device being used to execute the method for detecting foreign matter in tobacco described in the fourth aspect, comprising an image acquisition and preprocessing module, a layered processing module, an impurity detection module and a result output module; the image acquisition and preprocessing module being used to execute the steps of acquiring tobacco images from an image acquisition device of a tobacco production line and preprocessing the images; the layered processing module being used to execute the steps of processing the images of the tobacco production process layer by layer described in the third aspect on the images obtained in step (1); the impurity detection module being used to execute the steps of performing impurity detection on the images processed by the layered processing module using a machine learning algorithm; and the result output module being used to execute the steps of outputting the results of the impurity detection module.

[0037] In a sixth aspect, the present invention provides an electronic device, comprising: at least one processor, and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method for detecting impurities in tobacco described in the fourth aspect.

[0038] In a seventh aspect, the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein the program, when executed by a processor, implements the method for detecting foreign matter in tobacco described in the fourth aspect.

[0039] Compared with the prior art, the present invention has at least the following beneficial effects:

[0040] The present invention proposes a high-resolution tobacco debris detection method based on a multi-layer self-improving Transformer. The method processes images at multiple levels, gradually reduces the area of ​​interest, achieves optimal configuration of computing resources, reduces the requirements for hardware computing power, can efficiently process high-resolution images, and achieve accurate impurity detection. DETAILED DESCRIPTION

[0041] The technical solution of the present invention is further described below by specific implementation methods. However, the following examples are only simplified examples of the present invention and do not represent or limit the scope of protection of the present invention. The scope of protection of the present invention shall be subject to the claims.

[0042] If no specific techniques or conditions are specified in the examples, the techniques or conditions described in the literature in the field or the product instructions are used. If no manufacturer is specified for the reagents or instruments used, they are all conventional products that can be purchased through regular channels.

[0043] The present invention utilizes a multi-layer self-improving Transformer model and designs a layer-by-layer processing model for training images to reduce the consumption of computing resources and improve detection accuracy.

[0044] The training process of the multi-layer self-improving Transformer is implemented through a multi-stage training strategy, where each layer is trained separately and then end-to-end joint fine-tuning is performed to ensure the coordination between the layers. During the training process, a large-scale annotated dataset is used (the image data comes from the production line of a tobacco factory, such as using a 3CCD industrial camera for data collection), and the model is optimized through objective functions such as cross entropy loss and IoU loss.

[0045] Ultimately, the multi-layer self-improving Transformer can perform efficient impurity detection on high-resolution tobacco images, significantly improving the accuracy and efficiency of detection while reducing the demand for computing resources.

[0046] The training process of the multi-layer self-improving Transformer involves multiple stages, from single-layer training to multi-layer joint training and finally fine-tuning. During the training process, the loss function may be different at different stages to ensure that each layer can effectively learn its target and optimize the performance of the entire model in joint training. The following is a detailed training method and loss function design.

[0047] 1. Training preparation

[0048] 1.1 Dataset Preparation

[0049] Labeled dataset: A high-resolution tobacco image dataset with impurity annotations is used. Each image in the dataset contains an annotated impurity region (such as a rectangular box) and the corresponding impurity category label.

[0050] Data preprocessing: Standardize the image (such as normalization, data augmentation, etc.) to improve the generalization ability of the model.

[0051] 1.2 Model Initialization

[0052] Parameter initialization: All transformer and convolution module parameters in the model are initialized using standard initialization methods such as Xavier initialization.

[0053] Optimizer selection: Use Adam or SGD as the optimizer. The learning rate may be adjusted during training based on the performance of the validation set.

[0054] 2. Single layer training

[0055] 2.1 Objective: Train the L1 layer to perform preliminary region proposals on downsampled low-resolution images.

[0056] Input: low-resolution image I1 and corresponding target region annotations.

[0057] Loss function: The loss function of the L1 layer mainly consists of two parts: region proposal loss L RPN : Measures the difference between the region proposals generated by the model and the true annotations. Cross entropy loss or IoU (Intersection over Union) loss can be used. Classification loss L cls : Used to measure the accuracy of preliminary classification results, usually using cross entropy loss. The comprehensive loss function is expressed as: L L1 =L RPN +α1·L cls Among them, α1 is a coefficient used to weigh the two parts of the loss. If a larger α1 is set: the model pays more attention to the accuracy of region proposal (that is, the generated candidate region has a higher IoU with the real target); if a smaller α1 is set: the model pays more attention to classification accuracy (that is, the target category judgment of the candidate region is more accurate). The initial value of α1 is preset to 1.0, that is, the weights of region proposal and classification tasks are equal, and fine-tuning is performed according to the performance of the validation set. For example: if the accuracy (IoU) of region proposal is low, α1 can be gradually increased (such as increasing by 0.1 each time); if the classification accuracy is low, α1 can be gradually reduced (such as reducing by 0.1 each time). α1 is usually selected in the range of 0.5 to 2.0.

[0058] Training strategy: Use stochastic gradient descent (SGD) or Adam optimizer for optimization, and train until the loss function no longer decreases significantly on the validation set. For example, if the early stopping strategy is used, when the validation set loss function decreases by less than the set threshold of 0.01% for n consecutive training epochs, the model is considered to have converged and training is stopped.

[0059] 2.2 Training of the Second Layer (L2)

[0060] Objective: Train the L2 layer to perform fine-grained detection on the enlarged medium-resolution image.

[0061] Input: The enlarged region image I2 and the corresponding object annotation passed from layer L1.

[0062] Loss function: In addition to considering region proposal and classification, the loss function of the L2 layer also needs to consider the refined detection of the enlarged area. refine : Used to optimize the refined detection results. You can use a more sophisticated IoU loss or Focal Loss (suitable for dealing with category imbalance). Region proposal loss L RPN and classification loss L cls : Same as L1 layer. The comprehensive loss function is expressed as: L L2 =L RPN +α2·L cls +β2·L refine , where α2 and β2 are weight coefficients. α2 is used to weigh the classification loss L L2 The weight of α2 is as follows: a larger α2 indicates a higher focus on classification accuracy; a smaller α2 reduces the impact of classification loss and pays more attention to the effects of region proposal and refined detection. The optional range is 0.5 to 2.0. β2 is used to weigh the refined loss L refine The weight of β2 is as follows: a larger β2 indicates a higher focus on refined detection (such as IoU or Focal Loss of bounding boxes); a smaller β2 reduces the impact of refined loss and emphasizes the accuracy of classification or region proposals. The optional range is 1 to 5.0. The specific value can be adjusted dynamically through experiments: the initial setting is α2 = 1.0 and β2 = 2.0. During the training process, observe the main indicators of the validation set (such as mAP and Recall), and gradually adjust the two parameters. If the classification accuracy is low, α2 can be increased; if the bounding box positioning accuracy is low, β2 can be increased.

[0063] Training strategy: Gradually increase the magnification factor r2 and train images with different magnification ratios to ensure that the model can handle images of different scales.

[0064] 2.3 Training of the third layer (L3)

[0065] Objective: Train the L3 layer to perform final depth detection on key areas of high-resolution images.

[0066] Input: High-resolution region image I3 and object annotations passed from layer L2.

[0067] Loss function: The loss function of the L3 layer focuses on the final classification and detection accuracy. The final detection loss L final : Used to measure the accuracy of the final detection results, usually using more advanced loss functions, such as Dice loss, IoU loss, FocalLoss, etc. The comprehensive loss function is expressed as: L L3 =Lfinal +α3·L cls +β3·L refine , where α3 and β3 are trade-off coefficients. α3 is used to adjust the classification loss L cls The hyperparameter of the weight, a larger α3 indicates that the classification task is more important; a smaller α3 reduces the impact of the classification task, allowing more computing resources to be focused on bounding box refinement or optimization of the final detection results. The optional range is: 0.5~2.0. β3 is used to adjust the refinement loss L refine The hyperparameter of the weight; a larger β3 indicates a high focus on bounding box refinement, which is suitable for tasks with high requirements for target boundary positioning; a smaller β3 pays more attention to the performance of other parts (such as classification tasks or overall optimization of the final detection), and the optional range is 1 to 5.0. The initial values ​​are set to α3 = 1.0 and β3 = 3.0, and are dynamically adjusted according to the classification accuracy of the validation set and the bounding box detection accuracy. If the classification accuracy (Precision) is lower than 80%, increase α3 appropriately; if the bounding box IoU is lower than 0.75, increase β3 appropriately.

[0068] Training strategy: While minimizing the loss function, the feedback path is optimized to ensure that the detection results of each layer can be effectively fed back to the previous layer.

[0069] 3. Multi-layer joint training

[0070] After each layer is trained individually, multiple layers are trained jointly to optimize the overall performance of the entire multi-layer self-improving Transformer.

[0071] 3.1 Loss Function Design

[0072] Objective: The goal of joint training is to improve the comprehensive detection performance of the model through global optimization.

[0073] The total loss function L total :L total =γ1·L L1 +γ2·L L2 +γ3·L L3, where γ1, γ2, and γ3 represent the weights of the losses of each layer. γ1, γ2, and γ3 control the contribution weights of the losses of the first, second, and third layers to the total loss function, respectively. By adjusting these weights, the learning tasks of different layers can be balanced, such as the priority of region proposal, refined detection, and final classification. The value ranges of γ1, γ2, and γ3 are independently 0.5 to 3.0. Initial recommended values: γ1 = 1.0, γ2 = 1.0, γ3 = 1.0, indicating that the initial weights of the losses of each layer are equal. If the performance of a layer is significantly lower than expected (such as poor region proposals in the first layer), the weight of the layer can be appropriately increased (such as increasing γ1 to 1.5 or 2.0). If the computational overhead of a layer is high or it contributes less to the final result, the weight of the layer can be reduced. In joint training, in addition to considering the individual losses of each layer, the following items need to be introduced: feedback consistency loss L feedback :Ensure that feedback information between layers can be transmitted consistently and effectively to avoid inconsistencies between previous and next layers. Global optimization loss L global : It is used to globally optimize the entire model to ensure that the final output detection results have the best accuracy. The final comprehensive loss function is expressed as: L total =γ1·L L1 +γ2·L L2 +γ3·L L3 +λ·L feedback +δ·L global Among them, λ and δ represent the weights of feedback consistency and global optimization respectively. λ is used to control the feedback consistency loss L feedback Weight, δ is used to control the global optimization loss L global The weights of , these two parameters are mainly used to balance the weights between local (hierarchical loss) and global (cross-layer optimization) objectives. λ ranges from 0.1 to 1.0, and the initial recommended value is: λ = 0.5, indicating that feedback consistency has a moderate impact. If the consistency of the outputs of the previous and next layers is poor (such as high cross-layer error), it can be increased to 0.7 or 1.0. δ ranges from 1.0 to 5.0, and the initial recommended value is: δ = 2.0, indicating the importance of global optimization to the final performance; if the model performs poorly in global performance (such as mAP), it can be increased to 3.0 or 5.0.

[0074] 3.2 Joint Training Strategy

[0075] Phased optimization: In the early stage of training, the loss function of each layer is optimized independently to ensure that each layer can converge stably. In the later stage of training, feedback consistency and global optimization items are gradually introduced.

[0076] Learning rate adjustment: Use a learning rate decay strategy (such as cosine annealing or exponential decay) to gradually reduce the learning rate at different stages of training to avoid overfitting.

[0077] 4. Final fine-tuning

[0078] 4.1 Fine-tuning strategy

[0079] After the joint training is completed, the entire multi-layer self-improving Transformer is fine-tuned to further improve the performance of the model on specific tasks.

[0080] Loss function: In the fine-tuning stage, usually only the final detection loss L is considered final and the global optimization loss L global .

[0081] L finetune =L final +δ·L global

[0082] Strategy: During fine-tuning, you can use a smaller learning rate and perform frequent performance evaluations on the validation set to ensure the final accuracy of the model.

[0083] The present invention further develops an efficient tobacco impurity detection method based on the trained image layer-by-layer processing model, and ultimately accurately identifies the impurity position in the high-resolution image through layer-by-layer processing and optimization of the image.

[0084] 1. Image acquisition and preprocessing

[0085] Obtain high-resolution tobacco images from image acquisition devices in tobacco production lines. Image acquisition devices can be high-resolution industrial cameras, with resolutions typically ranging from millions to tens of millions of pixels to ensure that minute impurities in tobacco can be captured.

[0086] The preprocessing step includes the following key steps:

[0087] Denoising: Use filters (such as Gaussian filtering) to denoise the image and reduce random noise in the image;

[0088] Contrast enhancement: Enhance the contrast of the image through methods such as histogram equalization to improve the distinction between impurities and background;

[0089] Image normalization: Normalize the pixel values ​​of the image to keep the brightness and contrast of the image within a reasonable range and reduce the impact of lighting changes on detection.

[0090] The preprocessed image is denoted as I0, with a resolution of R x ×R y .

[0091] 2. Layered processing

[0092] The task of each layer is to gradually narrow the image area that needs to be focused on to reduce the consumption of computing resources and improve detection accuracy.

[0093] First level (L1): Macro-region screening

[0094] Input: High-resolution image I0.

[0095] Processing: Downsample I0 by the scaling factor s1 to obtain the reduced image I1, whose resolution is s1 controls the downsampling ratio of the high-resolution image I0 in the first layer (L1), which is used for the preliminary screening of macroscopic areas. The purpose is to reduce the image resolution to reduce computational overhead while ensuring that the region of interest (ROI) is still clearly visible in the downsampled image. The value range is 4 to 10. A smaller s1 (such as 4 to 6) is suitable for scenes that need to retain more details and is suitable for complex backgrounds or small impurities. A larger s1 (such as 8 to 10) is suitable for scenes with simple image backgrounds and large targets, which can significantly reduce computational complexity.

[0096] Output: A rough impurity detection is performed on I1 to locate regions that may contain impurities (region proposals). These regions are marked with coordinate information and passed as input to the next layer.

[0097] The formula means:

[0098] I1=Downsample(I0,s1)

[0099] Here, s1 is the scaling factor.

[0100] Second level (L2): Meso-region refinement

[0101] Input: The downscaled image I1 and the region of interest (ROI) obtained from the L1 layer.

[0102] Processing: Enlarge the region of interest marked in I1 by a factor of r2 to obtain a medium-resolution image I2. r2 controls the magnification ratio of the region of interest (ROI) in the second layer (L2) and is used for refined detection of the mesoscopic area. The purpose is to enlarge the ROI to extract more detailed features and improve the accuracy of classification and detection. The value range is 2 to 5. A smaller r2 (such as 2 to 3) is suitable for situations with less background noise and clearer target boundaries; a larger r2 (such as 4 to 5) is suitable for scenes with smaller targets or unclear boundaries.

[0103] Output: More detailed feature extraction and impurity detection of the enlarged area, marking more precise regions of interest.

[0104] The formula means:

[0105] I2=ZoomIn(I1,r2)

[0106] Where r2 is the amplification factor.

[0107] The third layer (L3): Micro-area depth detection

[0108] Input: The refined region image I2 output by the L2 layer.

[0109] Processing: Further zoom in on the region of interest with a zoom factor of r3 to obtain a high-resolution regional image I3, and perform the most detailed impurity detection on this basis. r3 is a factor used in the third layer (L3) to further zoom in on the refined region to achieve high-resolution detection. The purpose is to obtain higher detail information in the local range, thereby improving the accuracy of detection. It is especially suitable for small targets or blurred boundaries. The value range is 2 to 6. Smaller r3 (such as 2 to 3): The zoom ratio is moderate, suitable for situations where the resolution is already high or the target boundary is clear, can reduce the computational complexity, and is suitable for scenes with high real-time requirements. Larger r3 (such as 4 to 6): The zoom ratio is larger, which can capture more details, suitable for situations where the target is small or the boundary is blurred, may increase the computational overhead, but improve the detection accuracy.

[0110] Output: The precise location and category of the detected impurities, generating the final test results.

[0111] The formula means:

[0112] I3=ZoomIn(I2,r3)

[0113] Among them, r3 is the amplification factor.

[0114] In each layer, the model optimizes the detection results of the previous layer through a backward feedback mechanism. Specifically, the detection results of the high-resolution layer are fed back to the previous layer to improve its selection of regions of interest, thereby improving the detection accuracy layer by layer.

[0115] Feedback optimization formula:

[0116]

[0117] in, is the image representation of the kth layer, σ is the feedback weight coefficient, is the feedback information from the k+1th layer. σ is the weight coefficient that controls the influence of feedback optimization, which determines the degree of adjustment of the image representation of the low-resolution layer (k) by the feedback information Feedback from the high-resolution layer (k+1). A larger σ makes the feedback have a stronger influence on the optimization of the current layer; a smaller σ reduces the effect of feedback on the adjustment and relies more on the initial calculation of the current layer. The value range is 0.1 to 1.0. A smaller σ (such as 0.1 to 0.3): It is suitable for scenarios with low quality of feedback information (such as poor detection accuracy of the high-resolution layer), reducing the interference of feedback on the decision of the current layer; a larger σ (such as 0.5 to 1.0): It is suitable for scenarios where the feedback information is reliable and significantly helps the optimization of the current layer, and increases the influence of feedback on the selection of the region of interest during the adjustment process. Initial recommended value: σ = 0.5: It means that the influence of feedback and current layer information on the adjustment is balanced.

[0118] 3. Impurity detection

[0119] The impurity detection module is responsible for the final impurity detection of the image after layering. This module uses advanced target detection algorithms, such as detection models based on convolutional neural networks (CNN) or transformers. The process of the target detection model is as follows:

[0120] Feature extraction: Multi-scale feature maps are extracted from the layered processed image to detect impurities of different sizes.

[0121] Classification and localization: Impurities are classified and localized through an object detection head (such as YOLO or Faster R-CNN).

[0122] Output: Test results including the category c of impurities i and location information (x i ,y i ,w i ,h i ).

[0123] Target detection formula:

[0124] Output = {(c i ,(x i ,y i ,w i ,h i ))}

[0125] Among them, c i Indicates the impurity category, (x i ,y i ,w i ,h i ) represents the coordinates and size of the detection box.

[0126] 4. Result Output

[0127] The result output module presents the test results in a visual form and generates a detailed test report. The report includes:

[0128] The type and location of each impurity;

[0129] Confidence (probability) of detection;

[0130] Possible treatment suggestions (such as whether further examination or manual intervention is needed).

[0131] In addition, the test results can be linked with the production line control method to achieve automated product quality control.

[0132] In one embodiment of the present invention, a device for detecting foreign matter in tobacco may be provided. The device is used to execute the method for detecting foreign matter in tobacco, and includes an image acquisition and preprocessing module, a layered processing module, an impurity detection module, and a result output module.

[0133] In another specific embodiment of the present invention, an electronic device may be provided, comprising: at least one processor, and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method for detecting impurities in tobacco.

[0134] In another specific embodiment of the present invention, a computer-readable storage medium may be provided, on which a computer program is stored, and when the program is executed by a processor, the method for detecting foreign matter in tobacco is implemented.

[0135] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods of each embodiment or some parts of the embodiment.

[0136] Example 1

[0137] (I) Training sample information

[0138] 1. Dataset source:

[0139] High-resolution tobacco images were obtained from the image acquisition equipment of the tobacco production line. 3CCD industrial cameras were used for data acquisition. The specific parameters are as follows: resolution: 2048 pixels; maximum line scan speed: 19,047 lines / second; output format: 3 formats: 7: acquisition or 3 formats: 7: acquisition; other functions: flat field correction, shading correction, color gradient compensation and white balance; constant light conditions during data acquisition to ensure consistency of data quality. The original size of each image is 2080 lines. The conditions are constant to ensure pixels.

[0140] The dataset includes annotated images of common impurities in 12 production lines, totaling 1,383 images.

[0141] 2. The data set contains the following main contents:

[0142] Impurity categories: hemp rope, waste paper, feathers, cloth, weeds, plastic, wood blocks, cardboard, wrapping paper, abnormal tobacco leaves (such as too dark or too light in color), fibers;

[0143] Background information: The background is mainly tobacco leaves and conveyor belts, among which the texture and color of the conveyor belt are clearly shown in the image.

[0144] The number of samples of different categories in the dataset is distributed as follows: hemp rope - 120; waste paper - 150; feathers - 100; cloth - 110; weeds - 130; plastic - 140; wood blocks - 90; cardboard - 100; wrapping paper - 115; abnormal tobacco leaves - 150; fiber - 120.

[0145] 3. Data preprocessing and annotation methods:

[0146] Preprocessing:

[0147] The original images are too large to be used directly for model training. Therefore, the image size is cropped to a fixed size of 2080×480, and a total of 1383 images are obtained after cropping.

[0148] The LabelImg tool is used to annotate the impurities and tobacco leaf areas with bounding boxes. The annotation standard is that the bounding box completely contains the target object.

[0149] Each image contains annotated impurity areas, with rectangular boxes marking the impurity locations and category labels attached.

[0150] Number of annotations:

[0151] Number of impurity samples: 920 images containing mixed impurities.

[0152] Number of pure impurity samples: 463 (only one impurity).

[0153] The dataset is augmented in the following ways:

[0154] Methods: Mirror flip, random rotation (0-360°), scaling (±10%)

[0155] Results: After enhancement, the total number of samples in the dataset increased to 6915.

[0156] The dataset is divided into training set, validation set and test set in a ratio of 8:1:1: training set: 5532 images; validation set: 691 images; test set: 692 images; the test set only contains mixed impurity images to ensure the actual effect of model evaluation.

[0157] (II) Model training process

[0158] 1. Model structure:

[0159] Based on a multi-layer self-improving Transformer model, combined with a layer-by-layer processing framework.

[0160] The model consists of three layers: macro-region screening, meso-region refinement, and micro-region in-depth detection.

[0161] 2. Training parameter settings:

[0162] Optimizer: Adam, initial learning rate 0.001, dynamic adjustment using cosine annealing strategy.

[0163] Loss function:

[0164] First layer: region proposal loss (IoU loss) + classification loss (cross entropy).

[0165] Second layer: region proposal loss + classification loss + refinement loss (Focal Loss).

[0166] The third layer: final detection loss (Dice loss) + classification loss.

[0167] Number of training rounds: 100 rounds, early stopping strategy: validation set loss did not decrease significantly after 5 rounds (threshold was 0.0001).

[0168] Data augmentation: random rotation, scaling, mirror flipping.

[0169] 3. Hardware configuration:

[0170] GPU: NVIDIA GeForce RTX 4080.

[0171] Framework: PyTorch 1.8.

[0172] Data set division: training set (5532 images), validation set (692 images), and test set (692 images).

[0173] (III) Specific testing process

[0174] 1. Testing steps:

[0175] Input: High-resolution tobacco image (original resolution is 2080 × 24000 pixels).

[0176] Layer by layer processing:

[0177] The first layer (L1): downsamples the image to a resolution of 208×2400 to filter out areas that may contain impurities.

[0178] Second layer (L2): Enlarge the region of interest marked in L1 to a medium resolution of 520×6000 and refine the detection.

[0179] The third layer (L3): further enlarges the target area to a high resolution of 2080×24000 for final detection.

[0180] Output: Type and location of impurities detected.

[0181] 2. Model parameters:

[0182] Magnification factor:

[0183] From the first floor to the second floor: 10 times.

[0184] From the second floor to the third floor: 4 times.

[0185] Detection confidence threshold: 80%.

[0186] (4) Detection effect evaluation

[0187] 1. Test results:

[0188] A total of 692 images containing 920 impurity regions were detected in the test set.

[0189] Detection indicators:

[0190] Mean Average Precision (mAP): 95.01%.

[0191] Recall: 98.00%.

[0192] Precision: 80.29%.

[0193] 2. Performance comparison:

[0194] The method of the present invention is compared with mainstream models (YOLOv3, Faster R-CNN):

[0195] YOLOv3: mAP 40.09%, recall 57.57%.

[0196] Faster R-CNN: mAP 93.74%, recall 93.54%.

[0197] Model of the present invention: mAP increased by 1.27%, and recall rate increased by 4.46%.

[0198] 3. Industrial application effect:

[0199] After running for 100 hours on the actual production line, the impurity removal rate reached 97% without human intervention.

[0200] Detection speed: average processing time is 0.32 seconds per image, meeting industrial real-time detection needs.

[0201] In summary, the tobacco impurity detection method proposed in the present invention realizes efficient processing of high-resolution images and accurate detection of impurities through a multi-layer self-improving Transformer model. Compared with traditional methods, this method not only improves the accuracy of impurity detection, but also significantly reduces the demand for computing resources, making it more suitable for actual production environments.

[0202] The applicant declares that the above is only a specific implementation mode of the present invention, but the protection scope of the present invention is not limited thereto. Those skilled in the art should understand that any changes or substitutions that can be easily thought of by those skilled in the art within the technical scope disclosed by the present invention shall fall within the protection scope and disclosure scope of the present invention.

Claims

1. A training method for an image layer-by-layer processing model, characterized in that: The training method of the image layer-by-layer processing model includes: Obtain a tobacco image dataset with impurity annotations, wherein each image in the tobacco image dataset contains an annotated impurity region and a corresponding impurity category label, and use the tobacco image dataset as a training set; Standardizing the images in the training set, training the improved Transformer model, and obtaining a layer-by-layer image processing model, wherein the training includes single-layer training and multi-layer joint training; The single-layer training includes: First-layer training: input the images in the training set and the corresponding impurity region annotations for training, and use the loss function to calculate the first-layer region proposal loss L RPN and classification loss L cls , calculate the first layer comprehensive loss L L1 , L L1 =L RPN +α1·L cls , where α1 is the coefficient used to weigh the two parts of loss; Second-layer training: Input the image obtained from the first-layer training and the corresponding impurity region annotation for training, and use the loss function to calculate the second-layer refinement loss L refine , region proposal loss L RPN and classification loss L cls , calculate the second layer comprehensive loss L L2 , L L2 =L RPN +α2·L cls +β2·L refine , where α2 and β2 are trade-off coefficients; Third-layer training: Input the image obtained from the second-layer training and the corresponding impurity region annotation for training, and use the loss function to calculate the final detection loss L of the third layer final , classification loss L cls and the refinement loss L refine , calculate the third layer comprehensive loss L L3 , L L3 =L final +α3·L cls +β3·L refine , where α3 and β3 are trade-off coefficients; Adjust the model parameters according to the comprehensive loss of each layer until the comprehensive loss is lower than the preset threshold; The multi-layer joint training includes calculating the feedback consistency loss L feedback and the global optimization loss L global , calculate the total loss L total , L total =γ1·L L1 +γ2·L L2 +γ3·L L3 +λ·L feedback +δ·L global , where γ1, γ2, and γ3 represent the weights of the comprehensive loss of each layer, λ and δ represent the weights of feedback consistency and global optimization, respectively. The parameters of the model are adjusted according to the total loss until the total loss is lower than the preset threshold.

2. The training method of the image layer-by-layer processing model according to claim 1, characterized in that: The loss function for training each layer includes any one of the cross entropy loss function, IoU loss function, FocalLoss loss function or Dice loss function.

3. The training method of the image layer-by-layer processing model according to claim 1 or 2, characterized in that: The training also includes a model fine-tuning step, which specifically includes calculating the fine-tuning loss L finetune , L finetune =L final +δ·L global , adjust the parameters of the model according to the fine-tuning loss until the fine-tuning loss is lower than the preset threshold.

4. A layer-by-layer image processing model, characterized in that: The image layer-by-layer processing model is trained by the image layer-by-layer processing model training method described in any one of claims 1-3.

5. A method for layer-by-layer processing of tobacco production process images, characterized in that: The method for layer-by-layer processing of tobacco production process images comprises using the layer-by-layer image processing model of claim 4 to process the tobacco production process images layer by layer to obtain a target image area; The layer-by-layer processing includes: The first level of processing: macro-region screening, input tobacco production process image I0, image resolution R x ×R y , downsample the image by the scaling factor s1 to obtain the reduced image I1, whose resolution is Perform impurity detection on I1 to locate the target area that may contain impurities. According to the formula I1 = Downsample(I0, s1), the area is marked with coordinate information and passed as input to the second layer for processing; The second layer of processing: refine the mesoscopic region, input the reduced image I1 and the target region obtained from the first layer of processing, zoom in on the target region marked in I1, and the zoom factor is r2 to obtain a medium-resolution image I2. According to the formula I2 = ZoomIn (I1, r2), feature extraction and impurity detection are performed on the zoomed region, and the target region is marked; The third layer of processing: perform micro-area depth detection, input the image I2 obtained from the second layer of processing and the target area, zoom in on the target area with a zoom factor of r3, obtain a high-resolution regional image, and perform impurity detection on this basis. According to the formula I3=ZoomIn(I2,r3), output the precise location and category of the detected impurities to generate the final detection result.

6. The method for layer-by-layer processing of tobacco production process images according to claim 5, characterized in that: The method for layer-by-layer processing of tobacco production process images includes a feedback optimization step, and the formula is: in, is the image representation processed by the kth layer, σ is the feedback weight coefficient, is the feedback information from the k+1th layer processing.

7. A method for detecting foreign matter in tobacco, characterized in that: The method for detecting foreign matter in tobacco comprises the following steps: (1) Acquire tobacco images from an image acquisition device of a tobacco production line and perform preprocessing; (2) processing the image obtained in step (1) using the method for layer-by-layer processing of tobacco production process images as described in claim 5 or 6; (3) Use a machine learning algorithm to perform impurity detection on the image processed in step (2) and output the detection result.

8. The method for detecting foreign matter in tobacco according to claim 7, characterized in that: The machine learning algorithm includes YOLO or Faster R-CNN.

9. An electronic device, characterized in that: The electronic device includes: at least one processor, and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method for detecting foreign matter in tobacco as described in claim 7 or 8.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method for detecting foreign matter in tobacco according to claim 7 or 8 is implemented.

Images