Tobacco impurity detection method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention discloses a tobacco impurity detection method and device, electronic equipment and a storage medium. The method comprises the following steps: receiving a target tobacco image of sundries to be detected; based on a pre-trained light source estimation model, performing light source recovery processing on the target tobacco image, and taking an image obtained after the light source recovery processing as a to-be-detected image; and inputting the to-be-detected image into a pre-trained impurity detection model, and outputting impurity information in the tobacco corresponding to the to-be-detected image. According to the technical scheme, the target tobacco image can be subjected to light source recovery processing, the image quality is improved, the to-be-processed image obtained after the image quality is improved is input into the impurity detection model for impurity detection, and the accuracy and effectiveness of a detection result can be improved.

Description

Technical Field

[0001] The present invention relates to the field of image recognition technology, and in particular to a method, apparatus, electronic device and storage medium for detecting tobacco debris. Background Technology

[0002] During tobacco production, due to the complexity of various processes, impurities can easily get mixed into the tobacco. To ensure the quality of the produced tobacco, it is necessary to conduct impurity testing.

[0003] In existing technologies, impurities in tobacco are detected by acquiring images of tobacco on a conveyor belt. However, in the process of implementing this invention, it was found that the existing technology has at least the following technical problems: due to the unstable light quality in the tobacco production environment, the quality of the acquired tobacco images is affected, leading to inaccurate detection results in the process of detecting impurities based on tobacco images. Summary of the Invention

[0004] This invention provides a method, apparatus, electronic device, and storage medium for detecting tobacco impurities, aiming to improve the accuracy and effectiveness of detection results.

[0005] According to one aspect of the present invention, a method for detecting tobacco impurities is provided, comprising:

[0006] Receive the target tobacco image containing the impurities to be detected;

[0007] Based on a pre-trained light source estimation model, the target tobacco image is processed to restore the light source, and the image obtained after the light source restoration is used as the image to be detected.

[0008] The image to be detected is input into a pre-trained impurity detection model, which outputs the impurity information in the tobacco corresponding to the image to be detected.

[0009] According to another aspect of the present invention, a tobacco impurity detection device is provided, the device comprising:

[0010] The image receiving module is used to receive the target tobacco image of the impurity to be detected;

[0011] The light source restoration module is used to perform light source restoration processing on the target tobacco image based on a pre-trained light source estimation model, and use the image obtained after the light source restoration processing as the image to be detected;

[0012] The impurity information output module is used to input the image to be detected into a pre-trained impurity detection model and output the impurity information in the tobacco corresponding to the image to be detected.

[0013] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0014] At least one processor; and

[0015] A memory communicatively connected to the at least one processor; wherein,

[0016] The memory stores a computer program that can be executed by the at least one processor, which enables the at least one processor to perform the tobacco impurity detection method according to any embodiment of the present invention.

[0017] According to another aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions for causing a processor to execute and implement the tobacco impurity detection method according to any embodiment of the present invention.

[0018] The technical solution of this invention involves receiving a target tobacco image containing impurities, performing light source restoration processing on the target tobacco image based on a pre-trained light source estimation model, and using the image obtained after light source restoration as the image to be detected. This light source restoration process improves image quality. The image to be detected is then input into a pre-trained impurity detection model, which outputs impurity information in the tobacco corresponding to the image to be detected. In this embodiment, the light source restoration processing of the target tobacco image improves image quality, and the processed image is then input into the impurity detection model for impurity detection, which helps improve the accuracy and effectiveness of the detection results.

[0019] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 This is a flowchart of a tobacco impurity detection method according to an embodiment of the present invention;

[0022] Figure 2 This is a flowchart of another method for detecting tobacco impurities provided according to an embodiment of the present invention;

[0023] Figure 3 This is a schematic diagram of the structure of a tobacco impurity detection device according to an embodiment of the present invention;

[0024] Figure 4 This is a schematic diagram of the structure of an electronic device that implements the tobacco impurity detection method of this invention. Detailed Implementation

[0025] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0026] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "etc." and any variations thereof are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0027] It should be noted that the collection, gathering, updating, analysis, processing, use, transmission, and storage of user personal information involved in this disclosed technical solution all comply with relevant laws and regulations, are used for legitimate purposes, and do not violate public order and good morals. Necessary measures are taken to prevent unauthorized access to user personal information data and to safeguard user personal information security and network security.

[0028] Figure 1 This is a flowchart of a tobacco impurity detection method according to an embodiment of the present invention. This embodiment is applicable to determining impurity information in tobacco. The method can be executed by a tobacco impurity detection device, which can be implemented in hardware and / or software.

[0029] like Figure 1 As shown, the method in this embodiment may specifically include:

[0030] S110: Receive the target tobacco image of the impurities to be detected.

[0031] The target tobacco image is an image containing tobacco from which impurities need to be detected.

[0032] In this embodiment, tobacco can be transported via a conveyor belt on a production line. An image acquisition device captures images of the tobacco transported on the conveyor belt, and these captured images are used as target tobacco images. The image acquisition device can be an industrial camera, which has high sensitivity and high dynamic range, enabling it to capture detailed images in low-light and high-contrast scenes. For example, the resolution of the target tobacco image can be 8000×2000 pixels.

[0033] S120. Based on the pre-trained light source estimation model, perform light source restoration processing on the target tobacco image, and use the image obtained after light source restoration processing as the image to be detected.

[0034] Among them, the light source estimation model includes a convolutional neural network model.

[0035] Specifically, the light source restoration process for the target tobacco image is performed based on a pre-trained light source estimation model, including: inputting the target tobacco image into the light source estimation model to obtain the light source distribution information corresponding to the target tobacco image; and performing light source restoration process on the target tobacco image based on the light source distribution information.

[0036] The light source distribution information reflects the light intensity corresponding to each pixel in the target tobacco image. The light source estimation model includes a convolutional neural network model.

[0037] In this embodiment, the target tobacco image can be input into a pre-trained light source estimation model to extract the light source distribution information from the target tobacco image. The light source estimation model can adopt a U-Net structure, possessing global and local feature extraction capabilities, and can accurately estimate the illumination distribution in complex light source environments. The input image size of the light source estimation model is H×W×C, where H is the image height, W is the image width, and C is the number of channels. For example, the input image can be an RGB (red, green, blue) color mode image. The light source estimation model includes an encoder, a decoder, and an output layer. Features from corresponding layers of the encoder and decoder are fused through skip connections. The encoder consists of a series of convolutional layers, and the output of each convolutional layer can be represented as:

[0038]

[0039] in, This represents the feature map output by the current convolutional layer. This represents the convolution operation. and The first The weights and biases of convolutional layers, It is the output feature map of the previous convolutional layer. 1 is the ReLU (Rectified Linear Unit) activation function. l is a positive integer.

[0040] The decoder recovers the spatial resolution of the feature map through deconvolution or upsampling, and the output of each layer can be:

[0041]

[0042] in, This represents the feature map output by the current deconvolution layer. This represents the convolution operation. 1 is the ReLU (Rectified Linear Unit) activation function. and The first The weights and biases of the deconvolution layer This represents the output feature map of the previous deconvolutional layer.

[0043] The output layer generates light source distribution information through convolution operations. The formula for determining the light source distribution information is as follows:

[0044] in, Indicates the weights of the output layer. Indicates the bias of the output layer. This represents the feature obtained after skipping the output features of the last convolutional layer and the output features of the last deconvolutional layer. This represents the Sigmoid activation function. This represents the light source distribution information. The Sigmoid activation function is used to normalize the output to the range [0, 1].

[0045] Furthermore, based on the light source distribution information, light source restoration processing is performed on the target tobacco image, including: for each pixel in the target tobacco image, determining the brightness value corresponding to the pixel, determining the light intensity value of the pixel based on the light source distribution information, determining the light source restoration value corresponding to the pixel based on the brightness value and the light intensity value, and updating the brightness value of the pixel in the target tobacco image to the light source restoration value.

[0046] It should be noted that the goal of light source restoration is to remove lighting inconsistencies caused by insufficient or uneven illumination, making the image appear as if it were taken under uniform and sufficient lighting conditions. This involves determining the brightness and intensity values ​​of each pixel in the target tobacco image.

[0047] In practice, the method for determining the light source restoration value corresponding to a pixel based on the brightness value and light intensity value is as follows:

[0048]

[0049] in, Indicates the light source restoration value. Represents the brightness value of a pixel. This represents the light intensity value of a pixel. It is a tiny constant used to prevent division by zero.

[0050] In this embodiment, after determining the light source restoration value for each pixel, the brightness value of that pixel in the target tobacco image is replaced with the light source restoration value. The image obtained after adjusting the brightness value of each pixel is used as the image to be detected.

[0051] This implementation quickly determines the light source distribution information through a light source estimation model; and adjusts the brightness value in the target tobacco image based on the light source distribution information, thereby removing the illumination inconsistency caused by insufficient or uneven illumination.

[0052] S130. Input the image to be detected into the pre-trained impurity detection model and output the impurity information in the tobacco corresponding to the image to be detected.

[0053] To determine the impurity information in the target tobacco image, the image to be detected, restored by the light source, can be input into the impurity detection model. The output information of the impurity detection model can then be used as the impurity information of the tobacco contained in the image to be detected.

[0054] Optionally, the impurity detection model includes an object detection model, which utilizes a deep learning object detection algorithm to identify and locate impurities in the image to be detected. For example, the impurity detection model may include a YOLOv8 model or a Faster R-CNN model. The impurity information refers to the attribute information reflecting objects in the tobacco to be detected that do not belong to the tobacco category.

[0055] In specific implementation, the image to be detected is input into a pre-trained impurity detection model, and the impurity information in the tobacco corresponding to the image to be detected is output, including: inputting the image to be detected into a pre-trained target detection model, and outputting at least one of the impurity location, impurity type and confidence level in the tobacco corresponding to the image to be detected.

[0056] The impurity information includes at least one of the following: impurity location, impurity type, and confidence level. Impurity categories include foreign objects, dust, and metal fragments. Impurity location refers to the position of the impurity within the tobacco corresponding to the image being detected. The confidence level reflects the accuracy of the impurity information; a higher confidence level indicates higher accuracy, and vice versa.

[0057] In practical applications, the detected impurity information can also be visualized, allowing users to intuitively view the location, type, and degree of impurity, and generate corresponding test reports. The detection results can be used for subsequent production quality control and management to assess the accuracy and recall rate of the test results.

[0058] The technical solution of this invention involves receiving a target tobacco image containing impurities, performing light source restoration processing on the target tobacco image based on a pre-trained light source estimation model, and using the image obtained after light source restoration as the image to be detected. This light source restoration process improves image quality. The image to be detected is then input into a pre-trained impurity detection model, which outputs impurity information in the tobacco corresponding to the image to be detected. In this embodiment, the light source restoration processing of the target tobacco image improves image quality, and the processed image is then input into the impurity detection model for impurity detection, which helps improve the accuracy and effectiveness of the detection results.

[0059] Figure 2 This is a flowchart of another tobacco debris detection method provided by an embodiment of the present invention. Optionally, based on the above embodiments, before performing light source restoration processing on the target tobacco image based on a pre-trained light source estimation model, this embodiment further includes: filtering the target tobacco image based on a pre-built filter; normalizing the filtered image to obtain a normalized image; and updating the target tobacco image to the normalized image. The explanations of terms that are the same as or corresponding to those in the above embodiments are not repeated here. Figure 2 As shown, the method includes:

[0060] S210, Receive the target tobacco image of the impurity to be detected.

[0061] S220. Based on the pre-constructed filter, the target tobacco image is filtered; the image obtained after filtering is normalized to obtain a normalized image, and the target tobacco image is updated to the normalized image.

[0062] The filter can be a Gaussian filter.

[0063] In practice, the target tobacco image can be filtered to remove random noise generated during image acquisition. To meet the input requirements of the impurity detection model and the light source estimation model, the filtered image can be normalized. Specifically, the pixel value of each pixel in the filtered target tobacco image is normalized so that the pixel value ranges from [0, 1]. The normalized image is then used as the normalized image, and the target tobacco image is updated to the normalized image to facilitate subsequent light source restoration operations. For example, the pixel value can be the brightness value of the pixel in the image.

[0064] S230. Based on a pre-trained light source estimation model, perform light source restoration processing on the target tobacco image, and use the image obtained after light source restoration processing as the image to be detected.

[0065] S240. Input the image to be detected into the pre-trained impurity detection model and output the impurity information in the tobacco corresponding to the image to be detected.

[0066] To improve the accuracy of impurity identification, before inputting the image to be detected into the pre-trained impurity detection model, the method further includes: inputting the image to be detected into the pre-trained denoising model, and updating the image to be detected based on the output of the denoising model.

[0067] The denoising model is based on a residual network. It consists of multiple residual blocks, where the input of the l-th layer is directly added to the corresponding output, forming a skip connection that helps the gradient propagate more effectively.

[0068] In practice, in order to further eliminate noise in the image to be detected and fully preserve the important details in the acquired target tobacco image, noise reduction processing can be performed on the image to be detected, based on the restoration of the light source.

[0069] Specifically, the denoising model possesses the skip connections and deep feature extraction capabilities of residual networks. By inputting the image to be detected into the pre-trained denoising model, a denoised feature map is obtained. This denoised image is then combined with light source distribution information to obtain a combined image. The output layer outputs the combined image through convolution operations and updates the image to be detected using this combined image. The updated image to be detected can be:

[0070]

[0071] in, This represents the pixel values ​​corresponding to each pixel in the updated image to be detected. This represents the activation function used in the denoising model. This represents the weights of the output layer in the denoising model. This represents the bias of the output layer in the denoising model. This indicates a combined image.

[0072] This embodiment improves the impurity identification effect by denoising the image to be detected, and can fully preserve the important details in the acquired target tobacco image.

[0073] Furthermore, before inputting the image to be detected into the pre-trained impurity detection model, the method further includes: performing adaptive enhancement processing on the image to be detected based on the pre-trained adaptive enhancement network and the pixel values ​​of each pixel in the image to be detected; and updating the image to be detected based on the image obtained after the adaptive enhancement processing; wherein, the adaptive enhancement network can be a multi-scale convolutional neural network.

[0074] To improve the ease of impurity identification, the brightness, contrast, and color saturation of the image to be detected can be further adjusted before inputting it into the impurity detection model, so that the image to be detected can obtain appropriate enhancement effects in different brightness areas.

[0075] Optionally, the image to be detected is input into a pre-trained adaptive enhancement network. The adaptive enhancement network outputs the enhanced pixel value of each pixel in the image to be detected. The enhanced pixel value replaces the pixel value of each pixel in the image to be detected that has not undergone adaptive enhancement. The image with the replaced pixel value is then updated as the image to be detected.

[0076] It should be noted that the adaptive enhancement network is a network that adjusts pixel values ​​through a constructed adaptive function. The formula for adjusting the image using the adaptive function is as follows:

[0077]

[0078] in, Indicates the image to be detected The pixel values ​​obtained after adaptive adjustment This indicates that in the image to be detected, without adaptive adjustment... pixel values, (·) is an adaptive function used to adjust pixel values ​​by combining light source distribution information and local contrast. Indicates information about the distribution of light sources. Indicates local contrast. Based on pixels A measure of the brightness distribution of pixels within its neighborhood, which can be calculated using the following formula:

[0079]

[0080] in, Represents the pixels in the image to be detected brightness value, Represents pixels Surrounding Neighborhood Average brightness value of the inner pixel, This is a small constant used to prevent division by zero. It should be noted that those skilled in the art can determine the surrounding neighborhood of a pixel based on the specific application. For example, a pixel... The surrounding neighborhood can be defined as: pixels A circular area with a preset length as its radius, centered on a given area.

[0081] In practical applications, adaptive enhancement networks involve several steps: input, multi-scale feature extraction, feature fusion and adaptive adjustment, and output enhancement. The input to the adaptive enhancement network is the image to be detected and light source distribution information. Multi-scale convolutional layers are used to extract features at different scales, enhancing the details and overall contrast of the image to be detected. The multi-scale feature extraction process can be represented by the following formula:

[0082]

[0083] in, This represents the feature map corresponding to the l-th convolutional layer. It is the weight matrix of the multi-scale convolution kernel. This represents the bias of the multi-scale convolution kernel. This represents the pixel value of each pixel in the image to be detected. Let l represent the activation function of the adaptive augmentation network. l is a positive integer.

[0084] Furthermore, in the feature fusion and adaptive adjustment section, multi-scale feature maps and light source distribution information can be fused to calculate adaptive adjustment coefficients, as shown in the following formula:

[0085]

[0086] in, This represents the adaptive adjustment coefficient. For a pre-defined multilayer perceptron, This represents the image to be detected that is input into the adaptive enhancement network.

[0087] In the enhanced output section, an adaptive adjustment coefficient can be used to enhance the image to be detected, as shown in the following formula:

[0088]

[0089] in, This represents the pixel value of each pixel in the image to be detected after enhancement. This represents the adaptive adjustment coefficient. This represents the pixel value of each pixel in the image to be detected.

[0090] In this embodiment, to ensure the enhancement effect, the brightness histogram and contrast histogram of the image to be detected before and after enhancement can be compared to ensure that the image to be detected is visually smoother and more detailed after adaptive enhancement processing.

[0091] The adaptively enhanced image to be detected can be input into a pre-trained impurity detection model to determine impurity information. To better understand the impurity detection process, the structure of the impurity detection model is described below. The network structure of the impurity detection model includes an input layer, a feature extraction layer, a detection head, and an output layer. The input layer takes the image to be detected as input, and the feature extraction layer uses a convolutional neural network to extract global features. For example, a residual network or CSPDarknet (a cross-stage part of Darknet network) can be used to extract global features. The detection head includes classification and regression branches to predict the category and coordinates of each candidate region. The output layer outputs the detection results, namely the impurity category, impurity location, and confidence level of the impurities contained in the tobacco.

[0092] This embodiment utilizes an adaptive enhancement network to ensure that all brightness areas of the image under test possess clear details and natural colors after enhancement processing, which is beneficial for impurity identification. Furthermore, combined with the aforementioned light source restoration, it enables high-precision impurity detection even under low-light conditions.

[0093] The embodiments of the tobacco impurity detection method have been described in detail above. In order to enable those skilled in the art to further understand the technical solution of this method, the training process of the model used in the tobacco impurity detection process will be explained below.

[0094] In this embodiment, a light source estimation model, a denoising model, an adaptive enhancement network, and an impurity detection model are used in the tobacco impurity detection process. The training process of each model is described in detail below:

[0095] 1. The training objective of the light source estimation model is to enable it to accurately predict the light source distribution information in the target tobacco image, thereby correcting the influence of uneven illumination. The first loss function used to train the light source estimation model is usually based on the error between the actual light source distribution and the network-predicted light source distribution. The first loss function includes the mean squared error (MSE), and is shown in the following formula:

[0096]

[0097] Where N represents the number of pixels, which is a positive integer. Indicates the first element in the target tobacco image. The position of each pixel Indicates the predetermined first The true light intensity of each pixel Indicates the first The light intensity of each pixel is predicted by the light source estimation model to be trained. This is the first loss value corresponding to the first loss function.

[0098] To obtain a satisfactory light source estimation model, gradient descent can be used to minimize the loss function, thereby updating the first network parameters of the light source estimation model, as shown in the following equation:

[0099]

[0100] in, 1 is the first learning rate. The first network parameters are obtained in the t-th iteration. These are the first network parameters obtained in the (t+1)th iteration. Let be the partial derivative between the first loss value and the first network parameters. Here, t is a positive integer.

[0101] 2. The training objective of the denoising model is to remove noise from the image after the light source has been recovered, while preserving important image details. A second loss function is used to train the denoising model. This second loss function combines pixel-level L1 loss and perceptual loss. The L1 loss ensures that the pixel values ​​of the denoised image are close to the true values, while the perceptual loss preserves the image's structure and details.

[0102]

[0103] in, This represents the second loss function. This indicates the denoising result of the image to be detected after being denoised by the denoising model to be trained. The pixel value of each pixel. This represents the first noise-free image corresponding to the image to be detected. The pixel value of each pixel. It is perceived loss. It is the weighting factor corresponding to the perceptual loss, used to balance the L1 loss and the perceptual loss.

[0104] In this embodiment, the perceptual loss is based on the output of the high-level feature extraction network and is used to measure the difference between the noisy image and the real image in the high-level feature space:

[0105]

[0106] Where C represents the number of channels in the image to be detected, H represents the image height of the image to be detected, and W represents the image width of the image to be detected. This represents the features extracted when the input to the denoising model to be trained is the image to be detected; This represents the features extracted when the input to the denoising model to be trained is a noise-free image.

[0107] Furthermore, the second loss function can be minimized using gradient descent, and the second network parameters of the denoising model can be updated. The update process is as follows:

[0108]

[0109] in, 2 is the second learning rate. The second network parameters are obtained in the t-th iteration. The second network parameters are obtained in the (t+1)th iteration. Let be the partial derivative between the second loss value and the second network parameters. Here, t is a positive integer.

[0110] 3. The training objective of the adaptive augmentation network is to adaptively adjust the denoised image based on the light source distribution information, enhancing the image's contrast and brightness. The adaptive augmentation network can be trained using a third loss function, which considers both the contrast enhancement effect and visual fidelity. The third loss function combines the pixel-level error between the enhanced image and the image to be detected, brightness loss, and contrast loss. Specifically, contrast loss measures the difference between the contrast of the enhanced image and the preset standard contrast; brightness loss measures the difference between the average brightness of the enhanced image and the target brightness; and the pixel-level error between the enhanced image and the image to be detected can be represented by the L2 loss function. The third loss function... The formula is as follows:

[0111]

[0112] in, This represents the pixel value corresponding to the i-th pixel after the image to be detected has been enhanced by the adaptive enhancement network to be trained. This represents the enhanced standard pixel value corresponding to the pre-defined i-th pixel. It's a loss of contrast. It's a loss of brightness. The weighting factor for contrast loss. This represents the weighting factor corresponding to the brightness loss.

[0113] In this embodiment, the third loss function can be minimized using gradient descent, and the third network parameters in the adaptive enhancement network can be updated. Specifically, the third network parameters can be:

[0114] in, 3 is the third learning rate. The third network parameter is obtained in the t-th iteration. The third network parameter is obtained in the (t+1)th iteration. Let be the partial derivative between the third loss value and the third network parameters. Here, t is a positive integer.

[0115] 4. The training objective of the impurity detection model is to accurately detect and locate impurities on the enhanced image. A fourth loss function is used to train the impurity detection model. This fourth loss function includes classification loss and bounding box regression loss. The classification loss measures the difference between the network's predicted impurity category and the true category, typically using cross-entropy loss. The bounding box regression loss measures the difference between the predicted bounding box location and the true location, typically using smoothing L1 loss. The fourth loss function... It can be represented as:

[0116]

[0117] in, The weights used to balance the bounding box regression loss and classification loss, Represents classification loss. This represents the bounding box regression loss.

[0118] The fourth loss function can be minimized using gradient descent, and the parameters of the fourth network in the impurity detection model to be trained can be updated. The optimization process is as follows:

[0119]

[0120] in, 4 is the fourth learning rate. The fourth network parameter is obtained in the t-th iteration. The fourth network parameter is obtained in the (t+1)th iteration. Let be the partial derivative between the fourth loss value and the fourth network parameter. Here, t is a positive integer.

[0121] This embodiment provides training methods for each model, thereby using the trained light source estimation model, denoising model, adaptive enhancement network, and impurity detection model to process the target tobacco image, ensuring high-precision impurity detection in low-light environments and reducing reliance on expensive hardware.

[0122] Figure 3 This is a schematic diagram of a tobacco impurity detection device according to an embodiment of the present invention. This device is used to perform the tobacco impurity detection method provided in any of the above embodiments. This device and the tobacco impurity detection methods of the above embodiments belong to the same inventive concept. Details not described in detail in the embodiments of the tobacco impurity detection device can be found in the embodiments of the tobacco impurity detection methods described above. Figure 3 As shown, the device includes:

[0123] Image receiving module 10 is used to receive the target tobacco image of the impurity to be detected;

[0124] The light source restoration module 11 is used to perform light source restoration processing on the target tobacco image based on a pre-trained light source estimation model, and use the image obtained after light source restoration processing as the image to be detected.

[0125] The impurity information output module 12 is used to input the image to be detected into a pre-trained impurity detection model and output the impurity information in the tobacco corresponding to the image to be detected.

[0126] Optionally, based on any of the optional technical solutions in the embodiments of the present invention, the device further includes:

[0127] The filtering module is used to filter the target tobacco image based on a pre-built filter before performing light source restoration processing on the target tobacco image based on the pre-trained light source estimation model.

[0128] The normalization module is used to normalize the image obtained after filtering to obtain a normalized image, and to update the target tobacco image to a normalized image.

[0129] Based on any optional technical solution in the embodiments of the present invention, the light source restoration module 11 may optionally include:

[0130] The first image input unit is used to input the target tobacco image into the light source estimation model to obtain the light source distribution information corresponding to the target tobacco image; wherein, the light source distribution information is used to reflect the light intensity corresponding to each pixel in the target tobacco image; the light source estimation model includes a convolutional neural network model;

[0131] The light source restoration unit is used to perform light source restoration processing on the target tobacco image based on the light source distribution information.

[0132] Based on any optional technical solution in the embodiments of the present invention, the light source restoration unit optionally includes:

[0133] The light source restoration value determination subunit is used to determine the brightness value corresponding to each pixel in the target tobacco image, determine the light intensity value of the pixel based on the light source distribution information, determine the light source restoration value corresponding to the pixel based on the brightness value and light intensity value, and update the brightness value of the pixel in the target tobacco image to the light source restoration value.

[0134] Optionally, based on any of the optional technical solutions in the embodiments of the present invention, it further includes:

[0135] The image update module is used to input the image to be detected into a pre-trained denoising model before inputting the image to be detected into the pre-trained impurity detection model, and update the image to be detected based on the output of the denoising model.

[0136] The denoising model is a model generated based on a residual network.

[0137] Optionally, based on any of the optional technical solutions in the embodiments of the present invention, it further includes:

[0138] The adaptive enhancement module is used to perform adaptive enhancement processing on the image to be detected based on the pre-trained adaptive enhancement network and the pixel values ​​of each pixel in the image to be detected before inputting the image to be detected into the pre-trained impurity detection model. Based on the image obtained after adaptive enhancement processing, the image to be detected is updated.

[0139] Among them, the adaptive enhancement network can be a multi-scale convolutional neural network.

[0140] Based on any optional technical solution in the embodiments of the present invention, optionally, the impurity detection model may include a target detection model; the impurity information output module 12 includes:

[0141] The second image input unit is used to input the image to be detected into a pre-trained target detection model and output at least one of the following: the location of the impurity in the tobacco corresponding to the image to be detected, the type of the impurity, and the confidence level.

[0142] The technical solution of this invention involves receiving a target tobacco image containing impurities, performing light source restoration processing on the target tobacco image based on a pre-trained light source estimation model, and using the image obtained after light source restoration as the image to be detected. This light source restoration process improves image quality. The image to be detected is then input into a pre-trained impurity detection model, which outputs impurity information in the tobacco corresponding to the image to be detected. In this embodiment, the light source restoration processing of the target tobacco image improves image quality, and the processed image is then input into the impurity detection model for impurity detection, which helps improve the accuracy and effectiveness of the detection results.

[0143] It is worth noting that in the embodiments of the tobacco debris detection device described above, the various units and modules are divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be achieved; in addition, the specific names of each functional unit are only for easy differentiation and are not used to limit the scope of protection of the present invention.

[0144] Figure 4 This is a schematic diagram of an electronic device implementing the tobacco impurity detection method of this invention. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0145] like Figure 4 As shown, the electronic device 20 includes at least one processor 21 and a memory, such as a read-only memory (ROM) 22 or a random access memory (RAM) 23, communicatively connected to the at least one processor 21. The memory stores computer programs executable by the at least one processor. The processor 21 can perform various appropriate actions and processes based on the computer program stored in the ROM 22 or loaded from storage unit 28 into the RAM 23. The RAM 23 can also store various programs and data required for the operation of the electronic device 20. The processor 21, ROM 22, and RAM 23 are interconnected via a bus 24. An input / output (I / O) interface 25 is also connected to the bus 24.

[0146] Multiple components in electronic device 20 are connected to I / O interface 25, including: input unit 26, such as keyboard, mouse, etc.; output unit 27, such as various types of monitors, speakers, etc.; storage unit 28, such as disk, optical disk, etc.; and communication unit 29, such as network card, modem, wireless transceiver, etc. Communication unit 29 allows electronic device 20 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0147] Processor 21 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 21 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 21 performs the various methods and processes described above, such as tobacco debris detection methods.

[0148] In some embodiments, the tobacco impurity detection method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 28. In some embodiments, part or all of the computer program may be loaded and / or installed on electronic device 20 via ROM 22 and / or communication unit 29. When the computer program is loaded into RAM 23 and executed by processor 21, one or more steps of the tobacco impurity detection method described above may be performed. Alternatively, in other embodiments, processor 21 may be configured to perform the tobacco impurity detection method by any other suitable means (e.g., by means of firmware).

[0149] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0150] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0151] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0152] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0153] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or middleware components (e.g., application servers), or frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0154] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0155] This embodiment also provides a computer program product, including a computer program that, when executed by a processor, implements the tobacco impurity detection method as provided in any embodiment of this application.

[0156] In implementing the computer program product, computer program code for performing the operations of this invention can be written in one or more programming languages ​​or a combination thereof. Programming languages ​​include object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0157] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0158] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A method of detecting tobacco debris, characterized by, The method comprises the following steps: receiving a target tobacco image of impurities to be detected; performing light source restoration processing on the target tobacco image based on a pre-trained light source estimation model, and taking the image obtained after the light source restoration processing as a to-be-detected image; inputting the to-be-detected image into a pre-trained impurity detection model, and outputting impurity information in tobacco corresponding to the to-be-detected image.

2. The tobacco shred detection method according to claim 1, characterized by, Before the step of performing light source restoration processing on the target tobacco image based on the pre-trained light source estimation model, the method further comprises the following steps: performing filter processing on the target tobacco image based on a pre-constructed filter; performing normalization processing on the image obtained after the filter processing to obtain a normalized image, and updating the target tobacco image to the normalized image.

3. The tobacco shred detection method according to claim 1, characterized by, The step of performing light source restoration processing on the target tobacco image based on the pre-trained light source estimation model comprises the following steps: inputting the target tobacco image into the light source estimation model to obtain light source distribution information corresponding to the target tobacco image; wherein the light source distribution information is used to reflect light intensity corresponding to each pixel point in the target tobacco image; the light source estimation model comprises a convolutional neural network model; performing light source restoration processing on the target tobacco image based on the light source distribution information.

4. The tobacco shred detection method according to claim 3, characterized by, The step of performing light source restoration processing on the target tobacco image based on the light source distribution information comprises the following steps: for each pixel point in the target tobacco image, determining a brightness value corresponding to the pixel point, determining a light intensity value of the pixel point based on the light source distribution information, determining a light source restoration value corresponding to the pixel point based on the brightness value and the light intensity value, and updating the brightness value of the pixel point in the target tobacco image to the light source restoration value.

5. The tobacco shred detection method according to claim 1, wherein Before the step of inputting the to-be-detected image into the pre-trained impurity detection model, the method further comprises the following steps: inputting the to-be-detected image into a pre-trained denoising model, updating the to-be-detected image based on an output result of the denoising model; wherein the denoising model is a model generated based on a residual network.

6. The tobacco shred detection method according to claim 1, wherein Before the step of inputting the to-be-detected image into the pre-trained impurity detection model, the method further comprises the following steps: performing adaptive enhancement processing on the to-be-detected image based on a pre-trained adaptive enhancement network and pixel values of each pixel point of the to-be-detected image, and updating the to-be-detected image based on an image obtained after the adaptive enhancement processing; wherein the adaptive enhancement network can be a multi-scale convolutional neural network.

7. The tobacco shred detection method according to claim 1, wherein The impurity detection model can comprise a target detection model; the step of inputting the to-be-detected image into the pre-trained impurity detection model and outputting impurity information in tobacco corresponding to the to-be-detected image comprises the following steps: inputting the to-be-detected image into a pre-trained target detection model, and outputting at least one of an impurity position, an impurity type and a confidence in tobacco corresponding to the to-be-detected image.

8. A tobacco shreds detection device, characterized by, The method comprises the following steps: an image receiving module is configured to receive a target tobacco image of impurities to be detected; The light source recovery module is configured to perform light source recovery processing on the target tobacco image based on a pre-trained light source estimation model, and take an image obtained after the light source recovery processing as a to-be-detected image. The impurity information output module is configured to input the to-be-detected image into a pre-trained impurity detection model, and output impurity information in tobacco corresponding to the to-be-detected image.

9. An electronic device, comprising: The electronic device comprises: at least one processor; and a memory connected to the at least one processor in communication; wherein the memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to execute the tobacco impurity detection method in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer readable storage medium stores computer instructions, and the computer instructions are used to enable the processor to implement the tobacco impurity detection method in any one of claims 1-7 when executed.

Images