Cigarette auxiliary material automatic color difference recognition method and system based on convolutional neural network

Abstract

The application provides a cigarette auxiliary material automatic color difference recognition method and system based on a convolutional neural network, relates to the technical field of cigarette auxiliary material color difference detection, and comprises the following steps: obtaining standard sample images and to-be-detected sample images of cigarette auxiliary materials, and performing multidimensional preprocessing and color space conversion to generate Lab standard sample images and Lab to-be-detected sample images; training a convolutional neural network model by using a second expanded cigarette auxiliary material sample image library; inputting the Lab standard sample images and the Lab to-be-detected sample images into the trained convolutional neural network model to perform feature extraction, feature fusion, and color difference comparison, generating a color difference detection result corresponding to the to-be-detected sample images, and pushing the color difference detection result to a data management platform, so that the accuracy, stability, and reliability of the cigarette auxiliary material color difference detection can be improved, the missed detection rate and the false detection rate of the cigarette auxiliary material color difference detection can be reduced, and the production quality and brand image of the cigarette auxiliary material can be ensured.

Description

Technical Field

[0001] This invention relates to the field of color difference detection technology for cigarette auxiliary materials, and in particular to an automated color difference recognition method and system for cigarette auxiliary materials based on convolutional neural networks. Background Technology

[0002] In the cigarette production quality control system, the consistency of the appearance color of cigarette auxiliary materials is an important factor in ensuring product quality uniformity and maintaining brand image. The accuracy of color difference detection of cigarette auxiliary materials directly affects the product presentation effect and consumer trust.

[0003] Existing color difference detection technologies for cigarette auxiliary materials have the following drawbacks: First, existing technologies rely on manual visual inspection, resulting in poor consistency of color difference detection results and difficulty in identifying subtle color differences, failing to meet the high-precision detection requirements of cigarette auxiliary materials. Second, traditional machine vision inspection methods rely on the RGB color space for color comparison of cigarette auxiliary materials. The RGB color space is out of sync with human visual perception of color, affecting the effectiveness of color difference detection. Third, traditional machine vision inspection methods are not only weakly resistant to interference from factors such as lighting changes and surface reflectivity in complex production environments, but also lack the ability to accurately capture color details as cigarette auxiliary materials become more diverse and complex. They cannot adaptively learn the deep features of complex textures, leading to inaccurate color difference detection, high false negative and false positive rates, and low stability, failing to meet the requirements of high precision, high stability, and full-process automation for color difference detection of cigarette auxiliary materials.

[0004] Therefore, it is necessary to provide an automated color difference recognition method and system for cigarette auxiliary materials based on convolutional neural networks to solve the above-mentioned technical problems. Summary of the Invention

[0005] To address the aforementioned technical problems, this invention provides an automated color difference recognition method and system for cigarette auxiliary materials based on convolutional neural networks. This method solves the problems of low color difference recognition accuracy, high false negative and false positive rates, weak anti-interference ability, and poor adaptability to complex scenarios in traditional cigarette auxiliary material color difference detection methods.

[0006] The present invention provides an automated color difference recognition method for cigarette auxiliary materials based on convolutional neural networks, the method comprising: A high-precision image acquisition system was built to acquire standard sample images and test sample images of cigarette auxiliary materials, and multi-dimensional preprocessing and color space conversion were performed to generate Lab standard sample images and Lab test sample images. The image library of cigarette auxiliary materials is retrieved, and data augmentation technology is used to expand the image library of cigarette auxiliary materials to generate a first expanded image library of cigarette auxiliary materials. Then, based on MixUp image blending technology, the first expanded image library of cigarette auxiliary materials is augmented to generate a second expanded image library of cigarette auxiliary materials, and a convolutional neural network model is trained. The Lab standard sample image and the Lab sample image to be detected are input into the trained convolutional neural network model. Feature extraction and fusion are performed on the Lab standard sample image and the Lab sample image to be detected respectively to generate Lab standard sample image features and Lab sample image to be detected features. The color difference of the Lab standard sample image features and the Lab sample image to be detected features are compared to generate the color difference detection result corresponding to the sample image to be detected and pushed to the data management platform.

[0007] Preferably, the step of building a high-precision image acquisition system to acquire standard sample images and test sample images of cigarette auxiliary materials, and performing multi-dimensional preprocessing and color space conversion to generate Lab standard sample images and Lab test sample images specifically includes: The high-precision image acquisition system is built by integrating hardware acquisition equipment and a precision linear testing platform. Based on the characteristics of the cigarette auxiliary materials, an adaptive acquisition scheme is used to acquire the standard sample image and the sample image to be tested, respectively. For the standard sample image and the sample image to be detected, the following steps are performed in sequence: denoising is performed using an adaptive Gaussian filtering algorithm; contrast enhancement is performed using a histogram equalization algorithm; and image size standardization is performed using a cubic convolution interpolation algorithm. The standard sample image and the sample image to be detected, after multi-dimensional preprocessing, are converted from the RGB color space to the XYZ color space using a linear transformation function. The corresponding transformation formula is as follows: In the formula, X, Y, and Z represent the stimulus values ​​of the X, Y, and Z color spaces, respectively; M represents the transformation matrix from the RGB color space to the XYZ color space. Using nonlinear transformation functions The standard sample image and the sample image to be detected are converted from the XYZ color space to the Lab color space to generate the Lab standard sample image and the Lab sample image to be detected. The corresponding conversion formulas are as follows: In the formula, The channel represents luminance; channel a represents chromaticity along the green-red axis; channel b represents chromaticity along the blue-yellow axis; x, y, and z represent the stimulus values ​​of the normalized X, Y, and Z color spaces, respectively. , , These represent the reference white point stimulus values ​​for the X, Y, and Z color spaces under the D65 standard light source, respectively; t represents the symbol for x, y, and z. This indicates the conversion threshold.

[0008] Preferably, the step of retrieving the cigarette auxiliary material sample image library and using data augmentation technology to expand the cigarette auxiliary material sample image library to generate a first expanded cigarette auxiliary material sample image library specifically includes: Retrieve the cigarette auxiliary material sample image library and read the cigarette auxiliary material sample images in the cigarette auxiliary material sample image library, and then perform a two-dimensional rotation matrix. The pixel space coordinates of the cigarette auxiliary material sample image Perform random rotation transformation to generate the corresponding pixel target coordinates The corresponding random rotation transformation formula is as follows: In the formula, Indicates the center coordinates of the cigarette auxiliary material sample image; Using a diagonal scaling matrix For the pixel target coordinates Perform random scaling transformation to generate the corresponding pixel floating-point coordinates The corresponding random scaling transformation formula is as follows: ; The cigarette auxiliary material sample image after random scaling transformation is reconstructed using bilinear interpolation to obtain the pixel floating-point coordinates. Corresponding smooth pixel value traverse the smoothed pixel values The image library of reconstructed cigarette auxiliary materials is generated, and the corresponding calculation formula is as follows: In the formula, This represents the floor function; m and n represent the summation indices in the horizontal and vertical directions, respectively. This indicates the weight of the pixel's floating-point coordinates in the horizontal direction; This indicates the weight of the pixel's floating-point coordinates in the vertical direction; Based on the reconstructed cigarette auxiliary material sample images in the reconstructed cigarette auxiliary material sample image library, a fixed pixel size effective cropping window is randomly selected in the reconstructed cigarette auxiliary material sample image, and the reconstructed cigarette auxiliary material sample image is randomly cropped and transformed to obtain an expanded cigarette auxiliary material sample image library. Data augmentation techniques are used to perform random color space perturbation and random noise addition operations on the expanded cigarette auxiliary material sample image library, and the first expanded cigarette auxiliary material sample image library is output.

[0009] Preferably, the step of further augmenting the first expanded cigarette auxiliary material sample image library using MixUp image blending technology to generate the second expanded cigarette auxiliary material sample image library specifically includes: Using the MixUp image blending technology, two sets of first expanded cigarette auxiliary material sample images are randomly selected from the first expanded cigarette auxiliary material sample image library. , and corresponding tag information , The image is then augmented using a linear weighting formula to generate a second extended sample image of cigarette auxiliary materials. and corresponding extended tags The corresponding calculation formula is as follows: In the formula, Represents the random mixing coefficients generated by the Beta distribution; Summarize the first expanded cigarette auxiliary material sample image library and the second expanded cigarette auxiliary material sample image library. and the corresponding extended tags The second expanded cigarette auxiliary material sample image library was obtained.

[0010] Preferably, training the convolutional neural network model using the second expanded cigarette auxiliary material sample image library specifically includes: The training environment was built based on the PyTorch deep learning framework, and the AdamW optimizer and cosine annealing learning rate scheduler were used to adjust the learning rate of the convolutional neural network model. The second extended cigarette auxiliary material sample image library is input into the convolutional neural network model, and the learning rate is used for learning and training. The training color difference value and training color difference category corresponding to the second extended cigarette auxiliary material sample image library are output. The mean squared error loss function and the cross-entropy loss function are used to perform loss prediction and training optimization on the training color difference values ​​and the training color difference categories; The 5-fold cross-validation method is used to validate and evaluate the optimized training color difference values ​​and the training color difference categories, and the validation and evaluation results are output. Based on the verification and evaluation results, the hyperparameters of the convolutional neural network model are adjusted accordingly, driving the convolutional neural network model to continuously train and optimize, and outputting the trained convolutional neural network model.

[0011] Preferably, the step of inputting the Lab standard sample image and the Lab sample image to be detected into the trained convolutional neural network model, and extracting and fusing features from the Lab standard sample image and the Lab sample image to be detected to generate Lab standard sample image features and Lab sample image features to be detected, respectively, specifically includes: The trained convolutional neural network model includes an image input layer, a feature extraction network, a feature fusion layer, and a classification output layer. The image input layer receives the Lab standard sample image and the Lab sample image to be detected, and performs Z-score normalization processing on them respectively. The Z-score normalized Lab standard sample image and the Lab sample image to be detected are then transmitted to the feature extraction network. The feature extraction network uses 3×3 and 1×1 convolutional kernels of the convolutional layer to perform sliding convolution operations on the Lab standard sample image and the Lab sample image to be detected after Z-score normalization, and combines the spatial attention mechanism module to extract features respectively, and outputs the corresponding standard multi-scale feature map sequence and the multi-scale feature map sequence to be detected. The feature fusion layer employs a multi-scale feature fusion strategy to spatially align the standard multi-scale feature map sequence, and then splices and fuses the spatially aligned standard multi-scale feature map sequence according to the channel dimension to generate the Lab standard sample image features. The feature fusion layer employs a multi-scale feature fusion strategy to spatially align the multi-scale feature map sequence to be detected, and then splices and fuses the spatially aligned multi-scale feature map sequence according to the channel dimension to generate the Lab sample image features to be detected.

[0012] Preferably, the feature extraction network uses 3×3 and 1×1 convolutional kernels of the convolutional layer to perform sliding convolution operations on the Z-score-normalized Lab standard sample image and the Z-score-normalized Lab sample image to be detected, respectively, and combines a spatial attention mechanism module to extract features, outputting a standard multi-scale feature map sequence corresponding to the Lab standard sample image and a detection multi-scale feature map sequence corresponding to the Lab sample image to be detected, specifically including: The feature extraction network includes convolutional layers, batch normalization layers, and activation functions; The detection color features and detection texture features of the Lab sample image after Z-score normalization are extracted using the 3×3 convolution kernel of the convolutional layer. Using the 3×3 convolution kernel and the 1×1 convolution kernel of the convolutional layer, combined with the spatial attention mechanism module, the intermediate semantic features and high-level abstract features of the Lab sample image to be detected after Z-score normalization are extracted; For the color features, texture features, intermediate semantic features, and high-level abstract features to be detected, the batch normalization layer is used for standardization and affine transformation, and the activation function is used for nonlinear mapping to output the multi-scale feature map sequence corresponding to the Lab sample image to be detected. The extraction method for the standard multi-scale feature map sequence corresponding to the Lab standard sample image is the same as above.

[0013] Preferably, the step of comparing the color difference between the Lab standard sample image features and the Lab sample image to be detected, generating a color difference detection result corresponding to the sample image to be detected, and pushing it to the data management platform specifically includes: The features of the Lab standard sample image and the features of the Lab sample image to be detected are input into the classification output layer of the trained convolutional neural network model, and the color difference defect value corresponding to the sample image to be detected is calculated using the CIELab color difference formula. The corresponding calculation formula is as follows: In the formula, This represents the brightness difference between the features of the Lab standard sample image and the features of the Lab sample image to be detected; This represents the difference in red and green color between the features of the Lab standard sample image and the features of the Lab sample image to be detected; The difference between the yellow and blue colors represents the difference between the features of the Lab standard sample image and the features of the Lab sample image to be detected. Based on the preset color difference defect threshold T according to the features of the Lab standard sample image, when the color difference defect value corresponding to the sample image to be detected is... If the color difference defect threshold T is greater than or equal to the threshold value, then the sample image to be detected is determined to have a color difference defect. When the sample image to be detected has color difference defects, the color difference defect values ​​are marked and summarized. For the corresponding pixels in the sample image to be detected, a defect mask image is generated. A contour detection algorithm is used to identify and extract the connected regions of the pixels in the defect mask image. The minimum bounding rectangle of the connected regions is calculated using the axis-aligned minimum bounding rectangle algorithm, and the vertex coordinates of the minimum bounding rectangle are output. , , and Used as coordinates for the color difference defect area; When the sample image to be detected has color difference defects, the spatial dimension of the Lab sample image features to be detected is mapped through the fully connected layer in the classification output layer to generate three-dimensional Lab sample image features to be detected. The probability distribution of color difference defect categories corresponding to the three-dimensional Lab sample image features is calculated using the Softmax activation function, and the color difference defect category with the highest probability value is selected as the color difference defect category of the sample image to be detected. Summarize the color difference defect values ​​corresponding to the sample images to be detected. The coordinates of the color difference defect area and the category of the color difference defect are used to generate the color difference detection result corresponding to the sample image to be detected; The basic information corresponding to the sample image to be tested and the color difference detection results corresponding to the sample image to be tested are associated and integrated to generate a color difference detection report corresponding to the sample image to be tested. Based on the coordinates of the color difference defect area, the color difference defect area is highlighted in the sample image to be tested. The color difference detection report corresponding to the sample image to be tested and the highlighted color difference defect area are pushed to the data management platform in real time.

[0014] An automated color difference recognition system for cigarette auxiliary materials based on convolutional neural networks, the system comprising: The image acquisition and processing module is used to build a high-precision image acquisition system to acquire standard sample images and test sample images of cigarette auxiliary materials, and to perform multi-dimensional preprocessing and color space conversion to generate Lab standard sample images and Lab test sample images. The image library generation and model training module is used to retrieve the cigarette auxiliary material sample image library, use data augmentation technology to expand the image library of cigarette auxiliary material sample image library to generate a first expanded cigarette auxiliary material sample image library, and then use MixUp image blending technology to augment the first expanded cigarette auxiliary material sample image library to generate a second expanded cigarette auxiliary material sample image library and train a convolutional neural network model. The feature extraction and color difference detection module is used to input the Lab standard sample image and the Lab sample image to be detected into the trained convolutional neural network model, extract and fuse features of the Lab standard sample image and the Lab sample image to be detected respectively, generate Lab standard sample image features and Lab sample image features to be detected, compare the color difference of the Lab standard sample image features and the Lab sample image features to be detected, generate the color difference detection result corresponding to the sample image to be detected, and push it to the data management platform.

[0015] Compared with existing technologies, the automated color difference recognition method and system for cigarette auxiliary materials based on convolutional neural networks provided by this invention has the following beneficial effects: This invention acquires standard sample images and test sample images of cigarette auxiliary materials by building a high-precision image acquisition system, and performs multi-dimensional preprocessing and color space conversion to generate Lab standard sample images and Lab test sample images. It then retrieves a cigarette auxiliary material sample image library, uses data augmentation technology to expand the image library to generate a first expanded cigarette auxiliary material sample image library, and further uses MixUp image blending technology to augment the first expanded cigarette auxiliary material sample image library to generate a second expanded cigarette auxiliary material sample image library, and trains a convolutional neural network model. Finally, it combines the Lab standard sample images and Lab test sample images... The sample images are input into a trained convolutional neural network model. Feature extraction and fusion are performed on the Lab standard sample images and the Lab sample images to be detected to generate Lab standard sample image features and Lab sample images to be detected features respectively. The color difference of the Lab standard sample image features and the Lab sample images to be detected features is compared to generate the color difference detection results corresponding to the sample images to be detected and pushed to the data management platform. This can improve the accuracy, stability and reliability of color difference detection of cigarette auxiliary materials, reduce the false negative rate and false positive rate of color difference detection of cigarette auxiliary materials, and ensure the production quality and brand image of cigarette auxiliary materials.

[0016] This invention addresses the problems of traditional manual visual color difference detection being heavily influenced by subjective experience and environmental factors by constructing a high-precision image acquisition system and performing multi-dimensional image preprocessing and Lab color space conversion. It converts the RGB color space to the Lab color space, capturing color information of cigarette auxiliary materials through Lab color space, accurately quantifying and effectively highlighting the color characteristics of these materials, thus overcoming the limitations of traditional machine vision detection methods that do not conform to human visual perception. Furthermore, this invention utilizes data augmentation technology combined with MixUp image blending technology to construct a first and a second extended sample image library of cigarette auxiliary materials and trains a convolutional neural network model, improving the feature learning ability and generalization performance of the convolutional neural network model, and enhancing its suitability for color difference recognition of cigarette auxiliary materials of different materials and processes. This invention effectively solves the problem of weak generalization ability caused by insufficient training samples and single data distribution in existing detection models. It uses a convolutional neural network model for feature extraction and fusion, and compares the features of Lab standard sample images and Lab test sample images to address the high false negative and false positive rates caused by traditional machine vision detection methods that only compare single color parameters and lack feature dimensions, thus improving the accuracy of identifying subtle color differences in cigarette auxiliary materials. Furthermore, by introducing a spatial attention mechanism module and a multi-scale feature fusion strategy, this invention deeply extracts features from Lab standard sample images and Lab test sample images, significantly improving the accuracy of color difference detection in cigarette auxiliary materials and its robustness in complex textures and scenes, comprehensively enhancing the quality control and process improvement effects throughout the entire cigarette auxiliary material production process. Attached Figure Description

[0017] Figure 1 A flowchart of an automated color difference recognition method for cigarette auxiliary materials based on a convolutional neural network, provided in an embodiment of the present invention; Figure 2 A system block diagram of an automated color difference recognition system for cigarette auxiliary materials based on a convolutional neural network, provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] like Figure 1 The diagram shown is a flowchart of an automated color difference recognition method for cigarette auxiliary materials based on a convolutional neural network, provided in an embodiment of the present invention. Figure 1 The execution entity of the method shown can be a software and / or hardware device. The execution entity of this application can include, but is not limited to, at least one of the following: user equipment, network equipment, etc. User equipment can include, but is not limited to, computers, smartphones, personal digital assistants (PDAs), and the aforementioned electronic devices. Network equipment can include, but is not limited to, a single network server, a server group consisting of multiple network servers, or a cloud based on cloud computing consisting of a large number of computers or network servers. Cloud computing is a type of distributed computing, consisting of a super virtual computer composed of a group of loosely coupled computers. This embodiment does not limit this. Steps S1 to S3 are detailed as follows: S1. Build a high-precision image acquisition system to acquire standard sample images and test sample images of cigarette auxiliary materials, and perform multi-dimensional preprocessing and color space conversion to generate Lab standard sample images and Lab test sample images. Cigarette auxiliary materials refer to various auxiliary materials used in cigarette production for cigarette forming, packaging, labeling, and ensuring a good user experience. These include label paper and tipping paper with complex surface treatments, such as hot stamping, embossing, laser engraving, metallization transfer, and water-based ink printing, showcasing unique material and texture characteristics. The high-precision image acquisition system integrates hardware acquisition equipment and a precision linear testing platform for digitally acquiring real-world image information of cigarette auxiliary materials. Standard sample images are high-resolution images of defect-free cigarette auxiliary material surfaces that conform to design standards, serving as a benchmark for color difference detection. Test sample images are high-resolution images of the cigarette auxiliary material surfaces acquired in real-time. Lab standard sample images and Lab test sample images are images that, after color space conversion, accurately reflect the characteristics of human visual perception; their brightness and chromaticity separation significantly improve the accuracy of color difference detection.

[0020] Understandably, multi-dimensional preprocessing can eliminate interference factors such as noise, uneven lighting, and size differences during the acquisition process. The traditional RGB color space does not conform to the characteristics of human visual perception and is difficult to accurately quantify subtle color differences. Converting the RGB color space to the Lab color space can convert device-related RGB values ​​into device-independent and perceptibly uniform Lab values, thereby solving the problem of the disconnect between the RGB color space and human visual perception.

[0021] The construction of a high-precision image acquisition system to acquire standard sample images and test sample images of cigarette auxiliary materials, and to perform multi-dimensional preprocessing and color space conversion to generate Lab standard sample images and Lab test sample images, specifically includes: The high-precision image acquisition system is built by integrating hardware acquisition equipment and a precision linear testing platform. Based on the characteristics of the cigarette auxiliary materials, an adaptive acquisition scheme is used to acquire the standard sample image and the sample image to be tested, respectively. For the standard sample image and the sample image to be detected, the following steps are performed in sequence: denoising is performed using an adaptive Gaussian filtering algorithm; contrast enhancement is performed using a histogram equalization algorithm; and image size standardization is performed using a cubic convolution interpolation algorithm. The standard sample image and the sample image to be detected, after multi-dimensional preprocessing, are converted from the RGB color space to the XYZ color space using a linear transformation function. The corresponding transformation formula is as follows: In the formula, X, Y, and Z represent the stimulus values ​​of the X, Y, and Z color spaces, respectively; M represents the transformation matrix from the RGB color space to the XYZ color space. Using nonlinear transformation functions The standard sample image and the sample image to be detected are converted from the XYZ color space to the Lab color space to generate the Lab standard sample image and the Lab sample image to be detected. The corresponding conversion formulas are as follows: In the formula, The channel represents luminance; channel a represents chromaticity along the green-red axis; channel b represents chromaticity along the blue-yellow axis; x, y, and z represent the stimulus values ​​of the normalized X, Y, and Z color spaces, respectively. , , These represent the reference white point stimulus values ​​for the X, Y, and Z color spaces under the D65 standard light source, respectively; t represents the symbol for x, y, and z. This indicates the conversion threshold.

[0022] First, an 8K resolution CMOS industrial line scan camera with a pixel size of 7.04µm × 7.04µm and a linear velocity of 48kHz was selected, paired with a low-distortion optical lens with a distortion rate of less than 0.1% and a high-speed image acquisition card with a bandwidth of 10Gbps to ensure the clarity and detail reproduction of the acquired standard sample images and the images to be tested. Simultaneously, a high CRI LED tunnel light source with a color temperature of 6000K and a color rendering index greater than or equal to 95 was used. Its effective flicker lifespan is no less than 20,000 hours, and its light decay is less than 30% after 20,000 hours. The power supply voltage is DC24V. By adjusting the incident angle of the light source to the range of 45°-60° and using a polarizer, specular reflections and shadows from highly reflective auxiliary materials such as hot stamping and aluminum plating transfers are effectively avoided to prevent color distortion in the acquired standard sample images and the images to be tested. A precision linear testing platform was built simultaneously, using a stepper motor drive and equipped with a linear encoder, achieving a positioning accuracy of ±0.01mm. This ensures that the cigarette auxiliary material moves smoothly at a constant speed during the acquisition process, guaranteeing the continuity and consistency of the acquisition of standard sample images and images of the sample to be tested, and avoiding motion blur and positional shift.

[0023] Then, an adaptive acquisition scheme was adopted for different types of cigarette accessories, such as printed tipping paper, hot stamping label paper, and special tipping paper. For printed tipping paper, the camera exposure time was set to 50µs and the gain to 6dB. For highly reflective cigarette accessories such as hot stamping label paper, the exposure time was shortened to 30µs and the polarizer angle was increased to 90° to suppress reflection. For dark-colored special tipping paper, the gain was increased to 12dB and multi-frame averaging noise reduction was used. The corresponding acquisition parameters were automatically adjusted for different color series of cigarette accessories such as red, blue, and gold to ensure that standard sample images and test sample images were acquired under exactly the same lighting, exposure, and white balance conditions, avoiding color deviation caused by differences in acquisition parameters.

[0024] Subsequently, the acquired standard sample images and the images to be tested underwent multi-dimensional preprocessing. First, an adaptive Gaussian filtering algorithm was used for denoising. Based on the noise distribution characteristics of the standard and test sample images, the filter kernel size was dynamically adjusted to remove Gaussian and salt-and-pepper noise, while fully preserving color features and texture information such as hot stamping edges and embossing details on the surfaces of the standard and test sample images. Second, a histogram equalization algorithm was used to enhance the contrast of the standard and test sample images. The gray-level histogram of the standard sample image was calculated and the pixel distribution was adjusted. Simultaneously, the gray-level histogram of the test sample image was calculated and the pixel distribution was adjusted to address uneven brightness and highlight color differences. Third, a cubic convolution interpolation algorithm was used for size standardization, uniformly adjusting the standard and test sample images to a resolution of 8192×4000 pixels to eliminate the influence of scale differences on color difference determination.

[0025] It's important to note that the RGB color space cannot be directly converted to the Lab color space. The XYZ color space is needed; first, the RGB color space is converted to the XYZ color space, and then the XYZ color space is converted back to the Lab color space. Because the components of the RGB color space are highly correlated, and the quantification of color differences does not conform to the characteristics of human visual perception, a linear transformation function is first used to convert the standard sample image and the sample image to be tested from the RGB color space to the XYZ color space, respectively, to initially standardize the color information and eliminate color deviations caused by device acquisition. Then, a non-linear transformation function is used to convert the standard sample image and the sample image to be tested from the XYZ color space to the Lab color space, respectively, to separate the luminance (L channel) and chromaticity (a and b channels). The changes in the values ​​of the a and b channels directly correspond to color differences perceptible to the human eye, conforming to human visual habits.

[0026] Understandably, combining a high-precision image acquisition system with an adaptive acquisition scheme effectively solves the problems of color distortion caused by reflections, shadows, and parameter mismatches in existing acquisition technologies. Multi-dimensional preprocessing eliminates interference factors such as noise, uneven brightness, and size differences, improving the consistency between standard sample images and images to be tested. Through the conversion from RGB color space to XYZ color space and then to Lab color space, the technical problems of inaccurate color quantization, lack of color feature prominence under complex textures, and poor color consistency under lighting interference in the traditional RGB color space are effectively solved.

[0027] S2, retrieve the cigarette auxiliary material sample image library, use data augmentation technology to expand the cigarette auxiliary material sample image library to generate a first expanded cigarette auxiliary material sample image library, and then use MixUp image blending technology to augment the first expanded cigarette auxiliary material sample image library to generate a second expanded cigarette auxiliary material sample image library and train a convolutional neural network model. The cigarette auxiliary material sample image library refers to a collection of historical quality inspection cigarette auxiliary material sample images, including labeled cigarette auxiliary material images. These images cover cigarette auxiliary materials such as hot stamping tipping paper, aluminized transfer label paper, and water-based ink printing paper, as well as color difference defect types such as hue deviation, brightness deviation, saturation deviation, and misregistration. Each cigarette auxiliary material sample image is associated with label information such as cigarette auxiliary material type, surface treatment process, coordinates of color difference defect area, color difference defect value, and color difference defect category. Data augmentation technology refers to image processing methods that expand the cigarette auxiliary material sample image library without changing the semantics of the cigarette auxiliary material sample images. These methods include random geometric transformations, random color space transformations, and random noise addition to address the problems of scarce and unevenly distributed cigarette auxiliary material sample images. Random geometric transformations include random rotation transformations, random scaling transformations, and random cropping transformations. MixUp image blending technology refers to generating a virtual second extended cigarette auxiliary material sample image by linearly interpolating and mixing two sets of first extended cigarette auxiliary material sample images and their corresponding label information. The first expanded cigarette accessory sample image library refers to a collection of cigarette accessory sample images that has been generated by performing operations such as random rotation transformation, random scaling transformation, random cropping transformation, random color space perturbation, and random noise addition on the cigarette accessory sample image library, resulting in a collection with richer sample diversity and a wider distribution coverage. The second expanded cigarette accessory sample image library refers to a collection of cigarette accessory sample images that is generated by fusing features of the first expanded cigarette accessory sample images using MixUp image blending technology and performing image augmentation. This collection includes the first expanded cigarette accessory sample image library, the second expanded cigarette accessory sample images, and corresponding expanded labels, and is used to further improve the generalization ability of convolutional neural network models.

[0028] Understandably, the limited number of cigarette auxiliary material sample images in the cigarette auxiliary material sample image library, the singular data distribution, and the imbalance of defect samples make it difficult to cover the various morphological changes in cigarette auxiliary material sample images caused by interference factors such as placement angle, shooting distance, and lighting fluctuations in actual production scenarios. It is necessary to use data augmentation technology to simulate various morphological changes in actual color difference detection, thereby expanding the cigarette auxiliary material sample image library and generating the first expanded cigarette auxiliary material sample image library. At the same time, by using MixUp image blending technology to fuse the features of the first expanded cigarette auxiliary material sample images and generate new feature combinations, the ability of the convolutional neural network model to identify subtle color differences and mixed defects is enhanced. This ensures that the convolutional neural network model can stably learn effective features, avoid overfitting or underfitting, and ultimately achieve high-precision color difference recognition.

[0029] The process of retrieving the cigarette auxiliary material sample image library involves using data augmentation technology to expand the image library and generate a first expanded cigarette auxiliary material sample image library. Specifically, this includes: Retrieve the cigarette auxiliary material sample image library and read the cigarette auxiliary material sample images in the cigarette auxiliary material sample image library, and then perform a two-dimensional rotation matrix. The pixel space coordinates of the cigarette auxiliary material sample image Perform random rotation transformation to generate the corresponding pixel target coordinates The corresponding random rotation transformation formula is as follows: In the formula, Indicates the center coordinates of the cigarette auxiliary material sample image; Using a diagonal scaling matrix For the pixel target coordinates Perform random scaling transformation to generate the corresponding pixel floating-point coordinates The corresponding random scaling transformation formula is as follows: ; The cigarette auxiliary material sample image after random scaling transformation is reconstructed using bilinear interpolation to obtain the pixel floating-point coordinates. Corresponding smooth pixel value traverse the smoothed pixel values The image library of reconstructed cigarette auxiliary materials is generated, and the corresponding calculation formula is as follows: In the formula, This represents the floor function; m and n represent the summation indices in the horizontal and vertical directions, respectively. This indicates the weight of the pixel's floating-point coordinates in the horizontal direction; This indicates the weight of the pixel's floating-point coordinates in the vertical direction; Based on the reconstructed cigarette auxiliary material sample images in the reconstructed cigarette auxiliary material sample image library, a fixed pixel size effective cropping window is randomly selected in the reconstructed cigarette auxiliary material sample image, and the reconstructed cigarette auxiliary material sample image is randomly cropped and transformed to obtain an expanded cigarette auxiliary material sample image library. Data augmentation techniques are used to perform random color space perturbation and random noise addition operations on the expanded cigarette auxiliary material sample image library, and the first expanded cigarette auxiliary material sample image library is output.

[0030] The step of further augmenting the first expanded cigarette auxiliary material sample image library using MixUp image blending technology to generate a second expanded cigarette auxiliary material sample image library specifically includes: Using the MixUp image blending technology, two sets of first expanded cigarette auxiliary material sample images are randomly selected from the first expanded cigarette auxiliary material sample image library. , and corresponding tag information , The image is then augmented using a linear weighting formula to generate a second extended sample image of cigarette auxiliary materials. and corresponding extended tags The corresponding calculation formula is as follows: In the formula, Represents the random mixing coefficients generated by the Beta distribution; Summarize the first expanded cigarette auxiliary material sample image library and the second expanded cigarette auxiliary material sample image library. and the corresponding extended tags The second expanded cigarette auxiliary material sample image library was obtained.

[0031] Training the convolutional neural network model using the second expanded cigarette accessory sample image library specifically includes: The training environment was built based on the PyTorch deep learning framework, and the AdamW optimizer and cosine annealing learning rate scheduler were used to adjust the learning rate of the convolutional neural network model. The second extended cigarette auxiliary material sample image library is input into the convolutional neural network model, and the learning rate is used for learning and training. The training color difference value and training color difference category corresponding to the second extended cigarette auxiliary material sample image library are output. The mean squared error loss function and the cross-entropy loss function are used to perform loss prediction and training optimization on the training color difference values ​​and the training color difference categories; The 5-fold cross-validation method is used to validate and evaluate the optimized training color difference values ​​and the training color difference categories, and the validation and evaluation results are output. Based on the verification and evaluation results, the hyperparameters of the convolutional neural network model are adjusted accordingly, driving the convolutional neural network model to continuously train and optimize, and outputting the trained convolutional neural network model.

[0032] The label information refers to the cigarette auxiliary material type, surface treatment process, coordinates of the color difference defect area, color difference defect value, and color difference defect category corresponding to the first extended cigarette auxiliary material sample image. The extended label refers to the information generated by linearly weighting and combining the label information of the two sets of first extended cigarette auxiliary material sample images using MixUp image mixing technology. This information is used to characterize the cigarette auxiliary material type, surface treatment process, coordinates of the color difference defect area, color difference defect value, and color difference defect category corresponding to the second extended cigarette auxiliary material sample image. It helps the convolutional neural network model learn continuous and smooth feature distributions, enhancing its high-precision recognition capability for subtle and complex color difference defects.

[0033] Specifically, a two-dimensional rotation matrix is ​​used to randomly rotate the pixel space coordinates of the cigarette auxiliary material sample image to simulate the arbitrary angular displacement of the cigarette auxiliary material sample image during actual color difference detection. The two-dimensional rotation matrix... Rotation angle in The random value range is 0°-360°. Setting the rotation center to the center coordinates of the cigarette auxiliary material sample image ensures the integrity of the cigarette auxiliary material sample image after random rotation transformation. A diagonal scaling matrix is ​​used to randomly scale the pixel target coordinates after random rotation transformation to simulate the scale difference caused by changes in the acquisition distance of the cigarette auxiliary material sample image. The diagonal scaling matrix... The scaling factor 's' in the image ranges randomly from 0.8 to 1.2, which can cover slight near-far distortion. After random rotation and scaling transformations, the pixel target coordinates of the cigarette auxiliary material sample image become floating-point pixel coordinates. At this point, bilinear interpolation is used for interpolation reconstruction. By calculating the weighted average of the four pixels adjacent to the floating-point pixel coordinate, a smooth pixel value is obtained, ensuring that the edges of the reconstructed cigarette auxiliary material sample image are smooth and the color transitions are natural. The original color distribution pattern and fine texture details such as embossing and hot stamping edges on the surface of the cigarette auxiliary material sample image are preserved, generating a reconstructed cigarette auxiliary material sample image library.

[0034] Furthermore, a fixed-pixel-size effective cropping window, such as a 1024×1024 pixel effective cropping window, is randomly selected from the reconstructed cigarette auxiliary material sample image. This random cropping transformation not only allows the convolutional neural network model to focus on the local color features of the reconstructed cigarette auxiliary material sample image during training but also reduces the size of the reconstructed cigarette auxiliary material sample image, thus adapting to the input requirements of the convolutional neural network model and obtaining an expanded cigarette auxiliary material sample image library. Data augmentation techniques are employed to perform random color space perturbation operations. By fine-tuning the values ​​of the L, a, and b channels in the Lab color space, the effects of illumination changes and ambient color temperature fluctuations are simulated. Simultaneously, random noise addition operations are performed, injecting Gaussian noise or salt-and-pepper noise into the expanded cigarette auxiliary material sample images to simulate real noise sources such as electronic noise from the acquisition device and environmental interference. This results in a first expanded cigarette auxiliary material sample image library with diversity and robustness.

[0035] Furthermore, by using MixUp image blending technology, two sets of first extended cigarette auxiliary material sample images and corresponding label information are randomly selected from the first extended cigarette auxiliary material sample image library. Then, a pixel-level linear weighted combination is performed using random blending coefficients generated by Beta distribution to generate a virtual second extended cigarette auxiliary material sample image and corresponding extended label. This effectively expands the feature space of the first extended cigarette auxiliary material sample image library, solves the problem of imbalance in the first extended cigarette auxiliary material sample image library, and ensures that the convolutional neural network model can not only learn the original features of the first extended cigarette auxiliary material sample images, but also fully learn the new features of the virtual mixed second extended cigarette auxiliary material sample images.

[0036] In practical applications, MixUp image blending technology can generate a large number of virtual second extended cigarette auxiliary material sample images that approximate the real distribution, along with corresponding extended labels, significantly improving the generalization ability and robustness of convolutional neural network models.

[0037] Then, a training environment was built using the PyTorch deep learning framework, employing a computing server equipped with an Intel 19th generation i7 or higher processor, at least 16GB of RAM, and an RTX 3060 or higher GPU. The parallel computing capabilities of the GPU significantly shortened the training time of the convolutional neural network model, meeting training requirements. The AdamW optimizer was used, incorporating a weight decay coefficient to effectively suppress overfitting in the convolutional neural network model. Combined with a cosine annealing learning rate scheduler, the initial learning rate of 0.001 was periodically adjusted to a minimum learning rate of 0.0001 according to a cosine function curve. A higher learning rate was used in the early stages of training to accelerate the convergence of the convolutional neural network model, while the learning rate was reduced in the later stages to finely optimize the model parameters. The mean squared error loss function was used to predict the deviation between the trained color difference values ​​and the true color difference values; the cross-entropy loss function was used for color difference category classification, accurately quantifying the deviation between the trained and true color difference categories. By setting the weight ratio of the mean squared error loss function and the cross-entropy loss function to 1:1, the regression and classification performance of the convolutional neural network model is optimized, which can accurately calculate the training color difference value and accurately determine the training color difference category.

[0038] Finally, a 5-fold cross-validation method was used to randomly divide the second extended cigarette auxiliary material sample image library into 5 mutually exclusive subsets. Each time, 4 subsets were selected as the training set and the remaining subset as the validation set, with this process repeated 5 times to complete the validation evaluation. This comprehensively assesses the generalization ability of the convolutional neural network model on different subsets. The validation set was used to evaluate the performance of the convolutional neural network model and output the validation evaluation results, including the mean absolute error of the training color difference values, the root mean square error of the training color difference values, and the accuracy, recall, F1 score, and confusion matrix of the training color difference category classification. Based on the validation evaluation results, hyperparameters such as the number of convolutional kernels, learning rate, and regularization coefficient were adjusted. For example, if the prediction error of the training color difference values ​​was large in the validation evaluation results, the number of convolutional kernels was appropriately increased to improve the feature extraction ability of the convolutional neural network model; if overfitting occurred, the regularization coefficient was increased or the learning rate was decreased to drive the convolutional neural network model to continuously train and optimize until the validation performance no longer improved. The final output was a well-trained convolutional neural network model with high accuracy and robustness.

[0039] S3, input the Lab standard sample image and the Lab sample image to be detected into the trained convolutional neural network model, extract and fuse features of the Lab standard sample image and the Lab sample image to be detected respectively to generate Lab standard sample image features and Lab sample image to be detected features, compare the color difference of the Lab standard sample image features and the Lab sample image to be detected features, generate the color difference detection result corresponding to the sample image to be detected and push it to the data management platform.

[0040] The trained convolutional neural network model refers to a deep learning model based on the PyTorch deep learning framework, obtained through iterative training and hyperparameter optimization using a second extended cigarette auxiliary material sample image library. This model possesses high-precision color difference recognition capabilities and strong generalization performance. It includes an image input layer, a feature extraction network, a feature fusion layer, and a classification output layer, achieving end-to-end color difference detection. Lab standard sample image features refer to high-dimensional features, processed by the feature extraction network and feature fusion layer, that represent the color, texture, and high-level semantic information of the standard sample image. Lab sample image features refer to high-dimensional features, processed by the feature extraction network and feature fusion layer, that represent the color, texture, and high-level semantic information of the sample image to be detected. The color difference detection result refers to a structured dataset containing the color difference defect values, coordinates of the color difference defect regions, and color difference defect categories of the sample image to be detected. This dataset is used to quantitatively evaluate the color difference defects between the sample image to be detected and the standard sample image and to locate the positions of the color difference defects.

[0041] Understandably, by using a convolutional neural network model for deep feature extraction and multi-scale fusion, the color details and texture correlation features are fully explored. Combined with the calculation of color difference defect values, the location of color difference defect area coordinates, and the determination of color difference defect categories, the entire process from feature learning to accurate judgment and then to the output of color difference detection results is automated, solving the problems of low recognition accuracy and high false detection rate of traditional methods.

[0042] The step of inputting the Lab standard sample image and the Lab sample image to be detected into the trained convolutional neural network model, and extracting and fusing features from the Lab standard sample image and the Lab sample image to be detected to generate Lab standard sample image features and Lab sample image features to be detected, specifically includes: The trained convolutional neural network model includes an image input layer, a feature extraction network, a feature fusion layer, and a classification output layer. The image input layer receives the Lab standard sample image and the Lab sample image to be detected, and performs Z-score normalization processing on them respectively. The Z-score normalized Lab standard sample image and the Lab sample image to be detected are then transmitted to the feature extraction network. The feature extraction network uses 3×3 and 1×1 convolutional kernels of the convolutional layer to perform sliding convolution operations on the Lab standard sample image and the Lab sample image to be detected after Z-score normalization, and combines the spatial attention mechanism module to extract features respectively, and outputs the corresponding standard multi-scale feature map sequence and the multi-scale feature map sequence to be detected. The feature fusion layer employs a multi-scale feature fusion strategy to spatially align the standard multi-scale feature map sequence, and then splices and fuses the spatially aligned standard multi-scale feature map sequence according to the channel dimension to generate the Lab standard sample image features. The feature fusion layer employs a multi-scale feature fusion strategy to spatially align the multi-scale feature map sequence to be detected, and then splices and fuses the spatially aligned multi-scale feature map sequence according to the channel dimension to generate the Lab sample image features to be detected.

[0043] The feature extraction network utilizes 3×3 and 1×1 convolutional kernels of the convolutional layers to perform sliding convolution operations on the Z-score-normalized Lab standard sample image and the Z-score-normalized Lab sample image to be detected, respectively. It then combines this with a spatial attention mechanism module for feature extraction, outputting a standard multi-scale feature map sequence corresponding to the Lab standard sample image and a detection multi-scale feature map sequence corresponding to the Lab sample image to be detected. Specifically, this includes: The feature extraction network includes convolutional layers, batch normalization layers, and activation functions; The detection color features and detection texture features of the Lab sample image after Z-score normalization are extracted using the 3×3 convolution kernel of the convolutional layer. Using the 3×3 convolution kernel and the 1×1 convolution kernel of the convolutional layer, combined with the spatial attention mechanism module, the intermediate semantic features and high-level abstract features of the Lab sample image to be detected after Z-score normalization are extracted; For the color features, texture features, intermediate semantic features, and high-level abstract features to be detected, the batch normalization layer is used for standardization and affine transformation, and the activation function is used for nonlinear mapping to output the multi-scale feature map sequence corresponding to the Lab sample image to be detected. The extraction method for the standard multi-scale feature map sequence corresponding to the Lab standard sample image is the same as above.

[0044] First, the Lab standard sample image and the Lab sample image to be detected are received through the image input layer and Z-score normalization is performed on them respectively. That is, the pixel mean and pixel standard deviation of the three channels in the Lab color space are calculated to eliminate the magnitude difference of the pixel values ​​of the three channels in the Lab color space.

[0045] Then, a feature extraction network is used to extract features from the Lab standard sample images and the Lab test sample images. This feature extraction network consists of multiple sequential convolutional blocks, each containing a convolutional layer, a batch normalization layer, and an activation function. The convolutional layers use 3×3 convolutional kernels with a stride of 1 to extract color and texture features, generating low-level feature maps, and then use 1×1 convolutional kernels for channel dimension reduction. A 3×3 convolutional kernel with a stride of 2 is used to expand the receptive field of the convolutional kernel, and the number of channels is gradually increased using 1×1 convolutional kernels to hierarchically extract intermediate semantic features and generate intermediate feature maps. Downsampling is performed using 3×3 convolutional kernels with a stride of 2, and the number of channels is further increased by combining 1×1 convolutional kernels. Simultaneously, a spatial attention mechanism module is embedded to automatically focus on features in areas of significant color difference and suppress features in background areas, accurately capturing high-level abstract features and generating high-level feature maps.

[0046] Finally, a batch normalization layer is used to unify the color and texture features in the low-level feature map, the intermediate semantic features in the intermediate feature map, and the high-level abstract features in the high-level feature map to a standard normal distribution, resulting in normalized color features, normalized texture features, normalized intermediate semantic features, and normalized high-level abstract features. This eliminates the internal covariate bias of different features and accelerates the convergence of the convolutional neural network model. Then, an affine transformation is used to introduce scaling and offset factors to adaptively adjust the normalized color features, normalized texture features, normalized intermediate semantic features, and normalized high-level abstract features, strengthening the discriminative information of different features and adapting to the personalized needs of different features. Finally, the LeakyReLU activation function is used to learn complex feature combination logic, mapping the standardized and affine-transformed color features, texture features, intermediate semantic features, and high-level abstract features into discriminative nonlinear features. The low-level, intermediate, and high-level feature maps after standardization, affine transformation, and nonlinear mapping are hierarchically summarized to obtain the detection multi-scale feature map sequence and the standard multi-scale feature map sequence, respectively.

[0047] The step of comparing the color difference between the Lab standard sample image features and the Lab sample image to be detected, generating the color difference detection result corresponding to the sample image to be detected, and pushing it to the data management platform specifically includes: The features of the Lab standard sample image and the features of the Lab sample image to be detected are input into the classification output layer of the trained convolutional neural network model, and the color difference defect value corresponding to the sample image to be detected is calculated using the CIELab color difference formula. The corresponding calculation formula is as follows: In the formula, This represents the brightness difference between the features of the Lab standard sample image and the features of the Lab sample image to be detected; This represents the difference in red and green color between the features of the Lab standard sample image and the features of the Lab sample image to be detected; The difference between the yellow and blue colors represents the difference between the features of the Lab standard sample image and the features of the Lab sample image to be detected. Based on the preset color difference defect threshold T according to the features of the Lab standard sample image, when the color difference defect value corresponding to the sample image to be detected is... If the color difference defect threshold T is greater than or equal to the threshold value, then the sample image to be detected is determined to have a color difference defect. When the sample image to be detected has color difference defects, the color difference defect values ​​are marked and summarized. For the corresponding pixels in the sample image to be detected, a defect mask image is generated. A contour detection algorithm is used to identify and extract the connected regions of the pixels in the defect mask image. The minimum bounding rectangle of the connected regions is calculated using the axis-aligned minimum bounding rectangle algorithm, and the vertex coordinates of the minimum bounding rectangle are output. , , and Used as coordinates for the color difference defect area; When the sample image to be detected has color difference defects, the spatial dimension of the Lab sample image features to be detected is mapped through the fully connected layer in the classification output layer to generate three-dimensional Lab sample image features to be detected. The probability distribution of color difference defect categories corresponding to the three-dimensional Lab sample image features is calculated using the Softmax activation function, and the color difference defect category with the highest probability value is selected as the color difference defect category of the sample image to be detected. Summarize the color difference defect values ​​corresponding to the sample images to be detected. The coordinates of the color difference defect area and the category of the color difference defect are used to generate the color difference detection result corresponding to the sample image to be detected; The basic information corresponding to the sample image to be tested and the color difference detection results corresponding to the sample image to be tested are associated and integrated to generate a color difference detection report corresponding to the sample image to be tested. Based on the coordinates of the color difference defect area, the color difference defect area is highlighted in the sample image to be tested. The color difference detection report corresponding to the sample image to be tested and the highlighted color difference defect area are pushed to the data management platform in real time.

[0048] Among them, the color difference defect value refers to an index that quantifies the difference in color between the sample image to be tested and the standard sample image, and is used to directly reflect the sensitivity of the human eye to color differences. The color difference defect threshold is the critical value for determining whether there is a color difference defect in the sample image to be tested, which can effectively balance the accuracy of color difference detection and the false detection rate. The basic information corresponding to the sample image to be tested includes the type of cigarette auxiliary material, production batch number, color difference detection time, and acquisition equipment number.

[0049] Specifically, when a color difference defect exists in the sample image to be detected, the classification output layer immediately locates the coordinates of the color difference defect region. First, it iterates through all pixels in the sample image, marking pixels with color difference defect values ​​greater than or equal to a color difference defect threshold as white, and pixels with color difference defect values ​​less than the threshold as black. The white-marked pixels are then used to obtain the defect mask image. A contour detection algorithm is used to identify connected regions of pixels in the defect mask image, filtering and extracting continuous connected regions, and automatically removing small noise interference. For the extracted connected regions, the minimum bounding rectangle is accurately calculated using an axis-aligned minimum bounding rectangle algorithm, and the vertex coordinates of this minimum bounding rectangle are output as the coordinates of the color difference defect region, with the coordinate unit being pixels, thus accurately selecting the color difference defect region.

[0050] Simultaneously, the features of the Lab sample image to be detected are mapped to three-dimensional space through a fully connected layer in the classification output layer, generating three-dimensional Lab sample image features that correspond to color difference defect categories such as hue deviation, brightness deviation, and saturation deviation. For example, hue deviation corresponds to the three-dimensional Lab sample image features (0.85, 0.12, 0.03). After normalization using the Softmax function, the probability distribution of each color difference defect category is obtained. The color difference defect category with the highest probability value is selected as the color difference defect category of the sample image to be detected, thus realizing the mapping from feature space to category space and improving the objectivity and accuracy of color difference defect category classification.

[0051] In practical applications, if the color difference defect value of the sample image to be detected is 4.2, which is greater than the color difference defect threshold, then the sample image to be detected is determined to have a color difference defect, and the corresponding yellow-blue pixels are marked as white, generating a mask image of the sample defect. Using the contour detection algorithm and the axis-aligned minimum bounding rectangle algorithm, the coordinates of the color difference defect region are output as (1250, 800), (1380, 800), (1250, 920), and (1380, 920). The 3D Lab sample image features generated after spatial dimension mapping through a fully connected layer are input into the Softmax activation function, which outputs the probability distribution of color difference defect categories. The color difference defect category with the highest probability value is selected as hue deviation - yellow-blue, and the color difference defect is then classified as... The phase deviation - yellowish-blue is used as the color difference defect category of the sample image to be tested. Finally, the cigarette auxiliary material type, production batch number, color difference detection time, acquisition equipment number, color difference defect value of 4.2, color difference defect area coordinates (1250,800), (1380,800), (1250,920), (1380,920), and color difference defect category hue deviation - yellowish-blue of the sample image to be tested are integrated to generate a color difference detection report. The color difference defect area is highlighted with a yellow mask in the sample image to be tested and pushed to the data management platform in real time. At the same time, a rejection instruction is sent to the controller of the production line and the unqualified cigarette auxiliary material is automatically rejected to prevent it from flowing into the subsequent packaging process and to ensure the uniformity of cigarette auxiliary material product quality.

[0052] like Figure 2 The diagram shown is a system block diagram of an automated color difference recognition system for cigarette auxiliary materials based on a convolutional neural network, provided in an embodiment of the present invention. The system includes: The image acquisition and processing module is used to build a high-precision image acquisition system to acquire standard sample images and test sample images of cigarette auxiliary materials, and to perform multi-dimensional preprocessing and color space conversion to generate Lab standard sample images and Lab test sample images. The image library generation and model training module is used to retrieve the cigarette auxiliary material sample image library, use data augmentation technology to expand the image library of cigarette auxiliary material sample image library to generate a first expanded cigarette auxiliary material sample image library, and then use MixUp image blending technology to augment the first expanded cigarette auxiliary material sample image library to generate a second expanded cigarette auxiliary material sample image library and train a convolutional neural network model. The feature extraction and color difference detection module is used to input the Lab standard sample image and the Lab sample image to be detected into the trained convolutional neural network model, extract and fuse features of the Lab standard sample image and the Lab sample image to be detected respectively, generate Lab standard sample image features and Lab sample image features to be detected, compare the color difference of the Lab standard sample image features and the Lab sample image features to be detected, generate the color difference detection result corresponding to the sample image to be detected, and push it to the data management platform.

[0053] Figure 2 The system of the illustrated embodiment can be used to perform corresponding operations. Figure 1 The steps in the method embodiments shown are implemented in a similar manner and have similar technical effects, and will not be repeated here.

[0054] An electronic device includes a memory and a processor, wherein the memory stores a computer program, and when the processor runs the computer program stored in the memory, the processor performs the steps of the automated color difference recognition method for cigarette auxiliary materials based on convolutional neural networks as described above.

[0055] like Figure 3 The diagram shown is a hardware structure schematic of an electronic device according to an embodiment of the present invention. The electronic device 30 includes: a processor 31, a memory 32, and a computer program; wherein... The memory 32 is used to store the computer program, and the memory may also be flash memory. The computer program is, for example, an application program or functional module that implements the above method.

[0056] Processor 31 is configured to execute the computer program stored in the memory to implement the various steps performed by the device in the above method. For details, please refer to the relevant descriptions in the preceding method embodiments.

[0057] Alternatively, the memory 32 can be either standalone or integrated with the processor 31.

[0058] When the memory 32 is a device independent of the processor 31, the device may further include: Bus 33 is used to connect the memory 32 and the processor 31.

[0059] A readable storage medium storing a computer program, which, when executed by a processor, is used to implement the steps of the automated color difference recognition method for cigarette auxiliary materials based on a convolutional neural network as described above.

[0060] The readable storage medium can be a computer storage medium or a communication medium. A communication medium includes any medium that facilitates the transfer of computer programs from one location to another. A computer storage medium can be any available medium accessible to a general-purpose or special-purpose computer. For example, a readable storage medium is coupled to a processor, enabling the processor to read information from and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can reside in an Application-Specific Integrated Circuit (ASIC). Alternatively, the ASIC can be located in a user equipment. Of course, the processor and the readable storage medium can also exist as discrete components in a communication device. The readable storage medium can be a read-only memory (ROM), random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.

[0061] The present invention also provides a program product including executable instructions stored in a readable storage medium. At least one processor of the device can read the executable instructions from the readable storage medium, and the at least one processor executes the executable instructions to cause the device to implement the methods provided in the various embodiments described above.

[0062] In the embodiments of the above-described device, it should be understood that the processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this invention can be directly manifested as execution by a hardware processor, or execution by a combination of hardware and software modules within the processor.

[0063] Through the above embodiments, this invention acquires standard sample images and test sample images of cigarette auxiliary materials by building a high-precision image acquisition system, and performs multi-dimensional preprocessing and color space conversion to generate Lab standard sample images and Lab test sample images; it retrieves a cigarette auxiliary material sample image library, uses data augmentation technology to expand the image library to generate a first expanded cigarette auxiliary material sample image library, and then uses MixUp image blending technology to augment the first expanded cigarette auxiliary material sample image library to generate a second expanded cigarette auxiliary material sample image library and trains a convolutional neural network model; the Lab standard sample images are then processed. The Lab standard sample image and the Lab sample image to be detected are input into a trained convolutional neural network model. Feature extraction and fusion are performed on the Lab standard sample image and the Lab sample image to be detected respectively to generate Lab standard sample image features and Lab sample image to be detected features. The color difference of the Lab standard sample image features and the Lab sample image to be detected features are compared to generate the color difference detection result corresponding to the sample image to be detected and pushed to the data management platform. This can improve the accuracy, stability and reliability of color difference detection of cigarette auxiliary materials, reduce the false detection rate and false detection rate of color difference detection of cigarette auxiliary materials, and ensure the production quality and brand image of cigarette auxiliary materials.

[0064] This invention addresses the problems of traditional manual visual color difference detection being heavily influenced by subjective experience and environmental factors by constructing a high-precision image acquisition system and performing multi-dimensional image preprocessing and Lab color space conversion. It converts the RGB color space to the Lab color space, capturing color information of cigarette auxiliary materials through Lab color space, accurately quantifying and effectively highlighting the color characteristics of these materials, thus overcoming the limitations of traditional machine vision detection methods that do not conform to human visual perception. Furthermore, this invention utilizes data augmentation technology combined with MixUp image blending technology to construct a first and a second extended sample image library of cigarette auxiliary materials and trains a convolutional neural network model, improving the feature learning ability and generalization performance of the convolutional neural network model, and enhancing its suitability for color difference recognition of cigarette auxiliary materials of different materials and processes. This invention effectively solves the problem of weak generalization ability caused by insufficient training samples and single data distribution in existing detection models. It uses a convolutional neural network model for feature extraction and fusion, and compares the features of Lab standard sample images and Lab test sample images to address the high false negative and false positive rates caused by traditional machine vision detection methods that only compare single color parameters and lack feature dimensions, thus improving the accuracy of identifying subtle color differences in cigarette auxiliary materials. Furthermore, by introducing a spatial attention mechanism module and a multi-scale feature fusion strategy, this invention deeply extracts features from Lab standard sample images and Lab test sample images, significantly improving the accuracy of color difference detection in cigarette auxiliary materials and its robustness in complex textures and scenes, comprehensively enhancing the quality control and process improvement effects throughout the entire cigarette auxiliary material production process.

[0065] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. An automated color difference recognition method for cigarette auxiliary materials based on convolutional neural networks, characterized in that, The method includes: A high-precision image acquisition system was built to acquire standard sample images and test sample images of cigarette auxiliary materials, and multi-dimensional preprocessing and color space conversion were performed to generate Lab standard sample images and Lab test sample images. The image library of cigarette auxiliary materials is retrieved, and data augmentation technology is used to expand the image library of cigarette auxiliary materials to generate a first expanded image library of cigarette auxiliary materials. Then, based on MixUp image blending technology, the first expanded image library of cigarette auxiliary materials is augmented to generate a second expanded image library of cigarette auxiliary materials, and a convolutional neural network model is trained. The Lab standard sample image and the Lab sample image to be detected are input into the trained convolutional neural network model. Feature extraction and fusion are performed on the Lab standard sample image and the Lab sample image to be detected respectively to generate Lab standard sample image features and Lab sample image to be detected features. The color difference of the Lab standard sample image features and the Lab sample image to be detected features are compared to generate the color difference detection result corresponding to the sample image to be detected and pushed to the data management platform.

2. The automated color difference recognition method for cigarette auxiliary materials based on convolutional neural networks according to claim 1, characterized in that, The construction of a high-precision image acquisition system to acquire standard sample images and test sample images of cigarette auxiliary materials, and to perform multi-dimensional preprocessing and color space conversion to generate Lab standard sample images and Lab test sample images, specifically includes: The high-precision image acquisition system is built by integrating hardware acquisition equipment and a precision linear testing platform. Based on the characteristics of the cigarette auxiliary materials, an adaptive acquisition scheme is used to acquire the standard sample image and the sample image to be tested, respectively. For the standard sample image and the sample image to be detected, the following steps are performed in sequence: denoising is performed using an adaptive Gaussian filtering algorithm; contrast enhancement is performed using a histogram equalization algorithm; and image size standardization is performed using a cubic convolution interpolation algorithm. The standard sample image and the sample image to be detected, after multi-dimensional preprocessing, are converted from the RGB color space to the XYZ color space using a linear transformation function. The corresponding transformation formula is as follows: In the formula, X, Y, and Z represent the stimulus values ​​of the X, Y, and Z color spaces, respectively; M represents the transformation matrix from the RGB color space to the XYZ color space. Using nonlinear transformation functions The standard sample image and the sample image to be detected are converted from the XYZ color space to the Lab color space to generate the Lab standard sample image and the Lab sample image to be detected. The corresponding conversion formulas are as follows: In the formula, The channel represents luminance; channel a represents chromaticity along the green-red axis; channel b represents chromaticity along the blue-yellow axis; x, y, and z represent the stimulus values ​​of the normalized X, Y, and Z color spaces, respectively. , , These represent the reference white point stimulus values ​​for the X, Y, and Z color spaces under the D65 standard light source, respectively; t represents the symbol for x, y, and z. This indicates the conversion threshold.

3. The automated color difference recognition method for cigarette auxiliary materials based on convolutional neural networks according to claim 1, characterized in that, The process of retrieving the cigarette auxiliary material sample image library involves using data augmentation technology to expand the image library and generate a first expanded cigarette auxiliary material sample image library. Specifically, this includes: Retrieve the cigarette auxiliary material sample image library and read the cigarette auxiliary material sample images in the cigarette auxiliary material sample image library, and then perform a two-dimensional rotation matrix. The pixel space coordinates of the cigarette auxiliary material sample image Perform random rotation transformation to generate the corresponding pixel target coordinates The corresponding random rotation transformation formula is as follows: In the formula, Indicates the center coordinates of the cigarette auxiliary material sample image; Using a diagonal scaling matrix For the pixel target coordinates Perform random scaling transformation to generate the corresponding pixel floating-point coordinates The corresponding random scaling transformation formula is as follows: ； The cigarette auxiliary material sample image after random scaling transformation is reconstructed using bilinear interpolation to obtain the pixel floating-point coordinates. Corresponding smooth pixel value traverse the smoothed pixel values The image library of reconstructed cigarette auxiliary materials is generated, and the corresponding calculation formula is as follows: In the formula, This represents the floor function; m and n represent the summation indices in the horizontal and vertical directions, respectively. This indicates the weight of the pixel's floating-point coordinates in the horizontal direction; This indicates the weight of the pixel's floating-point coordinates in the vertical direction; Based on the reconstructed cigarette auxiliary material sample images in the reconstructed cigarette auxiliary material sample image library, a fixed pixel size effective cropping window is randomly selected in the reconstructed cigarette auxiliary material sample image, and the reconstructed cigarette auxiliary material sample image is randomly cropped and transformed to obtain an expanded cigarette auxiliary material sample image library. Data augmentation techniques are used to perform random color space perturbation and random noise addition operations on the expanded cigarette auxiliary material sample image library, and the first expanded cigarette auxiliary material sample image library is output.

4. The automated color difference recognition method for cigarette auxiliary materials based on convolutional neural networks according to claim 1, characterized in that, The step of further augmenting the first expanded cigarette auxiliary material sample image library using MixUp image blending technology to generate a second expanded cigarette auxiliary material sample image library specifically includes: Using the MixUp image blending technology, two sets of first expanded cigarette auxiliary material sample images are randomly selected from the first expanded cigarette auxiliary material sample image library. , and corresponding tag information , The image is then augmented using a linear weighting formula to generate a second extended sample image of cigarette auxiliary materials. and corresponding extended tags The corresponding calculation formula is as follows: In the formula, Represents the random mixing coefficients generated by the Beta distribution; Summarize the first expanded cigarette auxiliary material sample image library and the second expanded cigarette auxiliary material sample image library. and the corresponding extended tags The second expanded cigarette auxiliary material sample image library was obtained.

5. The automated color difference recognition method for cigarette auxiliary materials based on convolutional neural networks according to claim 1, characterized in that, Training the convolutional neural network model using the second expanded cigarette accessory sample image library specifically includes: The training environment was built based on the PyTorch deep learning framework, and the AdamW optimizer and cosine annealing learning rate scheduler were used to adjust the learning rate of the convolutional neural network model. The second extended cigarette auxiliary material sample image library is input into the convolutional neural network model, and the learning rate is used for learning and training. The training color difference value and training color difference category corresponding to the second extended cigarette auxiliary material sample image library are output. The mean squared error loss function and the cross-entropy loss function are used to perform loss prediction and training optimization on the training color difference values ​​and the training color difference categories; The 5-fold cross-validation method is used to validate and evaluate the optimized training color difference values ​​and the training color difference categories, and the validation and evaluation results are output. Based on the verification and evaluation results, the hyperparameters of the convolutional neural network model are adjusted accordingly, driving the convolutional neural network model to continuously train and optimize, and outputting the trained convolutional neural network model.

6. The automated color difference recognition method for cigarette auxiliary materials based on convolutional neural networks according to claim 1, characterized in that, The Lab standard sample image and the Lab sample image to be detected are input into the trained convolutional neural network model. Feature extraction and fusion are performed on the Lab standard sample image and the Lab sample image to be detected, respectively, to generate Lab standard sample image features and Lab sample image to be detected features. Specifically... include: The trained convolutional neural network model includes an image input layer, a feature extraction network, a feature fusion layer, and a classification output layer. The image input layer receives the Lab standard sample image and the Lab sample image to be detected, and performs Z-score normalization processing on them respectively. The Z-score normalized Lab standard sample image and the Lab sample image to be detected are then transmitted to the feature extraction network. The feature extraction network uses 3×3 and 1×1 convolutional kernels of the convolutional layer to perform sliding convolution operations on the Lab standard sample image and the Lab sample image to be detected after Z-score normalization, and combines the spatial attention mechanism module to extract features respectively, and outputs the corresponding standard multi-scale feature map sequence and the multi-scale feature map sequence to be detected. The feature fusion layer employs a multi-scale feature fusion strategy to spatially align the standard multi-scale feature map sequence, and then splices and fuses the spatially aligned standard multi-scale feature map sequence according to the channel dimension to generate the Lab standard sample image features. The feature fusion layer employs a multi-scale feature fusion strategy to spatially align the multi-scale feature map sequence to be detected, and then splices and fuses the spatially aligned multi-scale feature map sequence according to the channel dimension to generate the Lab sample image features to be detected.

7. The automated color difference recognition method for cigarette auxiliary materials based on convolutional neural networks according to claim 6, characterized in that, The feature extraction network utilizes 3×3 and 1×1 convolutional kernels of the convolutional layers to perform sliding convolution operations on the Z-score-normalized Lab standard sample image and the Z-score-normalized Lab sample image to be detected, respectively. It then combines this with a spatial attention mechanism module for feature extraction, outputting a standard multi-scale feature map sequence corresponding to the Lab standard sample image and a detection multi-scale feature map sequence corresponding to the Lab sample image to be detected. Specifically, this includes: The feature extraction network includes convolutional layers, batch normalization layers, and activation functions; The detection color features and detection texture features of the Lab sample image after Z-score normalization are extracted using the 3×3 convolution kernel of the convolutional layer. Using the 3×3 convolution kernel and the 1×1 convolution kernel of the convolutional layer, combined with the spatial attention mechanism module, the intermediate semantic features and high-level abstract features of the Lab sample image to be detected after Z-score normalization are extracted; For the color features, texture features, intermediate semantic features, and high-level abstract features to be detected, the batch normalization layer is used for standardization and affine transformation, and the activation function is used for nonlinear mapping to output the multi-scale feature map sequence corresponding to the Lab sample image to be detected. The extraction method for the standard multi-scale feature map sequence corresponding to the Lab standard sample image is the same as above.

8. The automated color difference recognition method for cigarette auxiliary materials based on convolutional neural networks according to claim 1, characterized in that, The step of comparing the color difference between the Lab standard sample image features and the Lab sample image to be detected, generating the color difference detection result corresponding to the sample image to be detected, and pushing it to the data management platform specifically includes: The features of the Lab standard sample image and the features of the Lab sample image to be detected are input into the classification output layer of the trained convolutional neural network model, and the color difference defect value corresponding to the sample image to be detected is calculated using the CIELab color difference formula. The corresponding calculation formula is as follows: In the formula, This represents the brightness difference between the features of the Lab standard sample image and the features of the Lab sample image to be detected; This represents the difference in red and green color between the features of the Lab standard sample image and the features of the Lab sample image to be detected; The difference between the yellow and blue colors represents the difference between the features of the Lab standard sample image and the features of the Lab sample image to be detected. Based on the preset color difference defect threshold T according to the features of the Lab standard sample image, when the color difference defect value corresponding to the sample image to be detected is... If the color difference defect threshold T is greater than or equal to the threshold value, then the sample image to be detected is determined to have a color difference defect. When the sample image to be detected has color difference defects, the color difference defect values ​​are marked and summarized. For the corresponding pixels in the sample image to be detected, a defect mask image is generated. A contour detection algorithm is used to identify and extract the connected regions of the pixels in the defect mask image. The minimum bounding rectangle of the connected regions is calculated using the axis-aligned minimum bounding rectangle algorithm, and the vertex coordinates of the minimum bounding rectangle are output. , , and Used as coordinates for the color difference defect area; When the sample image to be detected has color difference defects, the spatial dimension of the Lab sample image features to be detected is mapped through the fully connected layer in the classification output layer to generate three-dimensional Lab sample image features to be detected. The probability distribution of color difference defect categories corresponding to the three-dimensional Lab sample image features is calculated using the Softmax activation function, and the color difference defect category with the highest probability value is selected as the color difference defect category of the sample image to be detected. Summarize the color difference defect values ​​corresponding to the sample images to be detected. The coordinates of the color difference defect area and the category of the color difference defect are used to generate the color difference detection result corresponding to the sample image to be detected; The basic information corresponding to the sample image to be tested and the color difference detection results corresponding to the sample image to be tested are associated and integrated to generate a color difference detection report corresponding to the sample image to be tested. Based on the coordinates of the color difference defect area, the color difference defect area is highlighted in the sample image to be tested. The color difference detection report corresponding to the sample image to be tested and the highlighted color difference defect area are pushed to the data management platform in real time.

9. An automated color difference recognition system for cigarette auxiliary materials based on convolutional neural networks, applied to the automated color difference recognition method for cigarette auxiliary materials based on convolutional neural networks as described in any one of claims 1-8, characterized in that, The system includes: The image acquisition and processing module is used to build a high-precision image acquisition system to acquire standard sample images and test sample images of cigarette auxiliary materials, and to perform multi-dimensional preprocessing and color space conversion to generate Lab standard sample images and Lab test sample images. The image library generation and model training module is used to retrieve the cigarette auxiliary material sample image library, use data augmentation technology to expand the image library of cigarette auxiliary material sample image library to generate a first expanded cigarette auxiliary material sample image library, and then use MixUp image blending technology to augment the first expanded cigarette auxiliary material sample image library to generate a second expanded cigarette auxiliary material sample image library and train a convolutional neural network model. The feature extraction and color difference detection module is used to input the Lab standard sample image and the Lab sample image to be detected into the trained convolutional neural network model, extract and fuse features of the Lab standard sample image and the Lab sample image to be detected respectively, generate Lab standard sample image features and Lab sample image features to be detected, compare the color difference of the Lab standard sample image features and the Lab sample image features to be detected, generate the color difference detection result corresponding to the sample image to be detected, and push it to the data management platform.

Images