Method and System for Identifying Cigarette Pack Texture Based on Binocular Camera and Convolutional Neural Network

Abstract

The present invention provides a method and system for cigarette packet texture recognition based on a binocular camera and a convolutional neural network, comprising steps S1: rapidly capturing cigarette packet texture images using a high-speed binocular camera; S2: selecting a CNN as the primary model to accurately identify cigarette packet surface texture features; and S3: generating an identification report based on the monitoring results, including at least the monitoring results and monitoring time information. By proposing a focusing strategy, the present invention enables the binocular camera system to achieve the required image resolution and quality under various production conditions, ensuring that the binocular camera quickly and accurately focuses and captures the subtle features of the cigarette packet, thereby meeting the stringent image quality requirements of automated monitoring systems and improving monitoring accuracy and efficiency.

Description

Technical Field

[0001] The present invention relates to the technical field of tobacco product monitoring, and in particular to a cigarette package texture recognition method and system based on a binocular camera and a convolutional neural network. Background Art

[0002] With the development of science and technology, image processing technology and machine vision technology have been gradually applied to the field of tobacco product monitoring technology. This is because traditional tobacco product monitoring methods can no longer meet modern needs. Existing tobacco product monitoring methods are mainly carried out through manual monitoring or simple machine vision technology.

[0003] For production lines and large-scale cigarette / tobacco factories, although manual monitoring can more accurately monitor the quality of tobacco products, it is inefficient and cannot meet the needs of large-scale production. Although simple machine vision technology can improve detection efficiency, its accuracy is not high and it is easily affected by environmental factors.

[0004] Furthermore, given the space constraints and continuous operation requirements of production lines, third-party monitoring equipment must be quickly deployed within gaps in the production line without disrupting overall line operations. However, due to the wide variety of tobacco products, each with its own distinct image characteristics, traditional machine vision technology often struggles to accurately identify these diverse products. Production lines within tobacco factories are often compactly designed to maximize production space. In such environments, third-party monitoring equipment, such as high-speed cameras and machine vision systems, must be quickly installed within the limited space of various production lines without disrupting the production process. These devices are tasked with real-time monitoring of cigarette pack quality, including defects such as loose transparent paper, printing errors, and seal integrity. Due to the diverse range of tobacco products, from classic hard cartons to soft packs to various special holiday packaging, each pack has unique image characteristics. Furthermore, the positioning of linkage mechanisms and conveyor systems within the production line varies. This results in traditional machine vision technology performing poorly when detecting loose seals in hard cartons, soft packs, and various types of cigarette packs.

[0005] Based on this, it is urgent to construct a focusing strategy to define the constraint relationship so that the cigarette package or detection device can achieve fast focusing on any line, ensuring that the camera focuses quickly and accurately and captures the subtle features of the cigarette package, thereby meeting the strict requirements of the automated monitoring system for cigarette package image quality. Summary of the Invention

[0006] In view of the shortcomings of the existing technology, the purpose of the present invention is to provide a cigarette package texture recognition method and system based on a binocular camera and a convolutional neural network to solve the problems raised in the above background technology.

[0007] In order to achieve the above object, the present invention is implemented through the following technical solution: a cigarette package texture recognition method based on a binocular camera and a convolutional neural network, comprising the steps of:

[0008] S1. Rapidly capture the texture image of the cigarette package using a high-speed binocular camera. The operation is as follows:

[0009] S1-1. Based on the width W and height H of the target cigarette package, the focal length f of the binocular camera, and the detail size D on the target cigarette package, determine and establish a geometric constraint relationship between the binocular camera and the target cigarette package to ensure that the binocular camera can clearly and fully capture all details of the target cigarette package;

[0010] S1-2. Construct a focusing strategy to determine the minimum focal length f that meets the basic viewing angle θ and resolution R required for binocular camera detection. min , and use the minimum focal length f min Calculate the viewing angle θ required to actually cover the target cigarette pack width and height on the tobacco production line w and θ h , performing a focusing operation so that the cigarette pack texture image generates a binary cigarette pack texture image representing the surface texture features of the cigarette pack in the clearest focus state;

[0011] S2. Select CNN as the primary model and build a convolutional neural network model with a network architecture including multiple convolutional layers, pooling layers, and fully connected layers to accurately identify the surface texture features of cigarette packs and ensure the monitoring of target cigarette packs;

[0012] S3. Generate an identification report based on the monitoring results, which at least includes the monitoring results and monitoring time information.

[0013] As a second aspect of the present invention, a cigarette package texture recognition system based on a binocular camera and a convolutional neural network, a memory, and a processor are proposed, wherein the memory includes a cigarette package texture recognition program based on a binocular camera and a convolutional neural network, and when the cigarette package texture recognition program based on the binocular camera and the convolutional neural network is executed by the processor, the cigarette package texture recognition method is implemented.

[0014] Compared with the prior art, the present invention has the following beneficial effects:

[0015] 1. Considering the spatial constraints and continuous operation requirements of the production line, this invention proposes a focusing strategy that enables the binocular camera system to achieve the required image resolution and quality under various production conditions, including different cigarette pack types, sizes, and production speeds. This strategy automates the focusing process, including the acquisition, processing, and feature extraction of cigarette pack texture analysis images, as well as the precise calculation of the defocus distance and the direction of cigarette pack texture changes. This ensures that the camera focuses quickly and accurately, capturing the subtle features of the cigarette pack. This meets the stringent image quality requirements of automated monitoring systems, improves monitoring accuracy and efficiency, ensures the continuity and stability of the production line, and achieves strict control of product quality.

[0016] 2. To determine whether the cigarette packaging meets production standards, has defects, or has been damaged during transportation, the present invention uses the Sobel operator to process the image texture of each partition, extracting the surface texture features of the cigarette package image. By accurately identifying these edges, the accuracy of focus is improved, while also ensuring that every detail of the cigarette package is clearly captured, thereby promptly discovering problems during the quality control process. For example, if the edges of a certain area of the cigarette package are blurred, it may indicate unclear printing or material defects in that area, which may be difficult to detect without edge monitoring. However, through edge monitoring, the present invention can more accurately focus on these key areas, improving the reliability and efficiency of monitoring.

[0017] 3. Because the changing lighting conditions of cigarette packs on the production line can affect grayscale values and lead to inaccurate segmentation, this invention integrates factors that generate noise, such as strong light and shadows, to construct a dynamic threshold for adjusting image content in the context of cigarette pack monitoring. This enhances the visibility of important features in the image, effectively suppresses noise, and improves image quality.

[0018] 4. Since the image types of each cigarette pack are different, the present invention determines to use CNN as the main model for image classification by adopting visual algorithms and large model technology, and constructs a CNN model that can accurately identify the image features of different types of tobacco products, and quickly monitor the characteristics of cigarette packs, such as packaging defects and printing errors, among tens of thousands of cigarette packs. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The disclosure of the present invention is described with reference to the accompanying drawings. It should be understood that the drawings are for illustrative purposes only and are not intended to limit the scope of protection of the present invention. In the drawings, the same reference numerals are used to refer to the same components. Among them:

[0020] Figure 1 This is a schematic diagram of the overall processing flow of a cigarette packet texture recognition method based on a binocular camera and a convolutional neural network, proposed in one embodiment of the present invention;

[0021] Figure 2 Schematic diagram of the principle of performing gradient calculation on pixels of a cigarette pack texture image in four directions (horizontal H, vertical V, diagonal D1, and diagonal D2) to identify cigarette pack texture changes, as proposed in one embodiment of the present invention;

[0022] Figure 3 Schematic diagram of the principle of optical path for collecting cigarette packet texture images proposed in one embodiment of the present invention;

[0023] Figure 4 A schematic diagram of a focus strategy construction process proposed in an embodiment of the present invention;

[0024] Figure 5 This is a flowchart of training a CNN model using a binary cigarette pack texture image that has been pre-processed in a focusing operation and / or a binarization generation step, as proposed in one embodiment of the present invention. DETAILED DESCRIPTION

[0025] It is easy to understand that according to the technical solution of the present invention, without changing the essential spirit of the present invention, a person skilled in the art can propose a variety of interchangeable structural modes and implementation modes. Therefore, the following specific embodiments and drawings are only exemplary descriptions of the technical solution of the present invention and should not be regarded as the entire invention or as a limitation or restriction of the technical solution of the present invention.

[0026] The present invention will be further described in detail below with reference to the accompanying drawings, but this does not limit the present invention.

[0027] To provide an understanding of the technical concept and implementation principles of the present invention, the present invention proposes a method and system for cigarette package texture recognition based on a binocular camera and a convolutional neural network. Due to the wide variety of tobacco products, from classic hard packs to soft packs to various special holiday packaging, each cigarette package has unique image features. Furthermore, the positions of linkage mechanisms and conveyor systems on the production line vary. Consequently, traditional machine vision technology struggles with quickly and accurately focusing and capturing subtle features of cigarette packages when monitoring loose seals on hard and soft packs, as well as various cigarette packages. Therefore, the present invention proposes a focusing strategy that enables the binocular camera system to achieve the required image resolution and quality under various production conditions, including varying cigarette package types, sizes, and production speeds. It can be understood that this focusing strategy defines constraints, enabling rapid focusing of cigarette packages or the present device on any line. This focusing is achieved by capturing image defects. Once focusing is complete, the image captured during the focusing process can be directly used, eliminating the need for repeated data acquisition and enabling rapid processing.

[0028] As a first aspect of the present invention, Figure 1-Figure 5As shown in the figure, a cigarette pack texture recognition method based on a binocular camera and a convolutional neural network is proposed, which includes the following steps:

[0029] S1. Build a high-speed binocular camera detection system capable of capturing cigarette pack images at a rate of 1,000 times per minute. It should be noted that this system requires a high-speed binocular camera. Preferred models include Hikvision's RGB-D intelligent stereo camera, Orbbec's Gemini 2 binocular structured light 3D camera equipped with a depth engine chip, or the DUO MLX depth camera based on the active infrared binocular principle. These cameras are capable of rapidly capturing cigarette pack texture images and meeting the high-speed demands of the production line. It is understood that in cigarette pack inspection, binocular camera-based recognition methods offer significant advantages in accuracy and reliability compared to traditional monocular camera methods due to the rich spatial information they provide. Cigarette packs often have complex three-dimensional textures and patterns, and binocular cameras are able to capture this depth information, playing a key role in cigarette pack inspection. The principle is that binocular cameras mimic the stereoscopic vision principles of the human visual system, using two cameras to capture images of the same object from different angles. The difference between these images, i.e., parallax, is used to calculate the depth information of the object. In cigarette package detection, the depth information of the texture and pattern on the cigarette package surface is crucial for identification and classification. Therefore, by adaptively adjusting the focus, the use of binocular cameras can ensure that the key features of the cigarette package, such as texture and pattern, are clearly captured. Since the use of binocular cameras in cigarette package detection also needs to consider the focal length matching problem, that is, the focal length difference between the two images will cause image mismatch or blur, which will seriously affect the accuracy of the cigarette package detection results. Therefore, by precisely controlling and adjusting the focus and focal length of the binocular camera, this difference can be minimized, thereby improving the accuracy and reliability of cigarette package detection. The specific implementation steps are as follows:

[0030] S1-0. Based on the actual needs of the tobacco factory's production line, hardware configuration is first completed. For example, a high-performance GPU is used to process large amounts of image data and execute complex visual algorithms, while a CPU is used for logical control. The logical control in this example at least includes: linkage control of the hardware configuration, namely, synchronization of the target cigarette pack conveyor belt with the binocular camera: the conveyor belt needs to be synchronized with the binocular camera's shooting speed to ensure that each cigarette pack is correctly photographed; light source control: the use of LED or laser light sources needs to be synchronized with the binocular camera's shooting speed to ensure that images are captured in the best condition. It is understandable that, in order to adapt to the monitoring of cigarette packs of different brands and models, the hardware configuration system needs to achieve high-speed and high-precision cigarette pack monitoring through high-performance hardware and advanced visual algorithms. The structural linkage needs to ensure the synchronization of components including the conveyor belt, camera, and light source to ensure efficient operation of the system.

[0031] S1-1. Based on the width W and height H of the target cigarette package, the focal length f of the binocular camera, the binocular camera sensor, and the detail size D on the target cigarette package, determine and construct the geometric constraint relationship between the binocular camera and the target cigarette package to ensure that the binocular camera can clearly and fully capture all details of the target cigarette package.

[0032] Based on the above technical concept, it should be noted that the purpose of determining the detailed geometric constraint relationship between the binocular camera and the target cigarette pack is to ensure that the binocular camera can clearly capture all details of the target cigarette pack. On the actual production line of a tobacco factory, in order to build a binocular camera detection system to monitor cigarette packs, it is necessary to integrate the following key parameters: the size of the cigarette pack (width W, height H), the focal length f of the camera, the size of the camera sensor (width l), and the detail size D on the cigarette pack. Among them, the preceding detail size D can be understood as when designing the binocular camera detection system, it is necessary to ensure that the resolution and focal length of the binocular camera can capture these detail sizes D for accurate monitoring and analysis. This usually involves precise calculation of the pixel size of the binocular camera sensor, the focal length of the camera, and the distance between the cigarette pack and the camera, so as to ensure that the three-dimensional image texture information and detail features of the cigarette pack can be clearly captured. All details of the target cigarette package include at least the clarity and integrity of the pattern, text or barcode on the cigarette package, and the detail size D is the minimum detectable size when checking printing defects, such as blur, missing printing or ghosting, or the minimum size when checking seals, such as loose seals or uneven seams at the seals, seams or folds of the cigarette package.

[0033] In one embodiment of the present invention, the geometric constraint relationship constructed includes: ensuring that the viewing angle of the binocular camera satisfies the viewing angle constraints required to cover the width and height of the target cigarette pack:

[0034]

[0035] The focal length f constraint required to ensure that the binocular camera's field of view can cover the maximum size of the target cigarette pack is:

[0036]

[0037] Ensure that the resolution R of the binocular camera meets the resolution constraint required to capture the detail size D on the target cigarette pack: R = D / d, where d is the corresponding pixel size on the binocular camera sensor; and

[0038] The distance constraint SD between the binocular camera and the target cigarette pack is:

[0039] As an embodiment of the present invention, the proposed cigarette packet texture recognition method further includes:

[0040] S1-2. Construct a focusing strategy to determine the minimum focal length f that meets the basic viewing angle θ and resolution R required for binocular camera detection. min , and use the minimum focal length f min Calculate the viewing angle θ required to actually cover the target cigarette pack width and height on the tobacco production line w and θ h , performing a focusing operation so that the cigarette pack texture image generates a binary cigarette pack texture image representing the surface texture features of the cigarette pack in the clearest focus state.

[0041] Based on the above technical concept, it can be understood that the focal length f determines the viewing angle and field of view of the binocular camera, which directly affects the target cigarette package size range that the binocular camera can cover. The appropriate focal length can ensure that the width and height of the cigarette package are within the camera's viewing angle, so that they can be clearly captured. The resolution R determines the minimum detail size that the binocular camera can capture. A high-resolution camera can capture smaller details, which is crucial for monitoring subtle features on cigarette packages. Based on this, it is necessary to use the resolution R size as the end condition based on the constructed focus strategy and obtain the minimum focal length f through iterative solution. min , and in practice, after adjustment and verification, the focal length that enables the camera system to achieve the best imaging effect.

[0042] The above calculation of the minimum focal length f min In the specific implementation process, first, the required resolution R of the camera is determined based on the detail size D that needs to be monitored on the target cigarette package and the corresponding pixel size d on the camera sensor. It can be understood that this resolution is the end condition of the focusing strategy, that is, the focal length f sought should make the actual resolution of the binocular camera meet or exceed this requirement.

[0043] Example: Since the specific dimensions of cigarette packs vary depending on the packaging type, the following common models are mainly available for domestic hard and soft pack cigarette boxes. The following table shows an example:

[0044]

[0045] Secondly, select an initial estimated focal length f0 based on experience or the recommended settings of the binocular camera, and calculate the viewing angle θ required for the binocular camera to cover the width and height of the target cigarette pack. w and θ h It can be understood that calculating the viewing angle θ can ensure that the field of view (FOV) of the binocular camera is large enough to cover the size of the entire cigarette pack. If the viewing angle is too small, the entire target cigarette pack cannot be captured, resulting in incomplete monitoring. Therefore, an appropriate viewing angle θ can avoid image distortion, such as perspective distortion. In this example,

[0046] Again, according to the calculated viewing angle θ w and θ h , determine the maximum viewing angle θ max ,θ max =max(31.36°,47.12°)=47.12°; calculate the distance SD between the binocular camera and the target cigarette pack through the focal length f constraint relationship, Based on this, the actual resolution R required is determined when the distance between the binocular camera and the target cigarette pack is appropriate. act ;

[0047] Finally, by comparing the actual resolution R act The resolution target value R is set according to the detail recognition requirements of the cigarette package monitoring system. tar , perform the fine-tuning step, where if R act <R tar , you need to fine-tune the focus: Repeat the above steps until R act >R tar , end, otherwise, directly record the final focal length end.

[0048] As an embodiment of the present invention, it should be noted that the purpose of constructing the focusing strategy is to achieve efficient integration and precise operation of the camera system in the production line that has been built in the cigarette factory. Taking into account the space limitations of the production line and the need for continuous operation, the equipment must be quickly built in the gaps of the production line while ensuring that it does not affect the operation of the entire production line. Therefore, by proposing a focusing strategy, the binocular camera system can achieve the required image resolution and quality under various production conditions, including different cigarette pack models, sizes and production speeds. During specific implementation, the focusing strategy ensures that the camera focuses quickly and accurately through an automated focusing process, including the acquisition, processing, and feature extraction of cigarette pack texture analysis images, as well as the precise calculation of the defocus distance and the direction of cigarette pack texture change, to capture the subtle features of the cigarette pack, thereby meeting the strict image quality requirements of the automated monitoring system.

[0049] In S1-2, the specific process of constructing the focus strategy includes:

[0050] S1-21, such as Figure 2 As shown, the cigarette pack texture image acquisition and preprocessing are performed as follows:

[0051] S1-211. Determine the focus position of the binocular camera to ensure that the acquisition area defined by the width W and height H of the target cigarette package can be clearly captured. Use the binocular camera to capture the laser reflection texture image on the target cigarette package. Within the acquisition area, perform gradient calculations on the pixel points of the cigarette package texture image in the four directions of horizontal H, vertical V, diagonal D1, and diagonal D2 to identify changes in the cigarette package texture.

[0052] Based on the above technical concept, in the process of obtaining and preprocessing the cigarette pack texture image, the gradient calculation is performed on the pixel points of the cigarette pack texture image in four directions. The specific operation of identifying the cigarette pack texture change is as follows: For the horizontal gradient of the cigarette pack texture image The cigarette pack texture changes are identified by comparing the brightness difference between each pixel (p, q) in the cigarette pack texture image and the pixels in the adjacent columns in the horizontal direction: Vertical gradient for cigarette pack texture image By comparing the brightness difference of adjacent rows in the same row, we can identify the texture changes of cigarette packs: Diagonal gradient for cigarette pack texture image and The texture changes of cigarette packs are identified by comparing the brightness differences of diagonally adjacent pixels (p, q) from the upper left to the lower right and from the upper right to the lower left: Where h p,q is the pixel value at the pixel position (p,q) in the cigarette pack texture image.

[0053] S1-212. Select a pixel point (p, q) in the cigarette pack texture image as the starting point for analysis, and define the diffusion radius r. Calculate the Euclidean distance d between each pixel point and the starting point (p0, q0).

[0054] Where p0 and q0 are the coordinates of the starting pixel point, respectively. The diffusion coefficient is applied to perform weight adjustment to further determine the precise area of the cigarette pack texture analysis within the acquisition area, ensuring that the precise analysis area covers the key texture features of the target cigarette pack. A filter (such as a Gaussian filter) is applied to the captured cigarette pack texture image within the precise area to reduce image noise and improve image smoothness. Histogram equalization or adaptive histogram equalization is used to enhance the image contrast and improve image quality.

[0055] Example: Taking the cigarette pack type (hard pack) in the above data table as an example, an example of the above steps in the cigarette pack texture area diffusion and weight adjustment is given. The known size parameters of the hard pack are as follows: width W = 56 mm, height H = 87 mm, thickness T = 22 mm, detail size D = 1 mm, and sensor pixel size d = 0.01 mm.

[0056] In specific implementation, a pixel point h(p0,q0) is selected in the texture image of the hard-pack type cigarette package as the starting point of the analysis. This point can be an area with obvious texture features in the cigarette package texture image, such as the edge of the cigarette package or the center of a specific pattern. At this time, a diffusion radius r is defined. This radius can be set according to the size of the cigarette package and the expected analysis area size or the cigarette factory production line area. Assuming that the starting pixel point (p0,q0) is located at the center of the cigarette package texture image, the pixel coordinates obtained are (W / 2,H / 2)=(56 / 2,87 / 2). The diffusion radius r is set to 1 / 10 of the width or height of the target cigarette package, that is, 5.6 or 8.7 mm, to ensure that the analysis area covers the key texture features of the target cigarette package. The calculated Euclidean distance

[0057] Based on the above technical concept, it should be noted that in order to further determine the precise area for analyzing the cigarette pack texture in the acquisition area and ensure that the precise area of analysis covers the key texture features of the target cigarette pack, in practice, the Gaussian function can also be used to calculate the weight W of each pixel point. p,q , Based on the calculated weights, the pixels in the cigarette pack texture image are weighted to highlight the texture features around the starting point. For example, cigarette packs often have brand names, patterns, and other decorative designs. The edges and contrast of these printed features are important features in texture analysis. The barcode on the cigarette pack is another important texture feature. The straight edges and black-white contrast of the barcode are crucial for identifying and tracking products. The texture features at the seals and seams of the cigarette pack, such as loose seals or uneven seams, can reflect the packaging quality of the cigarette pack. The cigarette pack may have scratches, stains, or other surface defects during the production process. This can be achieved by adjusting the brightness or color value of the pixels to make the texture features more prominent. Through the above method, the precise area for cigarette pack texture analysis in hard pack types can be determined, and by adjusting the weights, the characteristics of the cigarette pack texture can be made more prominent, thereby improving the accuracy of subsequent analysis.

[0058] S1-22: Perform cigarette packet texture image processing and feature extraction. The operations are as follows:

[0059] S1-221. Convert the acquired cigarette package texture image into a grayscale image to simplify the data and reduce computational complexity, and divide the grayscale image into three partitions: highlight, brighter, and darker according to the grayscale value. The customized highlight partition corresponds to the high-reflection area of the target cigarette package, the brighter partition corresponds to the medium-reflection area of the target cigarette package, and the darker partition corresponds to the low-reflection and / or shadow area of the target cigarette package.

[0060] Based on the above technical concept, in the process of cigarette packet texture image processing and feature extraction, the specific operations of grayscale post-partitioning corresponding to the reflective area are as follows:

[0061] First, a grayscale image G(p,q) with a single grayscale value is generated by weighted averaging the red, green, and blue channels of the obtained cigarette pack texture image. The common formula is: G p,q =0.299R+0.587G+0.114B, where R, G, and B are the pixel values of the red, green, and blue channels respectively. p,q is the pixel value of the converted grayscale image at position (p,q);

[0062] Secondly, the grayscale thresholds T1 and T2 are determined based on the grayscale cigarette packet texture image histogram or empirical values. The custom grayscale threshold T1 is used to distinguish between darker and brighter partitions, and the grayscale threshold T2 is used to distinguish between brighter and highlighted partitions. The grayscale image G(p,q) is divided into three partitions according to the grayscale thresholds:

[0063]

[0064] Where R(p,q) is the partition result, a, b, and c are the bright, bright, and dark partitions respectively;

[0065] Again, apply the partitioning results to the top, bottom, left, right, front and back of the target cigarette pack to obtain the face partitioning results. Where, F i is the partition result of the i-th surface of the target cigarette packet, i = 1, 2, ..., 6.

[0066] It can be understood that by partitioning the grayscale image, the texture features on different surfaces of the target cigarette package can be identified, and the highlighted area is set to correspond to the high-reflection area of the cigarette package, the brighter area corresponds to the medium-reflection area, and the darker area corresponds to the low-reflection or shadow area. This partitioning helps to analyze the texture features of the cigarette package more carefully and provide support for subsequent texture analysis and quality control.

[0067] S1-222, apply the Sobel operator in the horizontal H, vertical V, diagonal D1, and diagonal D2 directions to each partition to extract the surface texture features of the cigarette pack image in each partition and obtain the texture sensitivity coefficient T that characterizes the texture change of the cigarette pack sf , based on the texture sensitivity coefficient T sf Generate a binary cigarette pack texture image.

[0068] Based on the above technical concept, it can be understood that in the texture extraction and average grayscale value calculation, by applying the Sobel operator (edge monitoring algorithm) to process each area of the cigarette package texture image, the texture features of the cigarette package surface can be quickly identified and extracted. These features are crucial for the quality control and automated monitoring system of cigarette packages. By analyzing these texture features, it can be determined whether the packaging of the cigarette package meets the production standards, whether there are defects, or whether it is damaged during transportation. By accurately identifying these edges, the accuracy of focus can be improved, ensuring that every detail of the cigarette package is clearly captured, so that problems can be discovered in time during the quality control process. For example, if the edges of a certain area of the cigarette package are blurred, it may indicate that there is unclear printing or material defects in the area, which may be difficult to detect without edge monitoring. Through edge monitoring, these key areas can be focused more accurately, improving the reliability and efficiency of monitoring.

[0069] In one embodiment of the present invention, in the process of cigarette pack texture image processing and feature extraction, when using the Sobel operator to extract the surface texture features of the cigarette pack image in each face partition, it is based on respectively calculating the grayscale change rate of the cigarette pack texture image in each partition in the four directions of horizontal H, vertical V, diagonal D1, and diagonal D2. The specific operation is as follows: a 3×3 Sobel operator matrix is constructed in the four directions of horizontal H, vertical V, diagonal D1, and diagonal D2, wherein the middle value of the 3×3 Sobel operator matrix is set to 0 to represent The Sobel operator is insensitive to changes in the four directions of horizontal H, vertical V, diagonal D1, and diagonal D2, and at the same time, the positive and negative values in the Sobel operator are set to represent the brightness changes of the cigarette pack texture image in each partition in the four directions. When it is a positive value, it means that the brightness change in the horizontal H direction increases from left to right, the brightness in the vertical V direction increases from top to bottom, the brightness change in the diagonal D1 direction increases from the upper left to the lower right, and the brightness change in the diagonal D2 direction increases from the upper right to the lower left. When it is a negative value, the brightness change in the four directions decreases.

[0070] In specific implementation, the constructed 3×3 Sobel operator matrix can be: Used to monitor the horizontal edges of cigarette pack texture images. Used to monitor the vertical edges of cigarette pack texture images. Used to monitor the edge of the cigarette pack texture image from the upper left to the lower right direction, It is used to monitor the edge of the cigarette pack texture image from the upper right to the lower left, and then calculate the grayscale change rate in the four directions of horizontal H, vertical V, diagonal D1, and diagonal D2 through convolution operation.

[0071] In one embodiment of the present invention, the texture sensitivity coefficient T that characterizes the texture change of the cigarette pack is obtained. sf The specific operations are as follows:

[0072] First, by calculating the pixel points (p j ,q j ) is weighted according to its gradient amplitude in four directions, and the average texture grayscale value of the cigarette pack texture image in each partition is calculated

[0073]

[0074] Where h i (p j ,q j ) is the pixel point in the i-th partition (p j ,q j ) grayscale value, N i is the pixel point in the i-th partition (p j ,q j ) quantity, W ij is the pixel point in the i-th partition (p j ,q j ) is calculated based on the gradient magnitude,

[0075] W ij =1+α(G H (p j ,q j )+G V (p j ,q j )+G D1 (p j ,q j )+G D2 (p j ,q j )) (14);

[0076] Secondly, according to the average texture gray value And the brightness B and contrast C of the cigarette pack texture image in the partition, calculate the texture sensitivity coefficient T sf :

[0077]

[0078] Where a and b are the texture sensitivity coefficients T sf The minimum and maximum values of G(p j ,q j ) is the pixel point (p j ,q j) is the comprehensive gradient amplitude in four directions, β is the adjustment coefficient, which is used to balance the brightness B and contrast C to the texture sensitivity coefficient T sf The influence of texture sensitivity coefficient T sf The value range is

[0079] In one embodiment of the present invention, the grayscale value may be affected by the change of lighting conditions of the target cigarette pack on the production line, resulting in inaccurate segmentation. For example, if the target cigarette pack is photographed under strong light, the shadow part may be mistakenly identified as a noise defect. In an image, noise usually appears as randomly distributed high or low grayscale value pixels. Therefore, it is necessary to select an appropriate dynamic adjustment threshold P t , Effectively suppress noise and improve image quality.

[0080] The steps to generate a binary cigarette pack texture image are: for each partition i, use the calculated threshold P t To determine the binary result of the pixel; traverse each pixel in the cigarette pack texture image and compare its gray value with the corresponding threshold P t For each pixel, if the gray value is greater than or equal to the dynamic adjustment threshold P t , then the grayscale value of the pixel in the binary image is set to the maximum value (usually 255, representing white), indicating that the area is the highlight or brighter part of the target cigarette pack and contains important texture information. If the grayscale value is less than the dynamic adjustment threshold P t , the grayscale is set to the minimum value (usually 0, representing black), indicating that the area is the darker part or background of the cigarette pack. After completing the classification of all pixels, the generated binary image will clearly distinguish the different areas of the cigarette pack, providing intuitive visual information for subsequent quality control and defect monitoring.

[0081] S1-23. Calculate the focus error as follows: for each monitored feature pixel, calculate the deviation between its position in the current binary cigarette pack texture image and the ideal position, that is, for each monitored and identified feature pixel, use Hough transform to locate its actual position in the binary cigarette pack texture image, calculate the ideal position of each feature pixel, and obtain the focus error E used to evaluate the focus accuracy.

[0082] Based on the above technical concepts, it should be noted that in the cigarette factory's cigarette pack inspection process, after generating a binary cigarette pack texture image, it is necessary to further verify the focus error by measuring the positions of several pixels in the image. Ignoring this focus error and the actual positioning of the feature pixels will lead to a series of serious problems: a. Inaccurate focus will blur the cigarette pack texture image, thereby losing critical texture information. This not only affects the accurate extraction of cigarette pack texture features, but also prevents the automated defect detection system on the subsequent production line from identifying subtle defects such as scratches or color unevenness. b. Furthermore, blurred cigarette pack texture images can directly lead to cigarette pack classification errors, increasing classification uncertainty during the production process, requiring additional image acquisition, and increasing storage and processing costs. Therefore, the ideal position of the feature pixel can be calculated through the Hough transform. If the deviation between the actual position and the ideal position (focus error E) is within an acceptable range, the focus is considered accurate. If the deviation exceeds a preset threshold, the system will automatically adjust the focus to ensure that the image quality meets monitoring requirements.

[0083] In the specific implementation, the first step is to build a grayscale value based on the average texture The matching algorithm evaluates the actual binary cigarette pack texture image F in each partition i. S Compared with the ideal binary cigarette pack texture image F T The similarity between them is calculated by calculating the average texture gray value difference ΔD of each feature pixel p,q , quantifies the degree of texture matching between two images. The smaller the difference, the more similar the two images are in that area:

[0084]

[0085] Where U and V represent the length and width of the area to be detected, that is, the number of pixels in the horizontal and vertical directions, respectively. It can be understood that the area to be detected is the small area divided from the cigarette pack texture image. This area is the basic unit for image processing and analysis. By partitioning the entire cigarette pack texture image, the image processing process can be more finely controlled, and the accuracy of focus error calculation and binarization processing can be improved. S (fs,ft) is the actual binary cigarette pack texture image F S Gray value at position (fs, ft), F T (fs,ft) is the ideal binary cigarette pack texture image F T The grayscale value at position (fs, ft) should be noted that in image processing, the grayscale value is a numerical value that describes the brightness of a pixel, usually ranging from 0 (black) to 255 (white). In this embodiment, the position (fs, ft) is specifically a pixel position in the binary cigarette pack texture image, and the grayscale value of the position is the color information of the pixel (the same as the grayscale value of the pixel (pj ,q j )) are different applications of the same concept. It is a double summation of fs from 0 to U-1 and ft from 0 to V-1 when traversing each pixel in the entire area to be detected.

[0086] Secondly, compare the average texture gray value difference ΔD p,q , find the pixel position that minimizes the difference as the actual binary cigarette pack texture image F representing the actual focus error S Compared with the ideal binary cigarette pack texture image F T The best matching position (x, y) is used to obtain the focus error E that needs to be adjusted: E H =F S (x,y)-F T (x Hi ,y) (18),E V =F S (x,y)-F T (x,y Vi ) (19), Where x Hi 、y Vi 、x D1i and y D2i They are the ideal binary cigarette pack texture images F T At the positions of the horizontal H, vertical V, diagonal D1, and diagonal D2, x, y, x D1 and y D2 are the actual binary cigarette pack texture images F S At this point, corresponding correction and adjustment measures can be taken to ensure the quality and accuracy of the image.

[0087] S1-24. Adjust the focus of the binocular camera and verify it. The operation is as follows: judge and adjust the focal length based on the obtained focus error. After adjustment, recapture the binary cigarette package texture image and repeat the feature pixel monitoring and focus error calculation process until the current focus error is reduced to within the system preset range. Verify the focus effect to ensure that the feature pixels are clear and / or the edges are sharp, and the focus error is minimized. It should be noted that in cigarette package quality inspection, the minimum acceptable clarity of key features (such as cigarette package edges and textures) is 0.05 mm, and the resolution of the binocular camera and sensor is 0.1 mm / pixel. Based on this information, the system preset range can be set to 0.1 mm to ensure that in most cases, the focus error will not affect the recognition of key features. If the monitoring system needs to identify very small defects or texture changes, the system preset range should be correspondingly smaller.

[0088] Based on the above technical concepts, it is understandable that after focus adjustment, the focus error has been reduced to an acceptable range and the image quality meets monitoring requirements, so additional image acquisition is not required, reducing computing power. However, since each cigarette pack image is different, powerful data processing capabilities are required to store and process the classification and identification of a large amount of cigarette pack image data.

[0089] Based on this, as an embodiment of the present invention, the proposed cigarette pack texture recognition method further includes:

[0090] S2. Select CNN as the main model. Convolutional neural network model is constructed with a network architecture that includes multiple convolutional layers for feature extraction, pooling layers to reduce feature dimensionality and computational complexity, and fully connected layers for classification. This architecture accurately identifies the surface texture features of cigarette packs. ReLU activation function is added to the network architecture to increase nonlinearity and prevent overfitting. The operation is as follows:

[0091] S2-1. In a cigarette pack texture image preprocessing stage, determining whether the input cigarette pack texture image is a binary cigarette pack texture image. If it is a binary image, skipping normalization processing and directly proceeding to subsequent processing steps; otherwise, performing normalization processing first and then proceeding to subsequent processing steps;

[0092] S2-2. For binary images, since they only contain two grayscale levels, the preprocessing step of subtracting the mean RGB is not applicable. However, if the binary image is converted from a color image or needs further processing to adapt to a specific process, subtracting the mean can reduce the noise in the image. Ensure that the image size and format match the requirements of the network input layer. Then determine the input layer of the network architecture, which is usually a convolutional layer to receive the preprocessed cigarette pack texture image, and set the input layer size to match the size of the preprocessed cigarette pack texture image;

[0093] S2-3. After initializing the network architecture weights, begin training the CNN model using the binarized cigarette pack texture images pre-processed in the focusing and / or binarization steps. It's understandable that these images have already been prepared through the focusing and binarization steps. During training, the CNN automatically learns to extract useful features from the images. Predictions are calculated through forward propagation, followed by backpropagation to adjust the weights to minimize a loss function, such as cross-entropy loss. Optimization algorithms, such as Adam or SGD, are then used to update the network weights. Cross-validation is then used to evaluate the model's performance on unseen data. Hyperparameters are adjusted accordingly, completing CNN model evaluation and optimization to ensure accurate monitoring of the target cigarette packs.

[0094] In one embodiment of the present invention, during the CNN model evaluation and optimization stage, it is first necessary to evaluate the model performance on an independent test set, using indicators such as accuracy, precision, and recall to analyze images that are misclassified by the model. Then, based on the evaluation results and error analysis, the network architecture or hyperparameters are adjusted, such as adding more convolutional layers, adjusting the learning rate, or using data enhancement technology to improve the recognition accuracy of the model. Through the above steps, an efficient CNN model can be constructed and optimized to monitor features in tens of thousands of cigarette packs, such as packaging defects, printing errors, etc., to ensure high-quality monitoring of tobacco products.

[0095] As an embodiment of the present invention, the proposed cigarette packet texture recognition method further includes: S3, generating an identification report including at least the monitoring result and monitoring time information according to the monitoring result.

[0096] As a second aspect of the present invention, a cigarette package texture recognition system based on a binocular camera and a convolutional neural network, a memory, and a processor are proposed, wherein the memory includes a cigarette package texture recognition program based on the binocular camera and the convolutional neural network, and when the cigarette package texture recognition program based on the binocular camera and the convolutional neural network is executed by the processor, a cigarette package texture recognition method is implemented.

[0097] The technical scope of the present invention is not limited to the contents of the above description. Those skilled in the art can make various deformations and modifications to the above embodiments without departing from the technical idea of the present invention, and these deformations and modifications should all fall within the protection scope of the present invention.

Claims

1. A cigarette packet texture recognition method based on a binocular camera and a convolutional neural network, characterized by: Including steps: S1. Rapidly capture the texture image of the cigarette package using a high-speed binocular camera. The operation is as follows: S1-1. Based on the width W and height H of the target cigarette package, the focal length f of the binocular camera, and the detail size D on the target cigarette package, determine and establish a geometric constraint relationship between the binocular camera and the target cigarette package to ensure that the binocular camera can clearly and fully capture all details of the target cigarette package; S1-2. Construct a focusing strategy to determine the minimum focal length f that meets the basic viewing angle θ and resolution R required for binocular camera detection. min , and use the minimum focal length f min Calculate the viewing angle θ required to actually cover the target cigarette pack width and height on the tobacco production line w and θ h , performing a focusing operation so that the cigarette pack texture image generates a binary cigarette pack texture image representing the surface texture features of the cigarette pack in the clearest focus state; S2. Select CNN as the primary model and build a convolutional neural network model with a network architecture including multiple convolutional layers, pooling layers, and fully connected layers to accurately identify the surface texture features of cigarette packs and ensure the monitoring of target cigarette packs; S3. Generate an identification report containing at least the monitoring results and monitoring time information according to the monitoring results, wherein the minimum focal length f min The calculation process is based on the focus strategy constructed above, with the resolution R as the end condition, and is obtained through iterative solution. The specific process is: First, based on the detail size D to be monitored on the target cigarette pack and the corresponding pixel size d on the binocular camera sensor, the required resolution R of the binocular camera is determined as the end condition of the focusing strategy; Secondly, select an initial estimated focal length f0 based on experience or the recommended settings of the binocular camera, and calculate the angle θ required for the binocular camera to cover the width and height of the target cigarette pack. w and θ h ; Again, according to the calculated viewing angle θ w and θ h , determine the maximum viewing angle θ max , the distance between the binocular camera and the target cigarette pack is calculated by the constraint relationship of focal length f, and the actual resolution R required is determined based on the appropriate distance between the binocular camera and the target cigarette pack. act ; Finally, by comparing the actual resolution R act The resolution target value R is set according to the detail recognition requirements of the cigarette package monitoring system. tar , perform the step of fine-tuning the focus, where If R act <R tar , you need to fine-tune the focus: Repeat the above steps until R act >R tar , end, otherwise, directly record the final focal length end.

2. The cigarette package texture recognition method based on a binocular camera and a convolutional neural network according to claim 1, characterized in that: The specific process of constructing the focusing strategy is as follows: First, the cigarette pack texture image is acquired and preprocessed: Determine the focus position of the binocular camera to ensure that the acquisition area defined by the width W and height H of the target cigarette pack can be clearly captured. Use the binocular camera to capture the cigarette pack texture image reflected by the laser on the target cigarette pack. Within the acquisition area, perform gradient calculation on the pixels of the cigarette pack texture image in the four directions of horizontal H, vertical V, diagonal D1, and diagonal D2 to identify changes in the cigarette pack texture. Select a pixel point (p, q) in the cigarette pack texture image as the starting point for analysis, and define a diffusion radius r. By calculating the Euclidean distance between each pixel point and the starting point, apply the diffusion coefficient for weight adjustment to further determine the precise area for analyzing the cigarette pack texture within the acquisition area, ensuring that the precise analysis area covers the key texture features of the target cigarette pack. Perform denoising and contrast enhancement preprocessing on the captured cigarette pack texture image within the precise area to improve image quality. Secondly, cigarette pack texture image processing and feature extraction: The acquired cigarette pack texture image is converted into a grayscale image to simplify the data and reduce the computational complexity. The grayscale image is divided into three partitions: highlight, brighter, and darker according to the grayscale value. The custom highlight partition corresponds to the high-reflection area of the target cigarette pack, the brighter partition corresponds to the medium-reflection area of the target cigarette pack, and the darker partition corresponds to the low-reflection and / or shadow area of the target cigarette pack. The Sobel operator in the four directions of horizontal H, vertical V, diagonal D1, and diagonal D2 is applied to each partition to extract the surface texture features of the cigarette pack image in each partition, and the texture sensitivity coefficient T that characterizes the texture change of the cigarette pack is obtained. sf , based on the texture sensitivity coefficient T sf Generate a binary cigarette pack texture image; Again, calculate the focus error: For each monitored feature pixel, the deviation between its actual position and ideal position in the current binary cigarette pack texture image is calculated to obtain the focus error E used to evaluate the focus accuracy, thereby determining the accuracy of the generated binary cigarette pack texture image; Finally, adjust the focus of the binocular camera and verify: The focal length is judged and adjusted based on the obtained focus error E. After adjustment, the binary cigarette pack texture image is recaptured and the feature pixel monitoring and focus error calculation process is repeated until the current focus error is reduced to within the system preset range. The focus effect is verified to ensure that the feature pixels are clear and / or the edges are sharp, and the focus error is minimized.

3. The cigarette package texture recognition method based on a binocular camera and a convolutional neural network according to claim 2, characterized in that: During the acquisition and preprocessing of cigarette pack texture images, gradient calculations are performed on the pixels of the cigarette pack texture images in four directions. The specific operations for identifying cigarette pack texture changes are as follows: Horizontal gradient of the cigarette pack texture image The cigarette pack texture changes are identified by comparing the brightness difference between each pixel (p, q) in the cigarette pack texture image and the pixels in the adjacent columns in the horizontal direction: Vertical gradient for cigarette pack texture image By comparing the brightness difference of adjacent rows in the same row, we can identify the texture changes of cigarette packs: Diagonal gradient for cigarette pack texture image and The texture changes of cigarette packs are identified by comparing the brightness differences of diagonally adjacent pixels (p, q) from the upper left to the lower right and from the upper right to the lower left: Where h p,q is the pixel value at the pixel position (p,q) in the cigarette pack texture image.

4. The cigarette package texture recognition method based on a binocular camera and a convolutional neural network according to claim 2, characterized in that: In the process of cigarette packet texture image processing and feature extraction, the specific operations for implementing grayscale post-partitioning corresponding to the reflective area are as follows: First, a grayscale image G(p,q) with a single grayscale value is generated by weighted averaging the red, green, and blue channels of the obtained cigarette pack texture image. Secondly, determine grayscale thresholds T1 and T2, where the custom grayscale threshold T1 is used to distinguish darker and brighter partitions, and the grayscale threshold T2 is used to distinguish brighter and highlighted partitions, and divide the grayscale image G(p,q) into three partitions according to the grayscale thresholds; Again, the partitioning result is applied to the top, bottom, left, right, front and back of the target cigarette pack to obtain the surface partitioning result.

5. The cigarette package texture recognition method based on a binocular camera and a convolutional neural network according to claim 2 or 4, characterized in that: In the process of cigarette pack texture image processing and feature extraction, the Sobel operator is used to extract the surface texture features of the cigarette pack texture image in each partition. This is achieved by calculating the grayscale change rate of the cigarette pack texture image in each partition in the horizontal H, vertical V, diagonal D1, and diagonal D2 directions. The specific operation is as follows: A 3×3 Sobel operator matrix is constructed in the four directions of horizontal H, vertical V, diagonal D1, and diagonal D2 respectively, wherein the middle value in the 3×3 Sobel operator matrix is set to 0 to indicate that the Sobel operator is insensitive to changes in the four directions of horizontal H, vertical V, diagonal D1, and diagonal D2, and at the same time, the positive and negative values in the Sobel operator are set to represent the brightness changes of the cigarette pack texture image in each partition in the four directions. When it is a positive value, it means that the brightness change in the horizontal H direction increases from left to right, the brightness in the vertical V direction increases from top to bottom, the brightness change in the diagonal D1 direction increases from upper left to lower right, and the brightness change in the diagonal D2 direction increases from upper right to lower left. When it is a negative value, the brightness change in the four directions decreases.

6. The cigarette package texture recognition method based on a binocular camera and a convolutional neural network according to claim 2, characterized in that: The texture sensitivity coefficient T that characterizes the texture change of the cigarette pack is obtained sf The specific operations are as follows: First, by calculating the pixel points (p j ,q j ) is weighted according to its gradient amplitude in four directions, and the average texture grayscale value of the cigarette pack texture image in each partition is calculated Where h i (p j ,q j ) is the pixel point in the i-th partition (p j ,q j ) grayscale value, N i is the pixel point in the i-th partition (p j ,q j ) quantity, W ij is the pixel point in the i-th partition (p j ,q j ) is calculated based on the gradient magnitude; Secondly, according to the average texture gray value And the brightness B and contrast C of the cigarette pack texture image in the partition, calculate the texture sensitivity coefficient T sf : Where a and b are the texture sensitivity coefficients T sf The minimum and maximum values of G(p j ,q j ) is the pixel point (p j ,q j ) is the comprehensive gradient amplitude in four directions, β is the adjustment coefficient, which is used to balance the brightness B and contrast C to the texture sensitivity coefficient T sf The influence of texture sensitivity coefficient T sf The value range is 7. The cigarette package texture recognition method based on a binocular camera and a convolutional neural network according to claim 6, characterized in that: Based on the texture sensitivity coefficient T sf When generating a binary cigarette pack texture image, it is necessary to dynamically adjust the threshold value P by setting t Then, for each partition i, according to the pixel points (p j ,q j ) grayscale value and dynamic adjustment threshold P t The comparison results are classified to generate a binary cigarette pack texture image representing the surface texture features of the target cigarette pack in different areas, where 8. The cigarette package texture recognition method based on a binocular camera and a convolutional neural network according to claim 1, characterized in that: All the details at least include the clarity and integrity of the pattern, text, and barcode on the target cigarette package, as well as any blurring, missing prints, or ghosting, and the integrity of the sealing seams or folds. The detail dimension D is the minimum dimension when the seal is not tight and / or the seams are uneven. The geometric constraint relationships constructed include: Ensure that the binocular camera's viewing angle meets the viewing angle constraints required to cover the width and height of the target cigarette pack; ensure that the binocular camera's viewing angle can cover the focal length f constraint required for the maximum size of the target cigarette pack; ensure that the binocular camera's resolution R meets the resolution constraint required to capture the detail size D on the target cigarette pack; and the distance constraint SD between the binocular camera and the target cigarette pack.

9. A cigarette packet texture recognition system based on a binocular camera and a convolutional neural network, characterized by: include: A memory and a processor, wherein the memory includes a cigarette pack texture recognition program based on a binocular camera and a convolutional neural network, and when the cigarette pack texture recognition program based on a binocular camera and a convolutional neural network is executed by the processor, the cigarette pack texture recognition method according to any one of claims 1 to 8 is implemented.

Images