Cigarette quantity and type verification method based on image recognition

Abstract

The invention discloses a cigarette number and type verification method based on image recognition, which comprises the following steps: S1, image acquisition and import: acquiring cigarette appearance information through an image acquisition device, S2, original image preprocessing: preprocessing the acquired image by adopting an image enhancement technology, S3, cigarette package characteristic extraction, S4, cigarette package characteristic identification, S5, cigarette package characteristic identification, S5, cigarette package characteristic identification, S6, cigarette package characteristic identification, and S6, cigarette package characteristic identification. Extracting characteristic information of the image preprocessed in the S2; and S4, identifying the quantity and the type of the cigarettes. According to the method, the current cigarette state is judged, different calculation methods are switched and executed, especially for cigarettes which have sheltered areas and cannot be completely exposed, the positions and postures of the cigarettes in the 3D space are predicted through the deep learning model, the recognition difficulty caused by sheltering is broken through, meanwhile, through modeling analysis of the overall spatial form, the recognition accuracy of the cigarettes is improved, and the recognition efficiency of the cigarettes is improved. The number of cigarettes can be accurately calculated, and the adaptability of the system to complex actual scenes is enhanced.

Description

Technical Field

[0001] The present invention relates to the technical field of image recognition, and in particular to a method for verifying the quantity and type of cigarettes based on image recognition. Background Art

[0002] Image recognition refers to the technology of using computers to process, analyze and understand images to identify various objects, scenes, text and other information in the images. It is based on digital image processing technology and machine learning algorithms. The computer first converts the image into a digital matrix and extracts features such as color, texture, shape, etc. Then it is classified or detected through the trained model.

[0003] In the prior art, people often determine the number of cigarettes by analyzing images of cigarette packages. For example, on an automated cigarette production and packaging line, a camera is used to capture images of the inside of a cigarette packaging box or a whole pack of cigarettes. By identifying and counting individual cigarettes in the image, the number of cigarettes can be quickly counted, with the accuracy being able to count the number of cigarettes per pack and the number of packs in the whole box, which greatly improves the efficiency of the production process and reduces the error of manual counting.

[0004] However, in actual use, the arrangement of cigarettes and their cigarette packaging is different. Some cigarettes will appear slightly messy as a whole due to the vibration generated by the transportation device during transportation. At this time, if the number of cigarettes is determined according to the neatly arranged calculation method, a large number of tilted and obscured cigarettes cannot be effectively identified, thus affecting the final calculation result. In view of this, we propose a cigarette quantity and type verification method based on image recognition. Summary of the invention

[0005] In view of the shortcomings of the prior art, the present invention provides a method for verifying the number and type of cigarettes based on image recognition, which solves the problem in the prior art that if the number of cigarettes is determined completely according to the calculation method of neat arrangement, there will be a large number of tilted and obscured cigarettes that cannot be effectively identified.

[0006] To achieve the above objectives, the present invention is implemented through the following technical solutions: a method for verifying the number and type of cigarettes based on image recognition, comprising the following steps:

[0007] S1: Image acquisition and import

[0008] The appearance information of the cigarette is collected by an image acquisition device, and the collected original image is transmitted in a lossless format and stored in a local image database, and metadata information including time and location during the image acquisition process is recorded;

[0009] S2: Raw image preprocessing

[0010] The collected images are preprocessed using grayscale conversion, noise filtering and image enhancement techniques;

[0011] S3: Cigarette packaging feature extraction

[0012] Extract characteristic information of the preprocessed image in S2, including shape features, texture image features and text features;

[0013] S4: Identification of cigarette quantity and type

[0014] According to the extracted cigarette packaging shape features, the placement of cigarettes is determined, and different recognition methods are divided according to different placement methods. At the same time, the current cigarette type is determined according to the characteristic information obtained in S3;

[0015] S5: Verification and correction of recognition results

[0016] Establish a verification mechanism to conduct a second verification of the identification results and mark the final identification results as "accurate" or "questionable". When there are abnormalities in the verification results, re-execute the cigarette quantity and type identification, and record the suspicious results and correction process in the log file.

[0017] Preferably, during the S1 image acquisition and import, a mechanical transmission device is used to adjust the position of the image recording device, and its image acquisition area is controlled to fully cover the area containing cigarettes. After the mechanical component adjustment is completed, the image recording device synchronously calibrates the shooting properties, including focal length, exposure and contrast information.

[0018] Preferably, in the S2 original image preprocessing, image partitions are established for different areas in the image, and image preprocessing is performed independently according to the partition information, including the following steps:

[0019] S201: Establishing image partitioning strategy

[0020] The original image is divided into multiple rectangular partitions based on spatial location partitioning according to different cigarette placement areas or shelf layouts in the image;

[0021] S202: Establishing image feature partitions

[0022] For each partition, extract the corresponding image features, including color, texture, and shape, and classify the partitions based on these features;

[0023] S203: Performing partition image preprocessing

[0024] Perform independent preprocessing operations on each classified image partition, including filtering and denoising, contrast enhancement, and brightness adjustment, to reduce the interference between images in different regions during preprocessing;

[0025] S204: Image partition preprocessing result integration

[0026] The preprocessed images of each partition are integrated to form a unified preprocessed image, and adaptive adjustments are made to the edges between the partitions.

[0027] Preferably, in the integration of the image partition preprocessing results in S204, the integrated image is checked and optimized as a whole to check whether there are image quality problems in local areas due to improper partitioning, including residual noise or uneven contrast in certain edge areas. If such problems are found, local repair or adjustment algorithms are used to process the specific areas.

[0028] Preferably, in the S3 cigarette package feature extraction, the Canny edge detection algorithm is used to extract the edge contour of the cigarette, the texture analysis method based on the gray level co-occurrence matrix is ​​used to extract the texture characteristics of the cigarette package surface, and the optical character recognition technology is used to extract the text information on the cigarette package.

[0029] Preferably, the arrangement of cigarettes determined in the S4 cigarette quantity and type recognition is divided into "neat arrangement" or "disordered arrangement". When it is determined to be "neat arrangement", the system estimates the number of cigarettes by calculating the length of the package edge contour in the image and the known length information of a single pack of cigarettes, combined with the gap characteristics between the packages.

[0030] Preferably, in the S4 cigarette quantity and type identification, when it is determined that the current cigarette arrangement is "disordered arrangement", the system will reconstruct the 3D model of the cigarettes in the area based on the visual sensing information collected and acquired, and then calculate the number of cigarettes in the area based on the volume information in the 3D model and the volume data ratio of a single pack of cigarettes.

[0031] Preferably, in the S4 cigarette quantity and type recognition, for cigarettes that are blocked and cannot be fully revealed, a deep learning model is used to predict the position and posture of each individual cigarette in 3D space, and the established learning model is self-optimized based on the verification feedback information in the S5 recognition result verification and correction.

[0032] Preferably, in the S5 recognition result verification and correction, the edge curves that have been identified and successfully used to determine the shape of the cigarette package are marked with green lines, the edge curves that have been identified and extracted but not used in the current recognition process are marked with red lines, and a secondary recognition judgment is performed on the red unused curves in the image.

[0033] Preferably, multi-sensor fusion verification is used in the verification and correction of the S5 recognition result, that is, the theoretical total weight is calculated based on the average weight information of a single pack of cigarettes of different brands and types, combined with the number of cigarettes recognized by the image, and the calculated theoretical total weight is compared with the weight actually measured by the weight sensor as an additional verification standard.

[0034] The present invention provides a method for verifying the number and type of cigarettes based on image recognition.

[0035] Beneficial effects:

[0036] 1. The present invention switches to execute different calculation methods by judging the current state of cigarettes. In particular, for cigarettes that cannot be fully revealed due to occlusion areas, the deep learning model is used to predict their position and posture in 3D space, breaking the recognition dilemma caused by occlusion. At the same time, through modeling and analysis of the overall spatial form, the number of cigarettes can be accurately calculated, enhancing the system's adaptability to complex actual scenarios.

[0037] 2. The present invention can process the dark or bright areas in the image in a targeted manner by partitioning the original image during the preprocessing process, thereby avoiding over-optimization of other areas of the image due to the optimization of the reflective and blocked areas of the cigarette packaging. At the same time, in the process of integrating all the partitioned images, the integration edges are partially repaired or adjusted, thereby avoiding distortion of the image integration edges during the image integration process.

[0038] 3. The present invention can sort out the features involved in the recognition process by marking the used and unused edge curves, and perform secondary recognition and judgment on the red unused curves to avoid missing important shape feature information. The secondary recognition can fully tap these potentially useful features, thereby more accurately determining the packaging shape of cigarettes, thereby improving the accuracy of cigarette type recognition. In addition, quality inspectors can quickly understand the working status of the recognition system and determine whether there are potential problems by viewing the marked images. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 This is a flow chart of the cigarette quantity and type verification method based on image recognition;

[0040] Figure 2 This is a flow chart of the image partitioning part of the S2 original image preprocessing of the present invention;

[0041] Figure 3 This is a recognition flow chart of the cigarette quantity recognition process of the present invention. DETAILED DESCRIPTION

[0042] The following will be combined with the drawings in the specification of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0043] Example:

[0044] Please refer to the attached Figure 1 - Attachment Figure 3 The embodiment of the present invention provides a method for verifying the number and type of cigarettes based on image recognition, comprising the following steps:

[0045] S1: Image acquisition and import

[0046] The appearance information of the cigarette is collected by an image acquisition device, and the collected original image is transmitted in a lossless format and stored in a local image database, and metadata information including time and location during the image acquisition process is recorded;

[0047] S2: Raw image preprocessing

[0048] The collected images are preprocessed using grayscale conversion, noise filtering and image enhancement techniques;

[0049] S3: Cigarette packaging feature extraction

[0050] Extract characteristic information of the preprocessed image in S2, including shape features, texture image features and text features;

[0051] S4: Identification of cigarette quantity and type

[0052] According to the extracted cigarette packaging shape features, the placement of cigarettes is determined, and different recognition methods are divided according to different placement methods. At the same time, the current cigarette type is determined according to the characteristic information obtained in S3;

[0053] S5: Verification and correction of recognition results

[0054] Establish a verification mechanism to conduct a second verification of the identification results and mark the final identification results as "accurate" or "questionable". When there are abnormalities in the verification results, re-execute the cigarette quantity and type identification, and record the suspicious results and correction process in the log file.

[0055] In the S1 image acquisition and import, a mechanical transmission device is used to adjust the position of the image recording device, and its image acquisition area is controlled to fully cover the area containing cigarettes. After the mechanical component adjustment is completed, the image recording device synchronously calibrates the shooting properties, including focal length, exposure and contrast information.

[0056] In the S2 original image preprocessing, image partitions are established for different areas in the image, and image preprocessing is performed independently according to the partition information, including the following steps:

[0057] S201: Establishing image partitioning strategy

[0058] The original image is divided into multiple rectangular partitions based on spatial location partitioning according to different cigarette placement areas or shelf layouts in the image;

[0059] S202: Establishing image feature partitions

[0060] For each partition, extract the corresponding image features, including color, texture, and shape, and classify the partitions based on these features;

[0061] S203: Performing partition image preprocessing

[0062] Perform independent preprocessing operations on each classified image partition, including filtering and denoising, contrast enhancement, and brightness adjustment, to reduce the interference between images in different regions during preprocessing;

[0063] S204: Image partition preprocessing result integration

[0064] The preprocessed images of each partition are integrated to form a unified preprocessed image, and adaptive adjustments are made to the edges between the partitions.

[0065] In the integration of the image partition preprocessing results in S204, the integrated image is checked and optimized as a whole to check whether there are image quality problems in local areas caused by improper partitioning, including residual noise or uneven contrast in some edge areas. If such problems are found, local repair or adjustment algorithms are used for processing the specific areas. For the image aggregation process, we propose the following integration algorithm and repair algorithm:

[0066] Integration algorithm:

[0067] Assume that the pixel point at the boundary of adjacent partition images I1 and I2 is (x, y). Calculate the gradient vector of I1 at (x, y) The gradient vector of I2 at (x,y)

[0068] Fusion weight

[0069] The integrated pixel value I combined (x,y)=w1I1(x,y)+w2I2(x,y);

[0070] For example,

[0071] but w2=1-w1. If I1(x,y)=80,I2(x,y)=90, I can be calculated based on the above weights. combined (x, y) values. After partitioning different areas of the shelf where cigarettes are stored, the above algorithm is used to integrate the images, which can make the entire shelf image more natural and accurate in subsequent recognition analysis, avoid recognition errors caused by partition boundary problems, and help to more accurately extract cigarette-related features for verification;

[0072] Adaptive Histogram Equalization Algorithm

[0073] First, divide the image into non-overlapping sub-blocks. Suppose the width of the image is W and the height is H. If it is to be divided into M×N sub-blocks, then the width of each sub-block is high For example, for an image with a width of 640 pixels and a height of 480 pixels, if it is to be divided into 8×6 sub-blocks:

[0074] The width of each sub-block Pixels, the height of each sub-block Pixels;

[0075] For each sub-block B m n(m=1,2,…,M,n=1,2,…,N), calculate its grayscale histogram H m n(k) (k=0,1,…,L-1, L is the gray level);

[0076] Histogram H mn (k) represents sub-block B mn The number of pixels with gray value k;

[0077] Let sub-block B mn The grayscale value of the pixel (x, y) is I(x, y), then the process of calculating the histogram can be simply completed by traversing all pixels in the sub-block in a loop and counting the number of times each grayscale value appears;

[0078] sub-blockB mn Cumulative distribution function of :

[0079]

[0080] Where N mn It is sub-block B mn The total number of pixels in, the cumulative distribution function CDF mn (k) represents sub-block B mn The proportion of pixels with a medium intensity value less than or equal to k;

[0081] For example, if sub-block B mn The histogram H mn =[10,20,30,40](assuming gray level L=4), total number of pixels N mn =10+20+30+40=100, then

[0082] For sub-block B mn The original grayscale value of the pixel point (x, y) in is I(x, y), and the grayscale value after equalization is I equal ized The calculation formula for (x,y) is:

[0083]

[0084] The purpose of this formula is to map the original grayscale value to the new grayscale value according to the cumulative distribution function, thereby achieving contrast enhancement;

[0085] For example, for an 8-bit grayscale image (L=256), if CDF mn (I(x,y))=0.4, then

[0086]

[0087] In the S3 cigarette package feature extraction, the Canny edge detection algorithm is used to extract the edge contour of the cigarette, the texture analysis method based on the gray level co-occurrence matrix is ​​used to extract the texture characteristics of the cigarette package surface, and the optical character recognition technology is used to extract the text information on the cigarette package.

[0088] The arrangement of cigarettes determined in the S4 cigarette quantity and type recognition is divided into "neat arrangement" or "disordered arrangement". When it is determined to be "neat arrangement", the system estimates the number of cigarettes by calculating the length of the package edge contour in the image and the known length information of a single pack of cigarettes, combined with the gap characteristics between the packages. For cigarettes determined to be "neat arrangement", we propose the following algorithm:

[0089] Establish a filtering formula to maintain image smoothness and reduce the impact of noise on edge detection. The formula is:

[0090]

[0091] Where (x, y) is the pixel coordinate in the image and σ is the standard deviation of the filter;

[0092] For example, when σ = 1, the weight of the pixel with coordinates (0,0) in the image after filtering is calculated:

[0093]

[0094] Next, we calculate the gradient magnitude and direction. For a pixel (x, y) in the image, the gradient magnitude is

[0095]

[0096] Gradient direction:

[0097]

[0098] Where I(x,y) is the pixel value of the image at point (x,y);

[0099] and are the partial derivatives in the x and y directions

[0100] Use the Sobel operator to calculate the partial derivatives. The template of the Sobel operator in the x direction is The template in the y direction is

[0101] Finally, non-maximum suppression and double threshold detection are performed to determine edge pixels. Non-maximum suppression is to check whether the pixel is a local maximum in the gradient direction, and double threshold detection is to set a high threshold T h and low threshold T i (T h >T l ) to filter edge pixels, greater than T h Pixels with a value smaller than T are edge pixels. l The pixels between them are not edge pixels, and the pixels between them are also considered edge pixels if they are connected to the edge pixels.

[0102] Suppose the pixel coordinates on the edge contour of the package obtained after edge detection are (x i ,y i ), i = 1, 2, ..., n, where n is the number of edge contour pixels. The edge contour length can be calculated using the Euclidean distance formula

[0103]

[0104] The length l of a single pack of cigarettes can be obtained by measuring the pixel length of a standard cigarette package in an image in advance, assuming that it is known to be l0 pixels. The average gap length g between packages can be obtained by statistically analyzing a certain number of sample images, assuming that the statistically obtained average gap length is g0 pixels;

[0105] The formula for calculating the estimated number of cigarette packs n is: Where L is the calculated package edge contour length, l = l0, g = g0.

[0106] In the S4 cigarette quantity and type recognition, when the current cigarette arrangement is determined to be "disordered", the system will reconstruct the 3D model of the cigarettes in the area based on the acquired visual sensor information, and then calculate the number of cigarettes in the area based on the volume information in the 3D model and the volume data ratio of a single pack of cigarettes. For cigarettes determined to be "neatly arranged", we propose the following algorithm:

[0107] Model volume calculation:

[0108] First, the volume of the reconstructed 3D cigarette model needs to be calculated, and the 3D model is represented by voxels, that is, the space is divided into small cubic units, and the volume is calculated by counting the number of voxels occupied by the cigarette model;

[0109] For 3D models with complex shapes, the volume is calculated using triple integrals based on their geometry. For example, for an upper surface represented by a mathematical function z = f(x, y) and a projection area D on the xy plane, the volume formula is:

[0110] V = ∫∫ D f(x,y)dxdy

[0111] At the same time, the volume of a standard single pack of cigarettes is obtained by 3D modeling. In the process, a single pack of cigarettes is scanned using a 3D scanning device, and then the accurate volume is obtained according to the above 3D model volume calculation method. It can also be calculated theoretically according to the size of the cigarette package. Assuming that a single pack of cigarettes is approximately a rectangular parallelepiped, its length, width, and height are l (unit: meter), w (unit: meter), and h (unit: meter), respectively, then the volume of a single pack of cigarettes v = l × w × h;

[0112] The final formula for calculating the number of cigarettes N is:

[0113] Where V is the total volume of cigarettes in the reconstructed 3D model, and v is the volume of a single pack of cigarettes.

[0114] In the S4 cigarette quantity and type recognition, for cigarettes that are blocked and cannot be fully revealed, a deep learning model is used to predict the position and posture of each individual cigarette in 3D space, and the established learning model is self-optimized according to the verification feedback information in the S5 recognition result verification and correction. The self-optimization part of the established learning model adopts a deep learning algorithm, and its process formula is as follows:

[0115] Assume that the input image data is X and the size is W in ×H in ×C in (width, height, number of channels). The convolution kernel is K and the size is W k ×H k ×C in , the step size is s, and the padding is p;

[0116] The width of the output feature map high Number of channels C out Equal to the number of convolution kernels;

[0117] For a pixel point (i, j, k) in the output feature map (i represents the width index, j represents the height index, and k represents the channel index), the calculation formula is:

[0118]

[0119] For example, if there is an input image of size 32×32×3 (color image, number of channels is 3), using a 3×3×3 convolution kernel, stride 1, padding 1, then the width of the output feature map Height o ut=32, number of channels C out Assume it is 16 (depending on the number of convolution kernels).

[0120] Output layer for position and pose prediction:

[0121] If we directly predict the position (x, y, z) and posture of the cigarette in 3D space (expressed by Euler angles α, β, γ), the output layer can have 6 neurons. Assume that the final output of the network is O, with a size of 1×6, and after linear transformation (weight W out , with a bias of b out ) is obtained from the hidden layer output, that is, O = W out h L +b out (h L is the output of the last hidden layer). For the predicted position and posture, the loss function can be defined according to the specific task;

[0122] For example, the mean square error (MSE) loss function is used for position prediction. Let the predicted position be (x p ,y p ,z p ), the real position is (x g ,y g ,z g ), the MSE loss of position prediction is:

[0123]

[0124] For posture prediction, the loss function can also be used for prediction;

[0125] Self-optimizing algorithm establishment:

[0126] Assume that the multilayer perceptron has L layers, and the input of the lth layer is a l -1, weight is W l , with a bias of b l , the activation function is σ l , then the output of the lth layer is:

[0127] a l =σ l (W l a l-1 +b l )

[0128] For the loss function J, let the output of the last layer be a L , the true label is y, and the mean square error loss function is:

[0129]

[0130] When calculating the gradient, backpropagation starts from the last layer. For the Lth layer

[0131]

[0132] where σ L′ is the activation function σ L The derivative of

[0133] For the middle layer l(l <L), in

[0134] δ l =((W l+1 ) T δ l+1 )σ l′ (W l a l-1 +b l ), σ l′ is the activation function σ l The derivative of

[0135] For example, suppose there is a simple three-layer neural network with 2 neurons in the input layer, 3 neurons in the hidden layer, and 1 neuron in the output layer, and the activation function is:

[0136]

[0137] Let the input be x=[x1,x2], and the weight W 1is a 3×2 matrix, b 1 is a 3×1 vector, W 2 is a 1×3 matrix, b 2 is a 1×1 vector, and the loss function J is calculated

[0138] About W 1 and b 1 The gradient of each layer is first forward propagated to calculate the output of each layer, and then the gradient is calculated by back propagation according to the above formula;

[0139] Weight and bias updates

[0140] According to the calculated gradient, the weights and biases are updated. The update formula is:

[0141]

[0142] Where o is the learning rate. For example, in the above three-layer neural network, assuming that the learning rate α = 0.01, we can calculate and Then, update W according to the above formula 1 and b 1 , in order to reduce the value of the loss function, thus enabling self-optimization of the model.

[0143] In the S5 recognition result verification and correction, the edge curves that have been identified and successfully used to determine the shape of the cigarette package are marked with green lines, and the edge curves that have been identified and extracted but not used in the current recognition process are marked with red lines, and a secondary recognition judgment is performed on the red unused curves in the image. For the secondary recognition judgment of the red unused curves in the image, we propose the following algorithm:

[0144] For the red unused curve, its features need to be re-extracted. First, the edge detection algorithm can be used to extract the geometric features of the curve, such as length and curvature. Taking length calculation as an example, for a curve represented by discrete pixel points, if the coordinates of the curve pixel point are (x i ,y i ), i = 1, 2, ..., n, the curve length L can be approximately calculated by the following formula:

[0145]

[0146] For example, if a curve consists of four pixels with coordinates (1,1), (2,2), (3,3), and (4,4), then the length of the curve is

[0147]

[0148] (unit is pixel length);

[0149] The curvature feature is extracted by calculating the curvature of the curve at each point. For the parametric curve r(t) = (x(t), y(t)), the curvature formula is:

[0150]

[0151] In the discrete case, the curvature is estimated by calculating the derivatives through differential approximation, and then these re-extracted features can be matched with the known library of cigarette package edge curve features;

[0152] For example, using a feature vector distance measurement method, such as Euclidean distance, let the feature vector of the extracted red unused curve be F r =(L r ,κ r ,…) (including length, curvature and other features), the cigarette packaging edge curve feature vector in the feature library is F c =(L c ,κ c ,…), the Euclidean distance between the two is:

[0153]

[0154] If the distance d is less than a certain threshold value θ, it can be determined that the red unused curve is a part of the edge curve of the cigarette package and can be used for further recognition correction.

[0155] The S5 recognition result verification and correction adopts multi-sensor fusion verification, that is, the theoretical total weight is calculated based on the average weight information of a single pack of cigarettes of different brands and types, combined with the number of cigarettes recognized by the image, and the calculated theoretical total weight is compared with the actual weight measured by the weight sensor as an additional verification standard.

[0156] Although embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for verifying the number and type of cigarettes based on image recognition, characterized in that: The following steps are involved: S1: Image acquisition and import The appearance information of the cigarette is collected by an image acquisition device, and the collected original image is transmitted in a lossless format and stored in a local image database, and metadata information including time and location during the image acquisition process is recorded; S2: Raw image preprocessing The collected images are preprocessed using grayscale conversion, noise filtering and image enhancement techniques; S3: Cigarette packaging feature extraction Extract characteristic information of the preprocessed image in S2, including shape features, texture image features and text features; S4: Identification of cigarette quantity and type According to the extracted cigarette packaging shape features, the placement of cigarettes is determined, and different recognition methods are divided according to different placement methods. At the same time, the current cigarette type is determined according to the characteristic information obtained in S3; S5: Verification and correction of recognition results Establish a verification mechanism to conduct a second verification of the identification results, and mark the final identification results as "accurate" or "questionable". When there are abnormalities in the verification results, re-execute the cigarette quantity and type identification, and record the suspicious results and correction process in the log file.

2. The method for verifying cigarette quantity and type based on image recognition according to claim 1, characterized in that: In the S1 image acquisition and import, a mechanical transmission device is used to adjust the position of the image recording device, and its image acquisition area is controlled to fully cover the area containing cigarettes. After the mechanical component adjustment is completed, the image recording device synchronously calibrates the shooting properties, including focal length, exposure and contrast information.

3. The method for verifying cigarette quantity and type based on image recognition according to claim 1, characterized in that: In the S2 original image preprocessing, image partitions are established for different areas in the image, and image preprocessing is performed independently according to the partition information, including the following steps: S201: Establishing image partitioning strategy The original image is divided into multiple rectangular partitions based on spatial location partitioning according to different cigarette placement areas or shelf layouts in the image; S202: Establishing image feature partitions For each partition, extract the corresponding image features, including color, texture, and shape, and classify the partitions based on these features; S203: Performing partition image preprocessing Perform independent preprocessing operations on each classified image partition, including filtering and denoising, contrast enhancement, and brightness adjustment, to reduce the interference between images in different regions during preprocessing; S204: Image partition preprocessing result integration The preprocessed images of each partition are integrated to form a unified preprocessed image, and adaptive adjustments are made to the edges between the partitions.

4. The method for verifying cigarette quantity and type based on image recognition according to claim 3, characterized in that: In the integration of the image partition preprocessing results in S204, the integrated image is checked and optimized as a whole to check whether there are image quality problems in local areas caused by improper partitioning, including residual noise or uneven contrast in certain edge areas. If such problems are found, local repair or adjustment algorithms are used to process the specific areas.

5. The method for verifying cigarette quantity and type based on image recognition according to claim 1, characterized in that: In the S3 cigarette package feature extraction, the Canny edge detection algorithm is used to extract the edge contour of the cigarette, the texture analysis method based on the gray level co-occurrence matrix is ​​used to extract the texture characteristics of the cigarette package surface, and the optical character recognition technology is used to extract the text information on the cigarette package.

6. The method for verifying cigarette quantity and type based on image recognition according to claim 1, characterized in that: The arrangement of cigarettes determined in the S4 cigarette quantity and type recognition is divided into "neat arrangement" or "disordered arrangement". When it is determined to be "neat arrangement", the system estimates the number of cigarettes by calculating the length of the package edge contour in the image and the known length information of a single pack of cigarettes, combined with the gap characteristics between the packages.

7. A method for verifying cigarette quantity and type based on image recognition according to claim 6, characterized in that: In the S4 cigarette quantity and type identification, when it is determined that the current cigarette arrangement is "random arrangement", the system will reconstruct the 3D model of the cigarettes in the area based on the visual sensing information collected and acquired, and then calculate the number of cigarettes in the area based on the volume information in the 3D model and the volume data ratio of a single pack of cigarettes.

8. The method for verifying cigarette quantity and type based on image recognition according to claim 7, characterized in that: In the S4 cigarette quantity and type recognition, for cigarettes that are blocked and cannot be fully revealed, a deep learning model is used to predict the position and posture of each individual cigarette in 3D space, and the established learning model is self-optimized based on the verification feedback information in the S5 recognition result verification and correction.

9. The method for verifying cigarette quantity and type based on image recognition according to claim 1, characterized in that: In the S5 recognition result verification and correction, the edge curves that have been identified and successfully used to determine the shape of the cigarette package are marked with green lines, and the edge curves that have been identified and extracted but not used in the current recognition process are marked with red lines, and a secondary recognition judgment is performed on the red unused curves in the image.

10. The method for verifying cigarette quantity and type based on image recognition according to claim 1, characterized in that: The S5 recognition result verification and correction adopts multi-sensor fusion verification, that is, the theoretical total weight is calculated based on the average weight information of a single pack of cigarettes of different brands and types, combined with the number of cigarettes recognized by the image, and the calculated theoretical total weight is compared with the actual weight measured by the weight sensor as an additional verification standard.

Images