Method for classifying damaged bar codes

Abstract

The invention provides a damaged bar code classification method, and relates to the technical field of bar code classification. The method comprises the steps of collecting a bar code image; performing bar code position detection on the bar code image to obtain a bar code area image; preprocessing the bar code area image to obtain a classified image; and inputting the classification image into the classification model for bar code classification, and outputting a classification result. Firstly, an efficient input layer module is used for exciting image features to more channels and reducing the space size while the calculated amount of classified images is low, and the calculation cost of subsequent modules is reduced; according to the classification model, the classification speed is increased by compressing the number of channels of the whole network, meanwhile, through efficient channel transmission and continuous iteration fusion of middle high latitude features, the transmission efficiency between network layers is improved, information loss is reduced, and the accuracy of the whole classification result is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of barcode classification, and in particular to a method for classifying damaged barcodes. Background Art

[0002] In traditional barcode scanning and decoding methods, traversal, template matching and other methods are usually used to determine the type of barcode for decoding. For damaged barcodes (scratched, blurred, bent, etc.), barcode classification is required before repairing the barcode. In recent years, deep learning has shown excellent performance in image classification tasks compared with traditional methods. Common barcode categories include Code 128, Data Matrix, QR Code, etc. There are differences in the corresponding algorithms for barcode repair and decoding for different categories of barcodes. Classification of barcodes has an important role and significance. Methods based on deep learning usually have a large amount of calculation, resulting in certain limitations in their application in mobile devices. Model pruning, distillation and other compression model methods are usually used to improve the speed of the model, but the compressed model will have a certain decrease in accuracy, and the speed improvement is very limited. How to reduce the amount of calculation of the algorithm itself, improve the computational efficiency of the network model, and ensure a high accuracy rate in mobile devices with limited resources, while increasing the speed without decreasing the accuracy, is an urgent problem to be solved in this industry. Summary of the invention

[0003] The present invention provides a damaged barcode classification method, which is used to solve the problem in the prior art that when a mobile device uses a network model to classify damaged barcodes, the classification accuracy is low and the speed is slow.

[0004] The present invention provides a damaged barcode classification method, comprising:

[0005] Capture barcode images;

[0006] Perform barcode position detection on the barcode image to obtain a barcode area image;

[0007] Preprocess the barcode area image to obtain a classified image;

[0008] Input the classified image into the classification model for barcode classification and output the classification result.

[0009] According to a damaged barcode classification method provided by the present invention, the barcode position detection is performed on the barcode image to obtain the barcode area image, including:

[0010] Binarize the barcode image to distinguish the barcode pattern from the background;

[0011] Extract the boundary features of the barcode pattern and obtain the barcode area image.

[0012] According to a damaged barcode classification method provided by the present invention, in the binarization process, the method comprises:

[0013] Get the pixel mean in the neighborhood window of each pixel. The pixel mean is the adaptive threshold of each pixel. If the gray value of the pixel is less than or equal to the adaptive threshold, the pixel becomes white; if the gray value of the pixel is greater than the adaptive threshold, the pixel becomes black.

[0014] According to a damaged barcode classification method provided by the present invention, the shape of the barcode region image is a circumscribed rectangle of the barcode pattern.

[0015] According to a damaged barcode classification method provided by the present invention, the preprocessing step includes:

[0016] Get the width and height size information of the barcode area image, and fill the shorter side of the width and height until the width and height are consistent;

[0017] The resolution of the resized barcode region image is adjusted to 96×96 to obtain a classification image.

[0018] According to a damaged barcode classification method provided by the present invention, the classification image is input into a classification model to classify the barcode, and the classification result is output, including:

[0019] The classified image is input into the efficient input layer module for convolution operation, and the efficient input layer module outputs the initial features;

[0020] The initial features are input into the feature extraction module, and the first, second and third stage convolution operations are performed on the initial features. The feature extraction module outputs the final classification features.

[0021] The final classification features are input into the feature fully connected module, and the feature fully connected module outputs the classification results.

[0022] According to a damaged barcode classification method provided by the present invention, the efficient input layer module sequentially performs 3×3 convolution and 1×1 convolution operations on the classification image to obtain initial features.

[0023] According to a damaged barcode classification method provided by the present invention, the convolution operation in the first stage includes: performing 3×3 convolution, 3×3 convolution and 1×1 convolution operations on the initial features in sequence to obtain the first stage features;

[0024] The convolution operation of the second stage includes: performing 3×3 convolution, 1×1 convolution, 3×3 convolution and 1×1 convolution operations on the first stage features in turn to obtain the second stage preprocessing features, then performing residual connection between the first stage features and the second stage preprocessing features, and then performing 1×1 convolution operation; the convolution operation of the second stage is iterated twice to obtain the second stage features;

[0025] The convolution operation of the third stage includes: performing 5×5 convolution, 1×1 convolution, 5×5 convolution and 1×1 convolution operations on the second stage features in turn to obtain the third stage preprocessing features, and then performing residual connection between the second stage features and the third stage preprocessing features, and then performing a 1×1 convolution operation; the convolution operation of the third stage is iterated four times to obtain the final classification features.

[0026] According to a damaged barcode classification method provided by the present invention, the feature fully connected module performs 1×1 convolution and average pooling operations on the final classification features in sequence to obtain the number of feature channels, uses a fully connected layer to map the number of feature channels to barcode categories, calculates the distribution probability of each category through the Softmax activation function, selects the category with the largest distribution probability as the damaged barcode category, and outputs the barcode category.

[0027] According to a damaged barcode classification method provided by the present invention, the collecting of barcode images includes:

[0028] Use the camera to capture the video stream on the object containing the barcode and capture the current frame as the barcode image.

[0029] The damaged barcode classification method provided by the present invention is designed for mobile devices. The collected barcode image is subjected to a series of processing. An adaptive threshold method is used in the binarization process to clearly distinguish the barcode pattern and the background. The barcode area image is obtained and preprocessed to obtain a classified image. The classified image is then input into a classification model for classification. In the classification model, firstly, the classified image is stimulated to have more channels and reduced spatial size while having a low computational amount through an efficient input layer module, thereby reducing the computational cost of subsequent modules. The classification model improves the classification speed by compressing the number of channels of the overall network. At the same time, through efficient channel transmission, the intermediate high-latitude features are continuously iterated to fuse, thereby improving the transmission efficiency between network layers, reducing information loss, and improving the accuracy of the overall classification result. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0031] Figure 1 It is a schematic diagram of the flow of the damaged barcode classification method provided by the present invention;

[0032] Figure 2 It is a schematic diagram of the framework of the classification model;

[0033] Figure 3 It is a schematic diagram of the process of classifying images in the classification model;

[0034] Figure 4 It is a schematic diagram of the edge algorithm extracting the boundary features of the barcode pattern;

[0035] Figure 5 This is a module connection diagram of the damaged barcode classification device provided by the present invention. DETAILED DESCRIPTION

[0036] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0037] Combine the following Figure 1-Figure 4 A method for classifying damaged barcodes of the present invention is described, comprising:

[0038] Step S100: collecting a barcode image; in this embodiment, the method for collecting a barcode image includes: using a camera to collect a video stream on an object containing a barcode, intercepting a current frame as a barcode image, and sending the barcode image to a barcode position detection module;

[0039] Step S200: performing barcode position detection on the barcode image to obtain a barcode area image; wherein the barcode position detection module detects the position of the barcode in the barcode image, and the specific detection method is:

[0040] First, the barcode image is binarized to distinguish the barcode pattern from the background; wherein, the binarization process includes: obtaining the pixel mean value in the neighborhood window of each pixel point, the pixel mean value is the adaptive threshold value of each pixel point, if the gray value of the pixel point is less than or equal to the adaptive threshold value, the pixel point becomes white; if the gray value of the pixel point is greater than the adaptive threshold value, the pixel point becomes black; that is, white is the background, and black is the barcode pattern;

[0041] Next, the boundary features of the barcode pattern are extracted; the specific method includes: using the Canny edge detection algorithm to extract the edge of the barcode pattern; since the barcode has obvious geometric shape features, the outer contour of the barcode pattern can be extracted more accurately through edge detection;

[0042] Finally, the barcode area image is obtained; specifically, after determining the outer contour of the barcode pattern, the portion outside the outer contour of the barcode pattern is removed to obtain the barcode area image, which is in the shape of the circumscribed rectangle of the barcode pattern. It can be simply understood that the barcode area image is an image cut out based on the circumscribed rectangle.

[0043] Furthermore, the adaptive threshold is obtained by calculating the following formula (1):

[0044]

[0045] Wherein, T(x,y) is the adaptive threshold at the pixel position (x,y), I(i,j) is the gray value of the pixel position (i,j), W(x,y) represents the neighborhood window centered on the pixel position (x,y), N is the total number of pixels in the neighborhood window W(x,y), and C is the offset constant. In this application, the offset constant is set to 25 for fine-tuning the threshold.

[0046] After calculating the adaptive threshold of the pixel point, the gray value of the pixel point is compared with its adaptive threshold and processed using the following formula (2):

[0047]

[0048] In the above formula (2), if the gray value of the pixel is less than or equal to the adaptive threshold, the pixel becomes white; if the gray value of the pixel is greater than the adaptive threshold, the pixel becomes black. This method can more accurately distinguish the barcode pattern from the background.

[0049] The above steps have distinguished the barcode pattern (black) from the background (white). Therefore, the edge algorithm can be used to extract the boundary features of the barcode pattern. The edge algorithm calculation adopts the following formula (3):

[0050] box=Arearect(C)=(Cx,Cy,w,h,θ) (3)

[0051] Through the above operation, the outer contour (i.e., the circumscribed rectangle) of the barcode pattern can be obtained, where C = {(x i ,y i )|i=1,2,...,n} are the n points of the outer contour, w is the width of the circumscribed rectangle, and h is the height of the circumscribed rectangle; Figure 4 As shown, the XY axis coordinates are established at the center point of the circumscribed rectangle, and a straight line L parallel to the X axis is set. The straight line L passes through a vertex of the circumscribed rectangle, and the vertex is located at the bottom of the center point of the circumscribed rectangle. There are two sides where the vertex is located. Take the side that is inclined to the positive side of the X axis relative to the Y axis. The angle between this side and the straight line L is θ.

[0052] Step S300: preprocess the barcode region image to obtain a classified image; specifically, the preprocessing step includes: obtaining the width and height dimension information of the barcode region image, filling the shorter side of the width and height until the width and height are consistent; adjusting the resolution of the resized barcode region image to 96×96 to obtain a classified image. It is worth noting that the width of the barcode region image is equal to the width of the circumscribed rectangle, and the height of the barcode region image is equal to the height of the circumscribed rectangle.

[0053] The purpose of preprocessing is to better classify the classified image in the subsequent classification model. In this application, the following formula (4) is used for preprocessing:

[0054]

[0055] Among them, nW is the width of the adjusted classified image, nH is the height of the adjusted classified image, S is the resolution, and in this application, in order to adapt the classification model, S is taken as 96; W is the width of the original barcode area image, and H is the height of the original barcode area image.

[0056] Step S400: Input the classified image into the classification model to perform barcode classification and output the classification result. Specifically, it includes:

[0057] The classified image is input into the efficient input layer module for convolution operation, and the efficient input layer module outputs the initial features;

[0058] The initial features are input into the feature extraction module, and the first, second and third stage convolution operations are performed on the initial features. The feature extraction module outputs the final classification features.

[0059] The final classification features are input into the feature fully connected module, and the feature fully connected module outputs the classification results.

[0060] Furthermore, if Figure 3 As shown in the figure, the efficient input layer module sequentially performs 3×3 convolution and 1×1 convolution operations on the classification image to obtain the initial features; in this step, the initial number of channels of the classification image is 3 and the resolution is 96×96. Through the efficient input layer module, the initial RGB information of the classification image can be stimulated to a larger number of channels (reaching the number of channels: 16) and the spatial size can be reduced (reaching the resolution of 24×24) to reduce the computational cost in the subsequent feature extraction module, because the spatial feature map is too large, which will increase the total time cost required for feature extraction. Through the above operations, the initial features are obtained as follows: 24×24×16 (where 24×24 is the resolution and 16 is the number of channels).

[0061] In the feature extraction module, the first-stage convolution operation includes: performing 3×3 convolution, 3×3 convolution, and 1×1 convolution operations on the initial features in sequence to obtain the first-stage features; in this step, the first-stage features are: 12×12×32 (where 12×12 is the resolution and 32 is the number of channels);

[0062] The convolution operation of the second stage includes: performing 3×3 convolution, 1×1 convolution, 3×3 convolution and 1×1 convolution operations on the first stage features in turn to obtain the second stage preprocessing features, then performing residual connection between the first stage features and the second stage preprocessing features, and then performing 1×1 convolution operations; the convolution operation of the second stage is iterated twice to obtain the second stage features; in this step, the number of channels is first increased from 32 to 96, and then reduced to 64, so the second stage features are: 6×6×64 (where 6×6 is the resolution and 64 is the number of channels). In the two iterations, the input features are locally small feature acquisition through 3×3 convolution, mainly capturing local information such as detail edges and enriching features.

[0063] The convolution operation of the third stage includes: performing 5×5 convolution, 1×1 convolution, 5×5 convolution and 1×1 convolution operations on the second stage features in turn to obtain the third stage preprocessing features, then performing residual connection between the second stage features and the third stage preprocessing features, and then performing 1×1 convolution operations; the third stage convolution operation is iterated four times to obtain the final classification features; in this step, the number of channels is first increased from 64 to 192, and then reduced to 96, so the final classification features are: 3×3×96 (where 3×3 is the resolution and 96 is the number of channels). In the four iterations, the 5×5 convolution is used to capture a wider range of contextual information with a larger receptive field, mainly capturing the barcode contour and structural information. The 1×1 convolution in the iterative module is mainly used to control the number of channels, fuse high-dimensional features, enrich and refine features, and obtain the final classification features.

[0064] In multiple iterations, a single iteration is performed by formula (5):

[0065]

[0066] in, Convolution between two features, F o is the output feature, F m is the intermediate refinement feature, F i is the intermediate feature, n is the input feature, and m is the convolution result of the current layer.

[0067] If the number of intermediate refinement feature channels is the same as the number of input feature channels, then there is no need to perform convolution of the intermediate refinement features, and formula (6) is used:

[0068]

[0069] In the above steps, by strictly limiting the number of channels, the transmission efficiency between channels can be improved, and the performance can be improved while maintaining a relatively fast speed. The initial features are sent to the feature extraction module. Since increasing the number of channels will significantly increase the computational cost of the next layer of features, the high-dimensional features are used as the intermediate layer to ensure that the final output layer is compressed into a low channel, and the high-dimensional features are fused and sent to the next layer. After mapping the features to high dimensions through multiple iterations, they are fused into low-dimensional features to obtain the final classification features.

[0070] After obtaining the final classification features, the feature fully connected module performs 1×1 convolution and average pooling operations on the final classification features in turn to obtain the number of feature channels. The fully connected layer is used to map the number of feature channels to the barcode category. The distribution probability of each category is calculated through the Softmax activation function. The category with the largest distribution probability is selected as the damaged barcode category and the barcode category is output. In this step, the final classification features are first mapped to the specified dimension (the specified dimension is 1280) through 1×1 convolution and expanded. Then, the feature map is mapped to a spatial size of 1×1 through adaptive average pooling to obtain the mean of each channel feature. Finally, the fully connected layer is used to map the number of feature channels to the number of barcode categories. The distribution probability of each category is calculated through the Softmax activation function. The Softmax activation function is shown in formula (7):

[0071]

[0072] Among them, S(Z i ) is the distribution probability that the classified image belongs to the i-th category, z i is the original output of the i-th category, e Zi Given a K-dimensional input, K is the number of categories, To perform an exponential operation on each category and sum them, the sum result is used as the normalized denominator, where Z = [Z1, Z2, ..., ZK].

[0073] Through the above operations, the score ranking of different categories can be obtained and converted into probability distribution, so that the output value is between [0, 1], and the sum of the probabilities of all categories is 1, and the category with the highest score (that is, the highest probability) is taken as the output result. The corresponding category label (that is, the barcode category) is matched by the output result, and the category label is saved as the final output.

[0074] The damaged barcode classification device provided by the present invention is described below. The damaged barcode classification device described below and the damaged barcode classification method described above can be referred to each other.

[0075] like Figure 5 As shown, the present invention also provides a damaged barcode classification device, the classification device is used to perform the damaged barcode classification method above, the classification device comprises:

[0076] A barcode acquisition module, used to acquire barcode images;

[0077] A barcode position detection module is used to perform barcode position detection on a barcode image and obtain a barcode area image;

[0078] The barcode preprocessing module is used to preprocess the barcode area image to obtain a classified image;

[0079] The barcode classification module is used to input the classification image into the classification model for barcode classification and output the classification result.

[0080] Perform barcode position detection on the barcode image and obtain the barcode area image, including:

[0081] Binarize the barcode image to distinguish the barcode pattern from the background;

[0082] Extract the boundary features of the barcode pattern and obtain the barcode area image.

[0083] Among them, the binarization process includes:

[0084] Get the pixel mean in the neighborhood window of each pixel. The pixel mean is the adaptive threshold of each pixel. If the gray value of the pixel is less than or equal to the adaptive threshold, the pixel becomes white; if the gray value of the pixel is greater than the adaptive threshold, the pixel becomes black.

[0085] The shape of the barcode area image is a circumscribed rectangle of the barcode pattern.

[0086] The preprocessing steps include:

[0087] Get the width and height size information of the barcode area image, and fill the shorter side of the width and height until the width and height are consistent;

[0088] The resolution of the resized barcode region image is adjusted to 96×96 to obtain a classification image.

[0089] Input the classified image into the classification model for barcode classification and output the classification results, including:

[0090] The classified image is input into the efficient input layer module for convolution operation, and the efficient input layer module outputs the initial features;

[0091] The initial features are input into the feature extraction module, and the first, second and third stage convolution operations are performed on the initial features. The feature extraction module outputs the final classification features.

[0092] The final classification features are input into the feature fully connected module, and the feature fully connected module outputs the classification results.

[0093] According to a damaged barcode classification method provided by the present invention, the efficient input layer module sequentially performs 3×3 convolution and 1×1 convolution operations on the classification image to obtain initial features.

[0094] According to a damaged barcode classification method provided by the present invention, the convolution operation in the first stage includes: performing 3×3 convolution, 3×3 convolution and 1×1 convolution operations on the initial features in sequence to obtain the first stage features;

[0095] The convolution operation of the second stage includes: performing 3×3 convolution, 1×1 convolution, 3×3 convolution and 1×1 convolution operations on the first stage features in turn to obtain the second stage preprocessing features, then performing residual connection between the first stage features and the second stage preprocessing features, and then performing 1×1 convolution operation; the convolution operation of the second stage is iterated twice to obtain the second stage features;

[0096] The convolution operation of the third stage includes: performing 5×5 convolution, 1×1 convolution, 5×5 convolution and 1×1 convolution operations on the second stage features in turn to obtain the third stage preprocessing features, and then performing residual connection between the second stage features and the third stage preprocessing features, and then performing a 1×1 convolution operation; the convolution operation of the third stage is iterated four times to obtain the final classification features.

[0097] The feature fully connected module performs 1×1 convolution and average pooling operations on the final classification features in turn to obtain the number of feature channels, uses the fully connected layer to map the number of feature channels to the barcode category, calculates the distribution probability of each category through the Softmax activation function, selects the one with the largest distribution probability as the category of the damaged barcode, and outputs the barcode category.

[0098] The barcode image is collected, including: using a camera to collect a video stream on an object containing a barcode, and capturing a current frame as the barcode image.

[0099] The working process of the damaged barcode classification device of the present invention is consistent with the damaged barcode classification method described above, and will not be described in detail here.

[0100] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for classifying damaged barcodes, characterized in that: include: Capture barcode images; Perform barcode position detection on the barcode image to obtain a barcode area image; Preprocess the barcode area image to obtain a classified image; Input the classified image into the classification model for barcode classification and output the classification result.

2. The damaged barcode classification method according to claim 1, characterized in that: The barcode position detection is performed on the barcode image to obtain the barcode area image, including: Binarize the barcode image to distinguish the barcode pattern from the background; Extract the boundary features of the barcode pattern and obtain the barcode area image.

3. The damaged barcode classification method according to claim 2, characterized in that: The binarization process includes: Get the pixel mean in the neighborhood window of each pixel. The pixel mean is the adaptive threshold of each pixel. If the gray value of the pixel is less than or equal to the adaptive threshold, the pixel becomes white; if the gray value of the pixel is greater than the adaptive threshold, the pixel becomes black.

4. The damaged barcode classification method according to claim 2, characterized in that: The shape of the barcode area image is the circumscribed rectangle of the barcode pattern.

5. The damaged barcode classification method according to claim 4, characterized in that: The preprocessing steps include: Get the width and height size information of the barcode area image, and fill the shorter side of the width and height until the width and height are consistent; The resolution of the resized barcode region image is adjusted to 96×96 to obtain a classification image.

6. The damaged barcode classification method according to claim 1, characterized in that: The step of inputting the classified image into the classification model to classify the barcode and outputting the classification result includes: The classified image is input into the efficient input layer module for convolution operation, and the efficient input layer module outputs the initial features; The initial features are input into the feature extraction module, and the first, second and third stage convolution operations are performed on the initial features. The feature extraction module outputs the final classification features. The final classification features are input into the feature fully connected module, and the feature fully connected module outputs the classification results.

7. The damaged barcode classification method according to claim 6, characterized in that: The efficient input layer module performs 3×3 convolution and 1×1 convolution operations on the classified image in sequence to obtain the initial features.

8. The damaged barcode classification method according to claim 7, characterized in that: The convolution operation in the first stage includes: performing 3×3 convolution, 3×3 convolution and 1×1 convolution operations on the initial features in sequence to obtain the first stage features; The convolution operation of the second stage includes: performing 3×3 convolution, 1×1 convolution, 3×3 convolution and 1×1 convolution operations on the first stage features in turn to obtain the second stage preprocessing features, then performing residual connection between the first stage features and the second stage preprocessing features, and then performing 1×1 convolution operation; the convolution operation of the second stage is iterated twice to obtain the second stage features; The convolution operation of the third stage includes: performing 5×5 convolution, 1×1 convolution, 5×5 convolution and 1×1 convolution operations on the second stage features in turn to obtain the third stage preprocessing features, and then performing residual connection between the second stage features and the third stage preprocessing features, and then performing a 1×1 convolution operation; the convolution operation of the third stage is iterated four times to obtain the final classification features.

9. The damaged barcode classification method according to claim 8, characterized in that: The feature fully connected module performs 1×1 convolution and average pooling operations on the final classification features in turn to obtain the number of feature channels, uses the fully connected layer to map the number of feature channels to the barcode category, calculates the distribution probability of each category through the Softmax activation function, selects the one with the largest distribution probability as the category of the damaged barcode, and outputs the barcode category.

10. The damaged barcode classification method according to claim 1, characterized in that: The collecting of barcode images comprises: Use the camera to capture the video stream on the object containing the barcode and capture the current frame as the barcode image.

Images