Bar code identification method and neural network model training method

Abstract

The invention provides a bar code identification method. The method comprises the following steps: acquiring a bar code image to be identified; performing feature extraction on the bar code image by using a convolutional neural network to obtain a multi-channel feature image; flattening feature data of each channel of the feature image to obtain feature vectors of multiple channels, and adding position codes to the feature vectors of the multiple channels; coding the feature vectors of the multiple channels added with the position codes by adopting a coding layer based on an attention mechanism to obtain coded data; decoding the coded data by adopting a decoding layer based on an attention mechanism to obtain decoded data; and processing the decoded data by adopting an output layer to obtain a character string corresponding to the bar code image. According to the bar code identification method and the neural network model training method provided by the invention, the overall information of the whole bar code can be effectively utilized to eliminate local fuzziness or noise, and the decoding capability is improved.

Description

Technical Field

[0001] This invention relates to the field of barcode reading technology, and in particular to a barcode recognition method and a neural network model training method. Background Technology

[0002] Barcodes, as a widely used machine-readable optical information carrier, encode numerical, alphanumeric, or symbolic information through their bar-space structure. They require specialized equipment for identification to enable rapid data entry (such as warehousing, retrieval, and querying). However, existing barcode reading and decoding technologies all require clear barcodes with distinct bars and spaces to differentiate between black and white bars, determine their widths, and then match and decode the binary information of the black and white bar combinations according to known encoding rules. Because barcode images have a wide range of applications, including but not limited to labels, printing, and laser engraving, and their backgrounds are often complex, they are frequently subject to scratches, distortions, dirt, damage, or excessively small barcodes. Under these complex conditions, existing reading and decoding methods struggle to extract barcode information, resulting in low decoding performance. Summary of the Invention

[0003] The barcode recognition method and neural network model training method provided by this invention can effectively utilize the overall information of the entire barcode to eliminate local ambiguity or noise and improve decoding capabilities.

[0004] In a first aspect, the present invention provides a barcode recognition method, the method comprising: Obtain the image of the barcode to be recognized; A convolutional neural network is used to extract features from the barcode image to obtain a multi-channel feature image; The feature data of each channel of the feature image is flattened to obtain a multi-channel feature vector, and position encoding is added to the multi-channel feature vector; An attention-based encoding layer is used to encode the multi-channel feature vector with added positional encoding to obtain encoded data; The encoded data is decoded using an attention-based decoding layer to obtain decoded data; The decoded data is processed using an output layer based on the attention mechanism to obtain a string corresponding to the barcode image.

[0005] Optionally, the step of using a convolutional neural network to extract features from the barcode image to obtain a multi-channel feature image includes: The multiple channels of the barcode image are weighted and summed to obtain a grayscale image; The grayscale image is convolved using a multi-channel convolution kernel to obtain a multi-channel feature image.

[0006] Optionally, the step of flattening the feature data of each channel of the feature image to obtain a multi-channel feature vector, and adding position encoding to the multi-channel feature vector, includes: Flatten the feature data of each channel into a one-dimensional vector by row or column; Construct a multi-channel feature vector from the one-dimensional vectors corresponding to the feature data of multiple channels; The positional encoding of each feature data is determined based on the row or column index of each feature data in each one-dimensional vector before flattening; The positional encoding of each feature data is added to the feature vector of the multi-channel array.

[0007] Optionally, the encoding layer based on an attention mechanism encodes the multi-channel feature vector with added positional encoding to obtain encoded data including: Two or more attention-based encoding layers are used to encode the multi-channel feature vector with added positional encoding in a serial manner to obtain encoded data.

[0008] Optionally, each attention-based encoding layer encodes the multi-channel feature vector with added positional encoding, resulting in encoded data including: The feature vector of a multi-channel system is split into multiple vectors to be calculated based on the channels; each vector to be calculated includes more than one channel. Each vector to be computed is processed using an attention mechanism to obtain multiple attention vectors; Multiple attention vectors are processed using a feedforward network to obtain the encoded data output by the current coding layer.

[0009] Optionally, the step of using an attention-based decoding layer to decode the encoded data to obtain decoded data includes: The encoded data is decoded serially using two or more attention-based decoding layers to obtain decoded data.

[0010] Optionally, the encoded data is decoded using each attention-based decoding layer to obtain decoded data including: The positional encoding and the previous output data obtained from the previous decoding are subjected to masked multi-head attention processing to obtain the input embedding data; The input embedded data and the encoded data are subjected to cross-multi-head attention processing to determine the data to be processed in the current decoding. The data to be processed obtained from the current decoding is processed using a feedforward network to determine the current output data obtained from the current decoding. If the current output data contains the end encoding, then the current output data is determined to be the current decoded data output by the current decoding layer; If the current output data does not contain the end code, return to the step of performing masked multi-head attention processing on the position code and the previous output data obtained from the previous decoding.

[0011] Secondly, the present invention also provides a neural network model training method for training a neural network model to perform the barcode recognition method as described in any of the preceding claims, wherein the neural network model training method includes: Based on the barcode length range, the barcode length is randomly determined, and characters are randomly selected from the barcode character set to generate a standard barcode image; Based on the usage scenario of the barcode, the corresponding image enhancement combination is selected, and the standard barcode image is subjected to image enhancement processing to obtain the barcode training sample to be parsed; The neural network model is trained using the training samples of the barcode to be parsed as input and the string corresponding to the standard barcode image as the ground truth.

[0012] Optionally, selecting the corresponding image enhancement combination based on the barcode's usage scenario includes: An image enhancement combination is formed by using two or more of the following image enhancement methods: Gaussian blur, motion blur, adding noise, contrast adjustment, brightness adjustment, image shift, image jitter, image distortion, random spots, and random occlusion.

[0013] Optionally, before randomly determining the barcode length based on the barcode length range and randomly selecting characters from the barcode character set to generate a standard barcode image, the method further includes: Based on the training requirements of the neural network model, select or create a character set corresponding to one encoding system or a character set corresponding to two or more encoding systems.

[0014] The technical solution provided by this invention extracts barcode images using a convolutional neural network, employs an attention-based encoding layer for encoding, and an attention-based decoding layer for decoding. The attention mechanism associates the encoded information at each position in the barcode image. Because the attention mechanism can better capture the dependencies between positions with long distances in the barcode, it can effectively improve the correlation of bar-space combination rules and the accuracy of barcode reading. Furthermore, since the attention mechanism has stronger robustness to interference such as blurring, tilting, and occlusion, the technical solution of this invention performs excellently in barcode recognition against complex backgrounds. Attached Figure Description

[0015] Figure 1 This is a flowchart of a barcode recognition method according to an embodiment of the present invention; Figure 2 This is a flowchart illustrating data processing in the encoding layer of a barcode recognition method according to another embodiment of the present invention. Figure 3 This is a flowchart illustrating data processing in the decoding layer of a barcode recognition method according to another embodiment of the present invention. Figure 4 This is a flowchart of a neural network model training method according to another embodiment of the present invention. Detailed Implementation

[0016] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0017] This invention provides a barcode recognition method, such as... Figure 1 As shown, the method includes: Obtain the image of the barcode to be recognized; In some embodiments, for barcode images that are damaged or blurred due to large field of view, severe distortion, dirt, or other reasons in practical applications, it is usually difficult to recognize them using conventional barcode scanners. Therefore, feature extraction, encoding, and decoding processes are required in subsequent steps to enable the recognition of the barcode string.

[0018] A convolutional neural network is used to extract features from the barcode image to obtain a multi-channel feature image; In some embodiments, a convolutional neural network is used to extract feature images from the barcode image. For example, given an input image of (H×W×3), convolutional downsampling is used to obtain a feature map with a size of (H / n)×(W / n)×L. Here, n is the convolutional downsampling ratio, and L is the number of channels in the feature map obtained through convolutional downsampling. Since the information conveyed by a barcode image will be lost when it is damaged or blurred, it is necessary to use a convolutional neural network to extract features from the barcode image so that the corresponding string can be accurately extracted during subsequent encoding and decoding using an attention-based encoding and decoding layer.

[0019] The feature data of each channel of the feature image is flattened to obtain a multi-channel feature vector, and position encoding is added to the multi-channel feature vector; In some embodiments, since the attention-based encoding and decoding layers typically compute sequential data, this step requires flattening the feature data for each channel. During flattening, the positional information of the feature data in each channel is lost. Therefore, positional encoding needs to be added to the multi-channel feature vectors to ensure that the positional information is not lost. For example, flattening the data for each channel of a (H / n)×(W / n)×L feature image yields L one-dimensional vectors with length (H / n)×(W / n).

[0020] An attention-based encoding layer is used to encode the multi-channel feature vector with added positional encoding to obtain encoded data; In some embodiments, the encoding layer encodes multi-channel feature vectors using a multi-head attention mechanism. This means that the attention mechanism is applied to encode multi-channel feature vectors in multiple subspaces, with different information being focused on during encoding within each subspace. This allows for encoding of multi-channel feature vectors from different perspectives. For example, a one-dimensional vector with L channels, each with a length of (H / n) × (W / n), can be processed by the encoding layer to obtain another one-dimensional vector with L channels, each with a length of (H / (n*m)) × (W / (n*m)), where m is the scaling factor applied to the vector length during encoding.

[0021] The encoded data is decoded using an attention-based decoding layer to obtain decoded data; In some embodiments, an attention-based decoding layer is used to decode the encoded data to obtain multi-channel sequence data. The multi-channel sequence data represents the string corresponding to the barcode in a high-dimensional way. Then, the output layer is processed to output the string corresponding to the barcode.

[0022] The decoded data is processed using an output layer based on the attention mechanism to obtain a string corresponding to the barcode image.

[0023] In some embodiments, the output layer processes the decoded data, for example, by using linear transformation and Softmax probability calculation to output the decoded data as a string.

[0024] In the technical solution provided by this invention, a convolutional neural network is used to extract barcode images, an attention-based encoding layer is used for encoding, and an attention-based decoding layer is used for decoding. The attention mechanism associates the encoded information at each position in the barcode image. Because the attention mechanism can better capture the dependencies between positions with long distances in the barcode, it can effectively improve the correlation of bar-space combination rules and the accuracy of barcode reading. Furthermore, since the attention mechanism has stronger robustness to interference such as blurring, tilting, and occlusion, the technical solution of this invention performs excellently in barcode recognition against complex backgrounds.

[0025] As an optional implementation, the step of using a convolutional neural network to extract features from the barcode image to obtain a multi-channel feature image includes: The multiple channels of the barcode image are weighted and summed to obtain a grayscale image; The grayscale image is convolved using a multi-channel convolution kernel to obtain a multi-channel feature image.

[0026] In some embodiments, since barcodes typically use black and white stripes as the information carrier, a weighted summation method can be used to obtain a grayscale image in order to facilitate control of the output dimension of the convolutional neural network. Then, the grayscale image is convolved to obtain the desired multi-channel feature image.

[0027] As an optional implementation, the step of flattening the feature data of each channel of the feature image to obtain a multi-channel feature vector, and adding positional encoding to the multi-channel feature vector, includes: Flatten the feature data of each channel into a one-dimensional vector by row or column; Construct a multi-channel feature vector from the one-dimensional vectors corresponding to the feature data of multiple channels; The positional encoding of each feature data is determined based on the row or column index of each feature data in each one-dimensional vector before flattening; The positional encoding of each feature data is added to the feature vector of the multi-channel array.

[0028] In some embodiments, since the attention-based encoding and decoding layers typically compute on sequence data, this step requires flattening the feature data for each channel. During flattening, the positional information of the feature data in each channel is lost; therefore, positional encoding needs to be added to the feature vectors of multiple channels to ensure that positional information is not lost. Flattening of feature data for a single channel can be done row-wise or column-wise. Since the position of each feature data can be determined using row and column indices before flattening, the positional encoding can be determined based on either the row or column index when adding it.

[0029] As an optional implementation, the encoding layer based on an attention mechanism encodes the multi-channel feature vector with added positional encoding, resulting in encoded data including: Two or more attention-based encoding layers are used to encode the multi-channel feature vector with added positional encoding in a serial manner to obtain encoded data.

[0030] In some embodiments, multiple coding layers are used for serial processing, which can extract features of multi-channel vectors at a deeper level in succession, and is more conducive to the accurate recognition of barcodes.

[0031] As an optional implementation method, such as Figure 2 As shown, each attention-based encoding layer encodes the multi-channel feature vector with added positional encoding, resulting in encoded data including: The feature vector of a multi-channel system is split into multiple vectors to be calculated based on the channels; each vector to be calculated includes more than one channel. Each vector to be computed is processed using an attention mechanism to obtain multiple attention vectors; Multiple attention vectors are processed using a feedforward network to obtain the encoded data output by the current coding layer.

[0032] In some embodiments, since each channel's feature vector typically represents different features in a multi-channel feature vector, the multi-channel feature vector can be split into multiple vectors to be computed. Each vector contains different features, and parallel processing of each vector using an attention mechanism can effectively accelerate recognition efficiency. The feedforward network can perform a linear transformation on multiple attention vectors, combining them into new multi-channel vectors or sequence data, i.e., forming encoded data.

[0033] As an optional implementation, the step of using an attention-based decoding layer to decode the encoded data to obtain decoded data includes: The encoded data is decoded serially using two or more attention-based decoding layers to obtain decoded data.

[0034] In some embodiments, multiple decoding layers are used for serial processing, which can extract the features of multi-channel vectors at a deeper level in succession, and is more conducive to the accurate recognition of barcodes.

[0035] As an optional implementation method, such as Figure 3 As shown, the encoded data is decoded using each attention-based decoding layer to obtain decoded data including: The positional encoding and the previous output data obtained from the previous decoding are subjected to masked multi-head attention processing to obtain the input embedding data; The input embedded data and the encoded data are subjected to cross-multi-head attention processing to determine the data to be processed in the current decoding. The data to be processed obtained from the current decoding is processed using a feedforward network to determine the current output data obtained from the current decoding. If the current output data contains the end encoding, then the current output data is determined to be the current decoded data output by the current decoding layer; If the current output data does not contain the end code, return to the step of performing masked multi-head attention processing on the position code and the previous output data obtained from the previous decoding.

[0036] In some embodiments, a masked multi-head attention approach is used to mask unidentified data to prevent unidentified data from affecting the currently identified data. At the same time, a cross-multi-head attention approach is used to process the input embedded data and encoded data, so that the currently identified data focuses on the already identified data, which is beneficial to the accurate identification of the data.

[0037] This invention also provides a neural network model training method for training a neural network model to perform the barcode recognition method as described in any of the preceding claims, such as... Figure 4 As shown, the neural network model training method includes: Based on the barcode length range, the barcode length is randomly determined, and characters are randomly selected from the barcode character set to generate a standard barcode image; Based on the usage scenario of the barcode, the corresponding image enhancement combination is selected, and the standard barcode image is subjected to image enhancement processing to obtain the barcode training sample to be parsed; The neural network model is trained using the training samples of the barcode to be parsed as input and the string corresponding to the standard barcode image as the ground truth.

[0038] In some embodiments, standard barcodes are generated randomly, and then image processing is performed on the standard barcodes according to the usage scenario to obtain training samples. Thus, without the need for annotation, the string of the standard barcode can be directly used as the ground truth, which can effectively improve the efficiency of obtaining training samples.

[0039] As an optional implementation, selecting the corresponding image enhancement combination based on the barcode's usage scenario includes: An image enhancement combination is formed by using two or more of the following image enhancement methods: Gaussian blur, motion blur, adding noise, contrast adjustment, brightness adjustment, image shift, image jitter, image distortion, random spots, and random occlusion.

[0040] In some embodiments, when forming an image enhancement combination, 2-5 transformations can be randomly selected from the enhancement methods to simulate actual usage scenarios. These enhancement methods include Gaussian blur, motion blur, noise addition, contrast adjustment, brightness adjustment, image shift, image jitter, image distortion, random speckles, random occlusion, and other image enhancement techniques. By sequentially applying these randomly selected 2-5 transformations to the image, a barcode image with a complex background and wide coverage can be obtained. This barcode image is difficult to read using conventional methods.

[0041] As an optional implementation, before randomly determining the barcode length based on the barcode length range and randomly selecting characters from the barcode character set to generate a standard barcode image, the method further includes: Based on the training requirements of the neural network model, select or create a character set corresponding to one encoding system or a character set corresponding to two or more encoding systems.

[0042] Those skilled in the art will understand that all or part of the processes in the above method embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0043] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A barcode recognition method, characterized in that, The method includes: Obtain the image of the barcode to be recognized; A convolutional neural network is used to extract features from the barcode image to obtain a multi-channel feature image; The feature data of each channel of the feature image is flattened to obtain a multi-channel feature vector, and position encoding is added to the multi-channel feature vector; An attention-based encoding layer is used to encode the multi-channel feature vector with added positional encoding to obtain encoded data; The encoded data is decoded using a decoding layer based on the attention mechanism to obtain decoded data; The decoded data is processed using an output layer based on the attention mechanism to obtain a string corresponding to the barcode image.

2. The method according to claim 1, characterized in that, The step of using a convolutional neural network to extract features from the barcode image to obtain a multi-channel feature image includes: The multiple channels of the barcode image are weighted and summed to obtain a grayscale image; The grayscale image is convolved using a multi-channel convolution kernel to obtain the multi-channel feature image.

3. The method according to claim 1, characterized in that, The step of flattening the feature data of each channel of the feature image to obtain a multi-channel feature vector, and adding position encoding to the multi-channel feature vector, includes: Flatten the feature data of each channel into a one-dimensional vector by row or column; The one-dimensional vectors corresponding to the feature data of multiple channels are used to construct the feature vector of the multiple channels; The positional encoding of each feature data is determined based on the row or column index of each feature data in each one-dimensional vector before flattening; The positional encoding of each feature data is added to the feature vector of the multi-channel.

4. The method according to claim 1, characterized in that, The attention-based encoding layer encodes the multi-channel feature vector with added positional encoding to obtain encoded data including: The encoded data is obtained by serially encoding the multi-channel feature vector with the added positional encoding using two or more encoding layers based on the attention mechanism.

5. The method according to claim 4, characterized in that, Each encoding layer based on the attention mechanism encodes the multi-channel feature vector with added positional encoding, resulting in the encoded data including: The multi-channel feature vector is split into multiple vectors to be calculated based on the channels; each vector to be calculated includes more than one channel. Each vector to be computed is processed using the attention mechanism to obtain multiple attention vectors; Multiple attention vectors are processed using a feedforward network to obtain the encoded data output by the current coding layer.

6. The method according to claim 1, characterized in that, The step of using a decoding layer based on the attention mechanism to decode the encoded data to obtain decoded data includes: The encoded data is decoded serially using two or more decoding layers based on the attention mechanism to obtain the decoded data.

7. The method according to claim 6, characterized in that, The encoded data is decoded using each decoding layer based on the attention mechanism to obtain the decoded data, which includes: The positional encoding and the previous output data obtained from the previous decoding are subjected to masked multi-head attention processing to obtain the input embedding data; The input embedded data and the encoded data are subjected to cross-multi-head attention processing to determine the data to be processed obtained in the current decoding. The data to be processed obtained from the current decoding is processed using a feedforward network to determine the current output data obtained from the current decoding. When the current output data contains the end encoding, the current output data is determined to be the current decoded data output by the current decoding layer; If the current output data does not contain the end code, return to the step of performing masked multi-head attention processing on the position code and the previous output data obtained from the previous decoding.

8. A method for training a neural network model, characterized in that, The method for training a neural network model to perform the barcode recognition method as described in any one of claims 1-7 includes: Based on the barcode length range, the barcode length is randomly determined, and characters are randomly selected from the barcode character set to generate a standard barcode image; Based on the usage scenario of the barcode, the corresponding image enhancement combination is selected, and the standard barcode image is subjected to image enhancement processing to obtain the barcode training sample to be parsed; The neural network model is trained using the training sample of the barcode to be parsed as input and the string corresponding to the standard barcode image as the ground truth.

9. The method according to claim 8, characterized in that, The selection of corresponding image enhancement combinations based on the barcode's usage scenario includes: The image enhancement combination is formed by using two or more of the following image enhancement methods: Gaussian blur, motion blur, adding noise, contrast adjustment, brightness adjustment, image shift, image jitter, image distortion, random spots, and random occlusion.

10. The method according to claim 8, characterized in that, Before randomly determining the barcode length based on the barcode length range and randomly selecting characters from the barcode character set to generate a standard barcode image, the method further includes: Based on the training requirements of the neural network model, select or create a character set corresponding to one code system or a character set corresponding to two or more code systems.

Images