Bar code decoding method and device based on deep learning, equipment and storage medium

Abstract

The application discloses a bar code decoding method and device based on deep learning, computer equipment and a storage medium. The method constructs a training set and a test set through a sample image and label information; a deep learning model is trained according to the training set, and a bar code prediction model is obtained; the bar code prediction model is tested according to the test set and a preset encoding rule, and if the preset test condition is met, the training is stopped, and a target bar code prediction model is obtained; a received bar code image to be decoded is input into the target bar code prediction model, a first prediction box and a plurality of second prediction boxes are obtained; corresponding decoding characters and a total number of decoding characters are obtained according to the plurality of second prediction boxes and the preset encoding rule, and a total number of characters is obtained according to the first prediction box and the preset encoding rule; and if the total number of decoding characters is equal to the total number of characters, the decoding characters are output. The application combines deep learning and a decoding method, realizes accurate positioning of a bar code, and improves decoding efficiency.

Description

Technical Field

[0001] This invention relates to the field of image recognition technology in artificial intelligence, and in particular to a barcode decoding method, apparatus, computer equipment, and storage medium based on deep learning. Background Technology

[0002] As my country enters the era of intelligent manufacturing and the Internet of Things, the popularization of barcode labels will accelerate, gradually entering the daily lives of ordinary people with more innovative applications. The demand for barcode labels will surge, and label production will become increasingly common. The development of barcode printing equipment towards miniaturization, wireless operation, integrated hardware and software solutions, and cost-effectiveness is an irreversible trend to meet the technical requirements of specialized performance and small-batch on-site printing. In industrial applications, workers typically align barcode scanners with the barcodes, and the scanners read the data. However, when scanning a large number of barcodes in a factory environment, the scanners may experience inaccurate barcode positioning during the reading process, leading to decoding errors or failures.

[0003] In existing technology, a laser diode emits a light beam, which is reflected by the bars or spaces of the barcode and then returns to the reader. The light is collected and focused by a mirror, converted into an electrical signal by a photoelectric converter, and then decoded. However, it is difficult to guarantee that every space in the barcode will reflect light, and in complex factory environments, the reader may not receive light effectively, leading to decoding errors or failures. Summary of the Invention

[0004] This invention provides a barcode decoding method, apparatus, computer device, and storage medium based on deep learning, aiming to solve the problem of inaccurate barcode location and decoding in complex usage scenarios.

[0005] In a first aspect, embodiments of the present invention provide a barcode decoding method based on deep learning, comprising:

[0006] Obtain a set of barcode images, and perform image preprocessing on each barcode image in the set to obtain sample images corresponding to each barcode image, thus forming a sample image set;

[0007] The location information of the barcode in each sample image in the sample image set is labeled to obtain the first true location information of each sample image; and the location information and bar / space type of each bar and space in each sample image are labeled to obtain multiple second true location information of each sample image. The first true location information and multiple second true location information of each sample image are combined to form a labeled information set.

[0008] A barcode prediction set is formed using a set of sample images and a set of labeled information, and the barcode prediction set is split into a training set and a test set;

[0009] A barcode prediction model is obtained by training a pre-built deep learning model using the training set.

[0010] The barcode prediction model is tested according to the test set and preset encoding rules. If the preset test conditions are met, training is stopped and the target barcode prediction model is obtained.

[0011] The system receives a barcode image to be decoded, inputs the barcode image to be decoded into a target barcode prediction model, and obtains a first prediction box and multiple second prediction boxes. The first prediction box is used to indicate the predicted position of the barcode in the barcode image to be decoded, and the second prediction boxes are used to indicate the predicted position of each bar or space in the barcode image to be decoded.

[0012] Based on the position information and bar / space type of multiple second prediction boxes, and the preset encoding rules, the decoded characters and the total number of decoded characters of the barcode image to be decoded are obtained, and the total number of characters of the barcode image to be decoded is obtained based on the position information of the first prediction box and the preset encoding rules.

[0013] If the total number of decoded characters in the barcode image to be decoded is equal to the total number of characters in the barcode image to be decoded, then the decoded characters of the barcode image to be decoded are output.

[0014] Secondly, embodiments of the present invention provide a barcode decoding device based on deep learning, comprising:

[0015] The sample image acquisition unit is used to acquire a set of barcode images and perform image preprocessing on each barcode image in the set to obtain a sample image corresponding to each barcode image, so as to form a sample image set.

[0016] The information annotation unit is used to annotate the position information of the barcode of each sample image in the sample image set to obtain the first true position information of each sample image; and to annotate the position information and bar / space type of each bar and space of each sample image to obtain multiple second true position information of each sample image. The first true position information and multiple second true position information of each sample image are combined to form an annotation information set.

[0017] The prediction set splitting unit is used to form a barcode prediction set using the sample image set and the annotation information set, and to split the barcode prediction set into a training set and a test set.

[0018] The model training unit is used to train a pre-built deep learning model based on the training set to obtain a barcode prediction model.

[0019] The model testing unit is used to test the barcode prediction model according to the test set and preset encoding rules. If the preset test conditions are met, training is stopped and the target barcode prediction model is obtained.

[0020] A barcode receiving unit is used to receive a barcode image to be decoded, input the barcode image to be decoded into a target barcode prediction model, and obtain a first prediction box and multiple second prediction boxes; wherein, the first prediction box is used to indicate the predicted position of the barcode in the barcode image to be decoded, and the second prediction box is used to indicate the predicted position of each bar or space in the barcode image to be decoded.

[0021] The barcode decoding unit is used to obtain the decoded characters and the total number of decoded characters of the barcode image to be decoded based on the position information and bar / space type of multiple second prediction boxes and preset encoding rules, and to obtain the total number of characters of the barcode image to be decoded based on the position information of the first prediction box and preset encoding rules.

[0022] The decoding character output unit is used to output the decoded characters of the barcode image to be decoded if the total number of decoded characters of the barcode image to be decoded is equal to the total number of characters of the barcode image to be decoded.

[0023] Thirdly, embodiments of the present invention also provide a computer device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the deep learning-based barcode decoding method of the first aspect described above.

[0024] Fourthly, embodiments of the present invention also provide a computer-readable storage medium storing a computer program that, when executed by a processor, causes the processor to perform the deep learning-based barcode decoding method described in the first aspect.

[0025] This invention provides a deep learning-based barcode decoding method, apparatus, computer device, and storage medium. The method constructs a training set and a test set using sample images and annotation information. A pre-built deep learning model is trained using the training set to obtain a barcode prediction model. The barcode prediction model is tested using the test set and preset encoding rules. If preset test conditions are met, training stops, resulting in a target barcode prediction model. A received barcode image to be decoded is input into the target barcode prediction model to obtain a first prediction box and multiple second prediction boxes. The corresponding decoded characters and the total number of decoded characters are obtained based on the multiple second prediction boxes and preset encoding rules, and the total number of characters is obtained based on the first prediction boxes and preset encoding rules. If the total number of decoded characters equals the total number of characters, the decoded characters are output. This invention, by combining deep learning and decoding methods, achieves accurate barcode positioning and improves decoding efficiency. Attached Figure Description

[0026] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0027] Figure 1 A flowchart illustrating the deep learning-based barcode decoding method provided in an embodiment of the present invention;

[0028] Figure 2 A schematic diagram of a sub-process of a deep learning-based barcode decoding method provided in an embodiment of the present invention;

[0029] Figure 3 The structure of the EAN code components with characters 9 and 2 provided in the embodiments of the present invention;

[0030] Figure 4 A schematic block diagram of a deep learning-based barcode decoding device provided in an embodiment of the present invention;

[0031] Figure 5 A schematic block diagram of a computer device provided for an embodiment of the present invention. Detailed Implementation

[0032] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0033] It should be understood that, when used in this specification and the appended claims, the terms "comprising" and "including" indicate the presence of the described features, integrals, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0034] It should also be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the invention. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0035] It should also be further understood that the term "and / or" as used in this specification and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0036] Please see Figure 1 , Figure 1 This is a flowchart illustrating the deep learning-based barcode decoding method provided in an embodiment of the present invention. The deep learning-based barcode decoding method provided in this embodiment is applied to a server and is executed by application software installed on the server. The deep learning-based barcode decoding method provided in this embodiment includes steps S11 to S18.

[0037] S11. Obtain a set of barcode images and perform image preprocessing on each barcode image in the set to obtain sample images corresponding to each barcode image, thus forming a sample image set.

[0038] In this embodiment, the acquired barcode image set includes a large number of images containing barcodes. A barcode is a graphic identifier that represents a set of information by arranging multiple black bars and spaces of varying widths according to a specific encoding rule. Common barcodes consist of parallel lines arranged with black bars (simply called bars) and white bars (simply called spaces) of significantly different reflectivities. Since the barcode images in the barcode image set contain much other information unrelated to the barcode itself, image processing is required to eliminate interference and reduce the impact on subsequent barcode feature extraction. Image preprocessing is performed on the barcode images in the barcode image set to obtain sample images for model training and testing. These processed sample images are then combined to form a sample image set.

[0039] In one embodiment, step S11 includes:

[0040] Multiple barcode images acquired by an image acquisition device are combined to form a barcode image set;

[0041] Each barcode image is subjected to noise reduction filtering to obtain each first intermediate image;

[0042] Each first intermediate image is converted to grayscale to obtain each second intermediate image;

[0043] Each second intermediate image is binarized to obtain each third intermediate image; each third intermediate image includes a barcode area and a background area.

[0044] The barcode area of ​​each third intermediate image is cropped to a preset size to obtain each fourth intermediate image;

[0045] Each fourth intermediate image is angled to obtain a sample image corresponding to each barcode image;

[0046] Each sample image is combined into a sample image set.

[0047] In this embodiment, barcodes on items in complex usage scenarios are captured by an image acquisition device, such as a CCD (Charge Coupled Device) camera, to obtain a large number of images containing barcodes. These numerous barcode images are then combined into a barcode image set. Each barcode image undergoes sequential noise reduction filtering, grayscale conversion, binarization, cropping, and angle adjustment to obtain a sample image corresponding to each barcode image. Specifically, the image signal of the barcode image is first converted into a digital signal, and a frequency domain filtering algorithm is used to perform noise reduction filtering on the barcode image. Then, the first intermediate image is grayscale converted to improve image processing efficiency; an averaging method can be used to process the image at grayscale levels, reducing the resource consumption of color barcodes. Next, redundant information such as color is removed from the second intermediate image to improve computational speed. This requires binarization. The target area (barcode area) and background area in the resulting third intermediate image are then distinguished. During binarization, a segmentation threshold is obtained using color or grayscale features based on deep learning. The image is compared to the threshold; areas above the threshold are classified as black, and areas below are classified as white, thus dividing the image into white and black parts. The barcode area in the third intermediate image is then cropped to a preset size. Finally, the angle of the fourth intermediate image is adjusted so that the barcode is vertical, resulting in sample images. These sample images form a sample image set.

[0048] S12. Label the position information of the barcode in each sample image in the sample image set to obtain the first true position information of each sample image; and label the position information and bar / space type of each bar and space in each sample image to obtain multiple second true position information of each sample image. Combine the first true position information and multiple second true position information of each sample image to form a label information set.

[0049] In this embodiment, the position information of the barcode in each sample image in the sample image set is labeled to obtain the first true position information of each sample image. The position information of the barcode can be represented by the position information of the rectangle that encloses the barcode. The first true position information is the coordinates of the four endpoints of the rectangle that encloses the barcode, and the width information of the barcode is calculated based on the first true position information. At the same time, the position information and bar / space type of each bar and space in each sample image are labeled. Since the barcode includes multiple bars and spaces, multiple second true position information of each sample image are obtained. The bar / space type includes bars and spaces, which are used to represent the bars and spaces in the barcode. The position information of the bars or spaces in the barcode can be represented by the position information of the rectangle that encloses the bars or spaces in the barcode. The second true position information is the coordinates of the four endpoints of the rectangle that encloses each bar or space in the barcode, and the width information of each bar or space can be calculated based on the position information of each bar or space. Then, the width information of each bar or space can be converted into the corresponding characters according to the preset encoding rules. The first true location information and multiple second true location information for each sample image are stored in a single annotation information file. Each sample image corresponds to one annotation information file, and all annotation information files are used to form the annotation information set for the sample image set. The barcodes and their bars and spaces in the sample images are annotated, and the annotation information is used to train the model so that the trained model can locate the barcodes and their bars and spaces in the images.

[0050] S13. Use the sample image set and the annotation information set to form a barcode prediction set, and split the barcode prediction set into a training set and a test set.

[0051] In this embodiment, a barcode prediction set is composed of a sample image set and a labeled information set, and this barcode prediction set is split into a training set and a test set. The sample images in the training set and the test set correspond one-to-one with the labeled information files. The training set is used to train the model's parameters, and the test set is used to evaluate the model's generalization ability. Specifically, the barcode prediction set can be randomly split into training and test sets, or it can be split proportionally.

[0052] S14. Train the pre-built deep learning model based on the training set to obtain the barcode prediction model.

[0053] In this embodiment, a pre-built deep learning model is trained using a training set to obtain model parameters, thereby obtaining a barcode prediction model. This barcode prediction model can predict the barcode in an image and the positional information of its bars and spaces. Furthermore, the pre-built deep learning model can be a machine learning model based on a deep convolutional neural network.

[0054] In one embodiment, the deep learning model includes a feature extraction module, a generative network, a first regression network, and a second regression network. Step S14 includes:

[0055] Each sample image in the training set is input into the feature extraction module to obtain the feature map corresponding to each sample image;

[0056] By generating a network, a first initial detection box is determined in each feature map according to each first real location information in the training set, and multiple second initial detection boxes are determined in each feature map according to each second real location information in the training set.

[0057] The location information of each initial detection box is obtained through the first regression network, the first error between the location information of each initial detection box and the corresponding first true location information is calculated, and the network parameters of the first regression network are adjusted according to the first error.

[0058] The location information of each second initial detection box is obtained through the second regression network. The second error between the location information of each second initial detection box and the corresponding second true location information is calculated. The network parameters of the second regression network are adjusted according to the second error.

[0059] If both the first error and the second error are less than the preset value, the network parameters of the first regression network and the network parameters of the second regression network will not be adjusted.

[0060] The model obtained from the current training is determined to be the barcode prediction model.

[0061] In this embodiment, the pre-constructed deep learning module includes a feature extraction module, a generator network, a first regression network, and a second regression network. For each sample image in the training set, the feature extraction module extracts image features to obtain a feature map corresponding to the sample image. The feature extraction module includes a feature extraction algorithm, which can employ the LBP (Local Binary Patterns) feature extraction algorithm. Then, the generator network generates detection boxes. Specifically, based on the first ground truth location information corresponding to the feature map (i.e., the coordinates of the four endpoints of the rectangle enclosing the barcode), a first initial detection box of the feature map can be generated. Furthermore, based on each second ground truth location information corresponding to the feature map (i.e., the coordinates of the four endpoints of the rectangle enclosing each bar or empty space in the barcode), multiple second initial detection boxes of the feature map can be generated. Next, the position of the generated initial detection boxes is adjusted using a first regression network to make them closer to the barcode. By obtaining the position information of the initial detection boxes in the feature map, a first error is calculated between the position information of the initial detection boxes and the corresponding first true position information. The network parameters of the first regression network are then adjusted based on this first error until the first error is less than a preset value. The positions of the generated second initial detection boxes are then adjusted using a second regression network to make each adjusted second initial detection box closer to the corresponding bar or space. By obtaining the position information of each second initial detection box in the feature map, a second error is calculated between the position information of each second initial detection box and the corresponding second true position information. The network parameters of the second regression network are then adjusted based on this second error until the second error is less than a preset value. If both the first and second error values ​​are less than the preset values, the network parameters of the first and second regression networks are no longer adjusted. The final network parameters of the first and second regression networks are then confirmed, and the currently trained model is determined as the barcode prediction model.

[0062] The methods for calculating the first and second error quantities are relatively mature technologies and will not be elaborated here.

[0063] S15. Test the barcode prediction model according to the test set and preset encoding rules. If the preset test conditions are met, stop training and obtain the target barcode prediction model.

[0064] In this embodiment, the barcode prediction model is tested according to a test set and preset encoding rules to evaluate its performance and improve its generalization ability. The barcode prediction model is then optimized based on the test results until preset test conditions are met, at which point training stops, thus obtaining the target barcode prediction model.

[0065] In one embodiment, see Figure 2 Step S15 includes:

[0066] S151. For each sample image in the test set, input the sample image into the barcode prediction model to obtain multiple first candidate boxes corresponding to the first true location information of the sample image, and multiple second candidate boxes corresponding to each second true location information of the sample image.

[0067] S152. Calculate the first overlap between the position information of each first candidate box and the first real position information, and filter out the first overlap greater than the first preset value from the first overlap as the first candidate overlap.

[0068] S153. Calculate the second overlap between the position information of each second candidate box and the corresponding second real position information, and filter out the second overlap greater than the second preset value from the second overlap corresponding to each second candidate box as the second candidate overlap corresponding to each second candidate box.

[0069] S154. Select the largest first candidate overlap from the first candidate overlap, and determine the first candidate box corresponding to the selected first candidate overlap as the first prediction box of the sample image.

[0070] S155. Select the largest second candidate overlap from the second candidate overlap corresponding to each second candidate box, and determine each second candidate box corresponding to the selected second candidate overlap as multiple second prediction boxes of the sample image.

[0071] S156. Based on the position information and bar / space type of multiple second prediction boxes in the sample image, and the preset encoding rules, obtain the corresponding decoded characters and the total number of decoded characters, and based on the position information of the first prediction box in the sample image and the preset encoding rules, obtain the corresponding total number of characters.

[0072] S157. If the total number of decoded characters corresponding to each sample image in the test set is equal to the total number of corresponding characters, then stop training and obtain the target barcode prediction model.

[0073] In this embodiment, the barcode prediction model is tested according to a test set and preset encoding rules. Specifically, for each sample image in the test set, the barcode prediction model obtains multiple first candidate boxes corresponding to the first true location information of the sample image, that is, multiple first candidate boxes for predicting the position of the barcode in the sample image. At the same time, multiple second candidate boxes corresponding to each second true location information of the sample image are obtained, that is, multiple second candidate boxes for predicting each bar or space in the sample image. Then, it is necessary to select the candidate box that is closest to the predicted target position from multiple first candidate boxes and multiple second candidate boxes as the corresponding prediction box. Specifically, the first overlap between the position information of each first candidate box and the first real position information is calculated, and the first overlap greater than the first preset value is selected from the calculated first overlap as the first candidate overlap. Furthermore, the largest first candidate overlap is selected from the first candidate overlap, and the first candidate box corresponding to the selected first candidate overlap is determined as the first prediction box of the sample image. The first prediction box of the sample image is closest to the rectangle formed by the first real position information of the sample image, thereby obtaining the first prediction box used to indicate the position information of the barcode in the sample image. The second overlap degree between the position information of each second candidate box and the corresponding second true position information is calculated. From the second overlap degrees corresponding to each second candidate box, those greater than a second preset value are selected as the second candidate overlap degrees for each second candidate box. Further, the largest second candidate overlap degree is selected from the second candidate overlap degrees corresponding to each second candidate box. Each second candidate box corresponding to the selected second candidate overlap degree is determined as multiple second prediction boxes of the sample image, thus obtaining multiple second prediction boxes indicating the position information of each bar or empty space in the sample image and the bar / empty type. Based on the multiple second prediction boxes of the sample image and the preset encoding rules, the corresponding decoded characters are obtained. The total number of decoded characters is then calculated to obtain the total number of decoded characters. Simultaneously, the total number of characters is obtained based on the first prediction box and the preset encoding rules of the sample image. This total number of characters is used to verify the total number of decoded characters. If the total number of decoded characters for each sample image in the test set is equal to the total number of characters, it indicates that the current barcode prediction model can accurately predict the position of the barcode and bars / spaces in the image. Based on the multiple second prediction boxes generated by the current barcode prediction model and the preset encoding rules, the barcode can be correctly decoded. Therefore, training can be stopped, and the target barcode prediction model can be obtained.

[0074] In one embodiment, step S156 includes:

[0075] Based on the position information of each second prediction box in the multiple second prediction boxes of the sample image, the width information of each second prediction box is obtained;

[0076] Based on the width information of each second prediction box, the bar / space type of each second prediction box, and the preset encoding rules, the width information is converted into the corresponding decoded characters, and the number of decoded characters is obtained as the total number of decoded characters.

[0077] The width information of the first prediction box is obtained based on the position information of the first prediction box in the sample image;

[0078] The total number of characters is calculated based on the width information of the first prediction box and the preset encoding rules.

[0079] In this embodiment, the barcode encoding method is modular combination encoding. Modular combination encoding specifies the width of a unit module (e.g., 0.33 mm), and the width of a bar or space is one or more times the width of the unit module. A bar or space of one unit module width represents binary "1" and "0" respectively. The multiple of the bar or space width to the unit module width represents the number of consecutive "1"s and consecutive "0"s respectively. Common barcodes using modular combination encoding include EAN code, Code 93, and Code 128. Taking EAN code as an example, the modular combination encoding method is briefly explained. Figure 3 As shown, Figure 3 The EAN code component structure provided in this embodiment of the invention consists of characters 9 and 2. EAN code is a continuous barcode. Each barcode character consists of 2 bars and 2 spaces. Each barcode character has a total of 7 unit module widths. The barcode characters "9" and "2" are represented by binary "0001011" and "0011011" respectively. The corresponding unit combinations are "3 times the width of space, 1 times the width of bar, 1 times the width of space, 2 times the width of bar" and "2 times the width of space, 2 times the width of bar, 1 times the width of space, 2 times the width of bar".

[0080] Based on the position information of each second prediction box in multiple second prediction boxes of the sample image, the width information of each second prediction box is obtained, that is, the width information of each bar and space in the sample image. The second prediction box is used to indicate the position information and bar / space type of each bar or space in the sample image. The coordinates of the four endpoints of the second prediction box are obtained, and the width information of the second prediction box is calculated using the two horizontal coordinates below or above the second prediction box. Based on the width information of each second prediction box, the bar / space type of each second prediction box, and the preset encoding rules, the width information is converted into corresponding decoded characters. For example, if the calculated width of a second prediction box is three times the width of a unit module as specified in the preset encoding rules, and the bar / space type of the second prediction box is empty, then the binary "000" corresponding to the second prediction box is obtained. The binary values ​​obtained from all second prediction boxes are converted into corresponding decoded characters, and the number of decoded characters is obtained as the total number of decoded characters. The preset encoding rules provided in this embodiment include, but are not limited to, EAN code, 93 code, and 128 code encoding rules. Furthermore, the width information of the first prediction box is obtained based on its position information, which is equivalent to the width information of the barcode in the sample image. The first prediction box indicates the position of the barcode in the sample image. The coordinates of the four endpoints of the first prediction box are obtained, and the width information of the first prediction box is calculated using the two horizontal coordinates below or above it. The calculated width information of the first prediction box is N times the width of the unit module specified in the preset encoding rules, where N is a positive integer. This gives the total number of characters N in the sample image. The total number of characters in the sample image is used to verify the total number of decoded characters converted from multiple second prediction boxes. If the total number of decoded characters in the sample image is not equal to the total number of characters, it indicates that multiple second prediction boxes overlap or are missing, leading to decoding errors.

[0081] In one embodiment, the deep learning-based barcode decoding method provided by this invention further includes:

[0082] In the test set, when the number of sample images whose total number of corresponding decoded characters is not equal to the total number of corresponding characters reaches a preset number, a supplementary test set is constructed using all sample images whose total number of corresponding decoded characters is not equal to the total number of corresponding characters, and a new training set and a new test set are obtained.

[0083] The new barcode prediction model is trained using the new training set, and the new barcode prediction model is tested using the new test set.

[0084] When the total number of decoded characters corresponding to each sample image in the new test set is equal to the total number of characters, the new barcode prediction model is supplemented and tested using the supplementary test set.

[0085] Training stops when the total number of decoded characters for each sample image in the supplementary test set is equal to the total number of characters, and the target barcode prediction model is obtained.

[0086] In this embodiment, during the verification process of the first prediction box and multiple second prediction boxes predicted from sample images in the test set, if an error occurs where the total number of decoded characters in a sample image is not equal to the total number of corresponding characters, the sample image is recorded as a historical error sample. When the number of historical error samples reaches a preset number, a supplementary test set is constructed using all historical error samples, and a new training set and a new test set are obtained to conduct a new round of training and testing on the currently obtained barcode prediction model until the latest round of testing is error-free. When the latest round of testing is error-free, that is, when the total number of decoded characters corresponding to each sample image in the new test set is equal to the total number of corresponding characters, the supplementary test set is used to conduct supplementary testing on the new barcode prediction model, and the historical error samples are used to test the new barcode prediction model. When the total number of decoded characters corresponding to all historical error samples is equal to the total number of corresponding characters, training stops, and the target barcode prediction model is obtained. By continuously training and testing the current barcode prediction model until no errors appear in the latest test set and all historical error samples are correctly predicted, the training ends and the target barcode prediction model is obtained. This results in a target barcode prediction model with better performance and higher accuracy, which can quickly and accurately predict the position information of barcodes, bars and spaces in the image, thus making the decoding results more accurate.

[0087] S16. Receive the barcode image to be decoded, input the barcode image to be decoded into the target barcode prediction model, and obtain a first prediction box and multiple second prediction boxes; wherein, the first prediction box is used to indicate the predicted position of the barcode in the barcode image to be decoded, and the second prediction box is used to indicate the predicted position of each bar or empty space in the barcode image to be decoded.

[0088] In this embodiment, the barcode image to be decoded is an image containing a barcode that has undergone image preprocessing. The barcode image to be decoded needs to be decoded. First, the barcode image to be decoded is input into the target barcode prediction model. The target barcode prediction model predicts a first prediction box that indicates the predicted position information of the barcode in the barcode image to be decoded, and a plurality of second prediction boxes that indicate the predicted position information and bar / space type of each bar or space in the barcode image to be decoded.

[0089] S17. Based on the position information and bar / space type of multiple second prediction boxes, and the preset encoding rules, obtain the decoded characters and the total number of decoded characters of the barcode image to be decoded, and based on the position information of the first prediction box and the preset encoding rules, obtain the total number of characters of the barcode image to be decoded.

[0090] In this embodiment, the barcode in the image to be decoded can be decoded based on the first prediction box, multiple second prediction boxes, and preset encoding rules. This avoids environmental interference and achieves accurate positioning of the barcode, bars, and spaces, thereby further improving decoding efficiency. The modular grouping encoding method specifies the width of the unit module (e.g., 0.33 mm). The width of the bars or spaces is one or more times the width of the unit module. A bar or space of one unit module width represents binary "1" and "0" respectively. The width of the bars and spaces is a multiple of the width of the unit module, representing a number of consecutive "1"s and consecutive "0"s respectively. Based on the position information (i.e., the coordinates of the four endpoints) of each second prediction box in the multiple second prediction boxes of the image to be decoded, the width information of each second prediction box is calculated using the two horizontal coordinates below or above each second prediction box. Furthermore, based on the width information of each second prediction box, the bar / space type of each second prediction box, and the preset encoding rules, the width information is converted into the corresponding decoded character, and the number of these decoded characters is obtained as the total number of decoded characters. Based on the position information (i.e., the coordinates of the four endpoints) of the first prediction box in the barcode image to be decoded, the width information of the first prediction box is calculated by the two horizontal coordinates below or above the first prediction box. Then, based on the width information of the first prediction box and the unit module width specified by the preset encoding rules, the total number of characters is calculated. This total number of characters is used to verify the total number of decoded characters converted from multiple second prediction boxes, and to verify whether the obtained decoded characters are correct, which helps to improve the accuracy of the barcode decoding results.

[0091] S18. If the total number of decoded characters in the barcode image to be decoded is equal to the total number of characters in the barcode image to be decoded, then output the decoded characters of the barcode image to be decoded.

[0092] In this embodiment, if the total number of decoded characters in the barcode image to be decoded is equal to the total number of characters in the barcode image to be decoded, it means that the target barcode prediction model has correctly predicted the position information of the barcode and each bar and space in the barcode image to be decoded, and there is no overlap or omission of multiple second prediction boxes. The decoded characters obtained are correct, and the decoded characters of the barcode image to be decoded are output. This invention can achieve accurate positioning and decoding of barcodes in images.

[0093] In one embodiment, step S18 is followed by:

[0094] If the total number of characters to be decoded in the barcode image is not equal to the total number of characters in the barcode image, a barcode decoding failure message will be output.

[0095] In this embodiment, if the total number of decoded characters in the barcode image to be decoded is not equal to the total number of characters in the barcode image to be decoded, it indicates that multiple second prediction boxes overlap or are missing, resulting in incorrect decoded characters. In this case, a barcode decoding failure message is output to facilitate re-acquiring the barcode image to be decoded and to provide a prompt to the user.

[0096] This invention discloses a deep learning-based barcode decoding method that constructs training and testing sets using sample images and annotation information. A pre-built deep learning model is trained using the training set to obtain a barcode prediction model. The barcode prediction model is then tested using the testing set and preset encoding rules. If preset testing conditions are met, training stops, resulting in a target barcode prediction model. The received barcode image to be decoded is input into the target barcode prediction model to obtain a first prediction box and multiple second prediction boxes. The corresponding decoded characters and the total number of decoded characters are obtained based on the multiple second prediction boxes and preset encoding rules, and the total number of characters is obtained based on the first prediction box and preset encoding rules. If the total number of decoded characters equals the total number of characters, the decoded characters are output. This invention, by combining deep learning and decoding methods, achieves accurate barcode positioning, improves decoding efficiency and accuracy, and avoids environmental interference.

[0097] This invention also provides a deep learning-based barcode decoding device, which is used to execute any of the aforementioned deep learning-based barcode decoding methods. Specifically, please refer to... Figure 4 , Figure 4 This is a schematic block diagram of a deep learning-based barcode decoding device provided in an embodiment of the present invention. The embodiment of the present invention provides a deep learning-based barcode decoding device 100, which includes a sample image acquisition unit 11, an information annotation unit 12, a prediction set splitting unit 13, a model training unit 14, a model testing unit 15, a barcode receiving unit 16, a barcode decoding unit 17, and a decoded character output unit 18.

[0098] The sample image acquisition unit 11 is used to acquire a set of barcode images and perform image preprocessing on each barcode image in the set to obtain a sample image corresponding to each barcode image, so as to form a sample image set.

[0099] In this embodiment, the acquired barcode image set includes a large number of images containing barcodes. A barcode is a graphic identifier that represents a set of information by arranging multiple black bars and spaces of varying widths according to a specific encoding rule. Common barcodes consist of parallel lines arranged with black bars (simply called bars) and white bars (simply called spaces) of significantly different reflectivities. Since the barcode images in the barcode image set contain much other information unrelated to the barcode itself, image processing is required to eliminate interference and reduce the impact on subsequent barcode feature extraction. Image preprocessing is performed on the barcode images in the barcode image set to obtain sample images for model training and testing. These processed sample images are then combined to form a sample image set.

[0100] In one embodiment, the sample image acquisition unit 11 is specifically used for:

[0101] Multiple barcode images acquired by an image acquisition device are combined to form a barcode image set;

[0102] Each barcode image is subjected to noise reduction filtering to obtain each first intermediate image;

[0103] Each first intermediate image is converted to grayscale to obtain each second intermediate image;

[0104] Each second intermediate image is binarized to obtain each third intermediate image; each third intermediate image includes a barcode area and a background area.

[0105] The barcode area of ​​each third intermediate image is cropped to a preset size to obtain each fourth intermediate image;

[0106] Each fourth intermediate image is angled to obtain a sample image corresponding to each barcode image;

[0107] Each sample image is combined into a sample image set.

[0108] In this embodiment, barcodes on items in complex usage scenarios are captured by an image acquisition device, such as a CCD (Charge Coupled Device) camera, to obtain a large number of images containing barcodes. These numerous barcode images are then combined into a barcode image set. Each barcode image undergoes sequential noise reduction filtering, grayscale conversion, binarization, cropping, and angle adjustment to obtain a sample image corresponding to each barcode image. Specifically, the image signal of the barcode image is first converted into a digital signal, and a frequency domain filtering algorithm is used to perform noise reduction filtering on the barcode image. Then, the first intermediate image is grayscale converted to improve image processing efficiency; an averaging method can be used to process the image at grayscale levels, reducing the resource consumption of color barcodes. Next, redundant information such as color is removed from the second intermediate image to improve computational speed. This requires binarization. The target area (barcode area) and background area in the resulting third intermediate image are then distinguished. During binarization, a segmentation threshold is obtained using color or grayscale features based on deep learning. The image is compared to the threshold; areas above the threshold are classified as black, and areas below are classified as white, thus dividing the image into white and black parts. The barcode area in the third intermediate image is then cropped to a preset size. Finally, the angle of the fourth intermediate image is adjusted so that the barcode is vertical, resulting in sample images. These sample images form a sample image set.

[0109] The information annotation unit 12 is used to annotate the position information of the barcode of each sample image in the sample image set to obtain the first true position information of each sample image; and to annotate the position information and bar / space type of each bar and space of each sample image to obtain multiple second true position information of each sample image. The first true position information and multiple second true position information of each sample image are combined to form an annotation information set.

[0110] In this embodiment, the position information of the barcode in each sample image in the sample image set is labeled to obtain the first true position information of each sample image. The position information of the barcode can be represented by the position information of the rectangle that encloses the barcode. The first true position information is the coordinates of the four endpoints of the rectangle that encloses the barcode, and the width information of the barcode is calculated based on the first true position information. At the same time, the position information and bar / space type of each bar and space in each sample image are labeled. Since the barcode includes multiple bars and spaces, multiple second true position information of each sample image are obtained. The bar / space type includes bars and spaces, which are used to represent the bars and spaces in the barcode. The position information of the bars or spaces in the barcode can be represented by the position information of the rectangle that encloses the bars or spaces in the barcode. The second true position information is the coordinates of the four endpoints of the rectangle that encloses each bar or space in the barcode, and the width information of each bar or space can be calculated based on the position information of each bar or space. Then, the width information of each bar or space can be converted into the corresponding characters according to the preset encoding rules. The first true location information and multiple second true location information for each sample image are stored in a single annotation information file. Each sample image corresponds to one annotation information file, and all annotation information files are used to form the annotation information set for the sample image set. The barcodes and their bars and spaces in the sample images are annotated, and the annotation information is used to train the model so that the trained model can locate the barcodes and their bars and spaces in the images.

[0111] The prediction set splitting unit 13 is used to form a barcode prediction set using the sample image set and the annotation information set, and to split the barcode prediction set into a training set and a test set.

[0112] In this embodiment, a barcode prediction set is composed of a sample image set and a labeled information set, and this barcode prediction set is split into a training set and a test set. The sample images in the training set and the test set correspond one-to-one with the labeled information files. The training set is used to train the model's parameters, and the test set is used to evaluate the model's generalization ability. Specifically, the barcode prediction set can be randomly split into training and test sets, or it can be split proportionally.

[0113] Model training unit 14 is used to train a pre-built deep learning model based on the training set to obtain a barcode prediction model.

[0114] In this embodiment, a pre-built deep learning model is trained using a training set to obtain model parameters, thereby obtaining a barcode prediction model. This barcode prediction model can predict the barcode in an image and the positional information of its bars and spaces. Furthermore, the pre-built deep learning model can be a machine learning model based on a deep convolutional neural network.

[0115] In one embodiment, the deep learning model includes a feature extraction module, a generative network, a first regression network, and a second regression network. The model training unit 14 is specifically used for:

[0116] Each sample image in the training set is input into the feature extraction module to obtain the feature map corresponding to each sample image;

[0117] By generating a network, a first initial detection box is determined in each feature map according to each first real location information in the training set, and multiple second initial detection boxes are determined in each feature map according to each second real location information in the training set.

[0118] The location information of each initial detection box is obtained through the first regression network, the first error between the location information of each initial detection box and the corresponding first true location information is calculated, and the network parameters of the first regression network are adjusted according to the first error.

[0119] The location information of each second initial detection box is obtained through the second regression network. The second error between the location information of each second initial detection box and the corresponding second true location information is calculated. The network parameters of the second regression network are adjusted according to the second error.

[0120] If both the first error and the second error are less than the preset value, the network parameters of the first regression network and the network parameters of the second regression network will not be adjusted.

[0121] The model obtained from the current training is determined to be the barcode prediction model.

[0122] In this embodiment, the pre-constructed deep learning module includes a feature extraction module, a generator network, a first regression network, and a second regression network. For each sample image in the training set, the feature extraction module extracts image features to obtain a feature map corresponding to the sample image. The feature extraction module includes a feature extraction algorithm, which can employ the LBP (Local Binary Patterns) feature extraction algorithm. Then, the generator network generates detection boxes. Specifically, based on the first ground truth location information corresponding to the feature map (i.e., the coordinates of the four endpoints of the rectangle enclosing the barcode), a first initial detection box of the feature map can be generated. Furthermore, based on each second ground truth location information corresponding to the feature map (i.e., the coordinates of the four endpoints of the rectangle enclosing each bar or empty space in the barcode), multiple second initial detection boxes of the feature map can be generated. Next, the position of the generated initial detection boxes is adjusted using a first regression network to make them closer to the barcode. By obtaining the position information of the initial detection boxes in the feature map, a first error is calculated between the position information of the initial detection boxes and the corresponding first true position information. The network parameters of the first regression network are then adjusted based on this first error until the first error is less than a preset value. The positions of the generated second initial detection boxes are then adjusted using a second regression network to make each adjusted second initial detection box closer to the corresponding bar or space. By obtaining the position information of each second initial detection box in the feature map, a second error is calculated between the position information of each second initial detection box and the corresponding second true position information. The network parameters of the second regression network are then adjusted based on this second error until the second error is less than a preset value. If both the first and second error values ​​are less than the preset values, the network parameters of the first and second regression networks are no longer adjusted. The final network parameters of the first and second regression networks are then confirmed, and the currently trained model is determined as the barcode prediction model.

[0123] The methods for calculating the first and second error quantities are relatively mature technologies and will not be elaborated here.

[0124] The model testing unit 15 is used to test the barcode prediction model according to the test set and preset encoding rules. If the preset test conditions are met, the training is stopped and the target barcode prediction model is obtained.

[0125] In this embodiment, the barcode prediction model is tested according to a test set and preset encoding rules to evaluate its performance and improve its generalization ability. The barcode prediction model is then optimized based on the test results until preset test conditions are met, at which point training stops, thus obtaining the target barcode prediction model.

[0126] In one embodiment, see Figure 2 Model test unit 15 is specifically used for:

[0127] For each sample image in the test set, the sample image is input into the barcode prediction model to obtain multiple first candidate boxes corresponding to the first true location information of the sample image, and multiple second candidate boxes corresponding to each second true location information of the sample image.

[0128] Calculate the first overlap between the position information of each first candidate box and the first real position information, and filter out the first overlap that is greater than the first preset value from the first overlap as the first candidate overlap.

[0129] Calculate the second overlap between the position information of each second candidate box and the corresponding second real position information, and filter out the second overlap greater than the second preset value from the second overlap corresponding to each second candidate box as the second candidate overlap corresponding to each second candidate box;

[0130] The largest first candidate overlap is selected from the first candidate overlap, and the first candidate box corresponding to the selected first candidate overlap is determined as the first predicted box of the sample image.

[0131] The maximum second candidate overlap is selected from the second candidate overlaps corresponding to each second candidate box, and each second candidate box corresponding to the selected second candidate overlap is determined as multiple second prediction boxes of the sample image.

[0132] Based on the position information and bar / space type of multiple second prediction boxes in the sample image, and the preset encoding rules, the corresponding decoded characters and the total number of decoded characters are obtained, and based on the position information of the first prediction box in the sample image and the preset encoding rules, the total number of characters is obtained.

[0133] If the total number of decoded characters for each sample image in the test set is equal to the total number of characters, then training stops, and the target barcode prediction model is obtained.

[0134] In this embodiment, the barcode prediction model is tested according to a test set and preset encoding rules. Specifically, for each sample image in the test set, the barcode prediction model obtains multiple first candidate boxes corresponding to the first true location information of the sample image, that is, multiple first candidate boxes for predicting the position of the barcode in the sample image. At the same time, multiple second candidate boxes corresponding to each second true location information of the sample image are obtained, that is, multiple second candidate boxes for predicting each bar or space in the sample image. Then, it is necessary to select the candidate box that is closest to the predicted target position from multiple first candidate boxes and multiple second candidate boxes as the corresponding prediction box. Specifically, the first overlap between the position information of each first candidate box and the first real position information is calculated, and the first overlap greater than the first preset value is selected from the calculated first overlap as the first candidate overlap. Furthermore, the largest first candidate overlap is selected from the first candidate overlap, and the first candidate box corresponding to the selected first candidate overlap is determined as the first prediction box of the sample image. The first prediction box of the sample image is closest to the rectangle formed by the first real position information of the sample image, thereby obtaining the first prediction box used to indicate the position information of the barcode in the sample image. The second overlap degree between the position information of each second candidate box and the corresponding second true position information is calculated. From the second overlap degrees corresponding to each second candidate box, those greater than a second preset value are selected as the second candidate overlap degrees for each second candidate box. Further, the largest second candidate overlap degree is selected from the second candidate overlap degrees corresponding to each second candidate box. Each second candidate box corresponding to the selected second candidate overlap degree is determined as multiple second prediction boxes of the sample image, thus obtaining multiple second prediction boxes indicating the position information of each bar or empty space in the sample image and the bar / empty type. Based on the multiple second prediction boxes of the sample image and the preset encoding rules, the corresponding decoded characters are obtained. The total number of decoded characters is then calculated to obtain the total number of decoded characters. Simultaneously, the total number of characters is obtained based on the first prediction box and the preset encoding rules of the sample image. This total number of characters is used to verify the total number of decoded characters. If the total number of decoded characters for each sample image in the test set is equal to the total number of characters, it indicates that the current barcode prediction model can accurately predict the position of the barcode and bars / spaces in the image. Based on the multiple second prediction boxes generated by the current barcode prediction model and the preset encoding rules, the barcode can be correctly decoded. Therefore, training can be stopped, and the target barcode prediction model can be obtained.

[0135] In one embodiment, the model testing unit 15 is further configured to:

[0136] Based on the position information of each second prediction box in the multiple second prediction boxes of the sample image, the width information of each second prediction box is obtained;

[0137] Based on the width information of each second prediction box, the bar / space type of each second prediction box, and the preset encoding rules, the width information is converted into the corresponding decoded characters, and the number of decoded characters is obtained as the total number of decoded characters.

[0138] The width information of the first prediction box is obtained based on the position information of the first prediction box in the sample image;

[0139] The total number of characters is calculated based on the width information of the first prediction box and the preset encoding rules.

[0140] In this embodiment, the barcode encoding method is modular combination encoding. Modular combination encoding specifies the width of a unit module (e.g., 0.33 mm), and the width of a bar or space is one or more times the width of the unit module. A bar or space of one unit module width represents binary "1" and "0" respectively. The multiple of the bar or space width to the unit module width represents the number of consecutive "1"s and consecutive "0"s respectively. Common barcodes using modular combination encoding include EAN code, Code 93, and Code 128. Taking EAN code as an example, the modular combination encoding method is briefly explained. Figure 3 As shown, Figure 3 The EAN code component structure provided in this embodiment of the invention consists of characters 9 and 2. EAN code is a continuous barcode. Each barcode character consists of 2 bars and 2 spaces. Each barcode character has a total of 7 unit module widths. The barcode characters "9" and "2" are represented by binary "0001011" and "0011011" respectively. The corresponding unit combinations are "3 times the width of space, 1 times the width of bar, 1 times the width of space, 2 times the width of bar" and "2 times the width of space, 2 times the width of bar, 1 times the width of space, 2 times the width of bar".

[0141] Based on the position information of each second prediction box in multiple second prediction boxes of the sample image, the width information of each second prediction box is obtained, that is, the width information of each bar and space in the sample image. The second prediction box is used to indicate the position information and bar / space type of each bar or space in the sample image. The coordinates of the four endpoints of the second prediction box are obtained, and the width information of the second prediction box is calculated using the two horizontal coordinates below or above the second prediction box. Based on the width information of each second prediction box, the bar / space type of each second prediction box, and the preset encoding rules, the width information is converted into corresponding decoded characters. For example, if the calculated width of a second prediction box is three times the width of a unit module as specified in the preset encoding rules, and the bar / space type of the second prediction box is empty, then the binary "000" corresponding to the second prediction box is obtained. The binary values ​​obtained from all second prediction boxes are converted into corresponding decoded characters, and the number of decoded characters is obtained as the total number of decoded characters. The preset encoding rules provided in this embodiment include, but are not limited to, EAN code, 93 code, and 128 code encoding rules. Furthermore, the width information of the first prediction box is obtained based on its position information, which is equivalent to the width information of the barcode in the sample image. The first prediction box indicates the position of the barcode in the sample image. The coordinates of the four endpoints of the first prediction box are obtained, and the width information of the first prediction box is calculated using the two horizontal coordinates below or above it. The calculated width information of the first prediction box is N times the width of the unit module specified in the preset encoding rules, where N is a positive integer. This gives the total number of characters N in the sample image. The total number of characters in the sample image is used to verify the total number of decoded characters converted from multiple second prediction boxes. If the total number of decoded characters in the sample image is not equal to the total number of characters, it indicates that multiple second prediction boxes overlap or are missing, leading to decoding errors.

[0142] In one embodiment, the model testing unit 15 is further configured to:

[0143] In the test set, when the number of sample images whose total number of corresponding decoded characters is not equal to the total number of corresponding characters reaches a preset number, a supplementary test set is constructed using all sample images whose total number of corresponding decoded characters is not equal to the total number of corresponding characters, and a new training set and a new test set are obtained.

[0144] The new barcode prediction model is trained using the new training set, and the new barcode prediction model is tested using the new test set.

[0145] When the total number of decoded characters corresponding to each sample image in the new test set is equal to the total number of characters, the new barcode prediction model is supplemented and tested using the supplementary test set.

[0146] Training stops when the total number of decoded characters for each sample image in the supplementary test set is equal to the total number of characters, and the target barcode prediction model is obtained.

[0147] In this embodiment, during the verification process of the first prediction box and multiple second prediction boxes predicted from sample images in the test set, if an error occurs where the total number of decoded characters in a sample image is not equal to the total number of corresponding characters, the sample image is recorded as a historical error sample. When the number of historical error samples reaches a preset number, a supplementary test set is constructed using all historical error samples, and a new training set and a new test set are obtained to conduct a new round of training and testing on the currently obtained barcode prediction model until the latest round of testing is error-free. When the latest round of testing is error-free, that is, when the total number of decoded characters corresponding to each sample image in the new test set is equal to the total number of corresponding characters, the supplementary test set is used to conduct supplementary testing on the new barcode prediction model, and the historical error samples are used to test the new barcode prediction model. When the total number of decoded characters corresponding to all historical error samples is equal to the total number of corresponding characters, training stops, and the target barcode prediction model is obtained. By continuously training and testing the current barcode prediction model until no errors appear in the latest test set and all historical error samples are correctly predicted, the training ends and the target barcode prediction model is obtained. This results in a target barcode prediction model with better performance and higher accuracy, which can quickly and accurately predict the position information of barcodes, bars and spaces in the image, thus making the decoding results more accurate.

[0148] The barcode receiving unit 16 is used to receive the barcode image to be decoded, input the barcode image to be decoded into the target barcode prediction model, and obtain a first prediction box and a plurality of second prediction boxes; wherein, the first prediction box is used to indicate the prediction position of the barcode in the barcode image to be decoded, and the second prediction box is used to indicate the prediction position of each bar or space in the barcode image to be decoded.

[0149] In this embodiment, the barcode image to be decoded is an image containing a barcode that has undergone image preprocessing. The barcode image to be decoded needs to be decoded. First, the barcode image to be decoded is input into the target barcode prediction model. The target barcode prediction model predicts a first prediction box that indicates the predicted position information of the barcode in the barcode image to be decoded, and a plurality of second prediction boxes that indicate the predicted position information and bar / space type of each bar or space in the barcode image to be decoded.

[0150] The barcode decoding unit 17 is used to obtain the decoded characters and the total number of decoded characters of the barcode image to be decoded based on the position information and bar / space type of multiple second prediction boxes and preset encoding rules, and to obtain the total number of characters of the barcode image to be decoded based on the position information of the first prediction box and preset encoding rules.

[0151] In this embodiment, the barcode in the image to be decoded can be decoded based on the first prediction box, multiple second prediction boxes, and preset encoding rules. This avoids environmental interference and achieves accurate positioning of the barcode, bars, and spaces, thereby further improving decoding efficiency. The modular grouping encoding method specifies the width of the unit module (e.g., 0.33 mm). The width of the bars or spaces is one or more times the width of the unit module. A bar or space of one unit module width represents binary "1" and "0" respectively. The width of the bars and spaces is a multiple of the width of the unit module, representing a number of consecutive "1"s and consecutive "0"s respectively. Based on the position information (i.e., the coordinates of the four endpoints) of each second prediction box in the multiple second prediction boxes of the image to be decoded, the width information of each second prediction box is calculated using the two horizontal coordinates below or above each second prediction box. Furthermore, based on the width information of each second prediction box, the bar / space type of each second prediction box, and the preset encoding rules, the width information is converted into the corresponding decoded character, and the number of these decoded characters is obtained as the total number of decoded characters. Based on the position information (i.e., the coordinates of the four endpoints) of the first prediction box in the barcode image to be decoded, the width information of the first prediction box is calculated by the two horizontal coordinates below or above the first prediction box. Then, based on the width information of the first prediction box and the unit module width specified by the preset encoding rules, the total number of characters is calculated. This total number of characters is used to verify the total number of decoded characters converted from multiple second prediction boxes, and to verify whether the obtained decoded characters are correct, which helps to improve the accuracy of the barcode decoding results.

[0152] The decoding character output unit 18 is used to output the decoded characters of the barcode image to be decoded if the total number of decoded characters of the barcode image to be decoded is equal to the total number of characters of the barcode image to be decoded.

[0153] In this embodiment, if the total number of decoded characters in the barcode image to be decoded is equal to the total number of characters in the barcode image to be decoded, it means that the target barcode prediction model has correctly predicted the position information of the barcode and each bar and space in the barcode image to be decoded, and there is no overlap or omission of multiple second prediction boxes. The decoded characters obtained are correct, and the decoded characters of the barcode image to be decoded are output. This invention can achieve accurate positioning and decoding of barcodes in images.

[0154] In one embodiment, the deep learning-based barcode decoding device 100 provided by this invention further includes:

[0155] The decoding failure prompt unit is used to output a barcode decoding failure prompt message if the total number of characters to be decoded in the barcode image is not equal to the total number of characters in the barcode image.

[0156] In this embodiment, if the total number of decoded characters in the barcode image to be decoded is not equal to the total number of characters in the barcode image to be decoded, it indicates that multiple second prediction boxes overlap or are missing, resulting in incorrect decoded characters. In this case, a barcode decoding failure message is output to facilitate re-acquiring the barcode image to be decoded and to provide a prompt to the user.

[0157] The barcode decoding device based on deep learning disclosed in this invention achieves accurate barcode positioning by combining deep learning and decoding methods, and improves decoding efficiency and accuracy while avoiding environmental interference.

[0158] The deep learning-based barcode decoding method described above can be implemented as a computer program, which can be used in various ways, such as... Figure 5 It runs on the computer device shown.

[0159] Please see Figure 5 , Figure 5 This is a schematic block diagram of a computer device provided in an embodiment of the present invention. The computer device 500 includes a processor 502, a memory, and a network interface 505 connected via a device bus 501, wherein the memory may include a storage medium 503 and internal memory 504.

[0160] The storage medium 503 may store the operating device 5031 and the computer program 5032. When the computer program 5032 is executed, it causes the processor 502 to execute a barcode decoding method based on deep learning.

[0161] The processor 502 provides computing and control capabilities to support the operation of the entire computer device 500.

[0162] The internal memory 504 provides an environment for the operation of the computer program 5032 in the storage medium 503. When the computer program 5032 is executed by the processor 502, the processor 502 can execute a barcode decoding method based on deep learning.

[0163] This network interface 505 is used for network communication, such as providing data transmission. Those skilled in the art will understand that... Figure 5 The structure shown is merely a block diagram of a portion of the structure related to the present invention and does not constitute a limitation on the computer device 500 to which the present invention is applied. The specific computer device 500 may include more or fewer components than shown in the figure, or combine certain components, or have different component arrangements.

[0164] The processor 502 is used to run the computer program 5032 stored in the memory to implement the deep learning-based barcode decoding method disclosed in the embodiments of the present invention.

[0165] Those skilled in the art will understand that Figure 5 The embodiments of the computer device shown do not constitute a limitation on the specific configuration of the computer device. In other embodiments, the computer device may include more or fewer components than illustrated, or combine certain components, or have different component arrangements. For example, in some embodiments, the computer device may include only memory and a processor. In such embodiments, the structure and function of the memory and processor are different from those shown. Figure 5 The embodiments shown are consistent and will not be repeated here.

[0166] It should be understood that, in this embodiment of the invention, the processor 502 may be a Central Processing Unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0167] In another embodiment of the present invention, a computer-readable storage medium is provided. This computer-readable storage medium may be a non-volatile computer-readable storage medium or a volatile computer-readable storage medium. The computer-readable storage medium stores a computer program, wherein the computer program, when executed by a processor, implements the deep learning-based barcode decoding method disclosed in the embodiments of the present invention.

[0168] Those skilled in the art will readily understand that, for the sake of convenience and brevity, the specific working processes of the devices, apparatuses, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here. Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this invention.

[0169] In the embodiments provided by this invention, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. Units with the same function may be grouped into one unit. For example, multiple units or components may be combined or integrated into another device, or some features may be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or units, or may be electrical, mechanical, or other forms of connection.

[0170] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of the embodiments of the present invention, depending on actual needs.

[0171] Furthermore, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0172] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a storage medium. Based on this understanding, the technical solution of this invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, a backend server, or a network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), magnetic disks, or optical disks.

[0173] The above are merely specific embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A barcode decoding method based on deep learning, characterized in that, include: A set of barcode images is obtained, and each barcode image in the set is preprocessed to obtain a sample image corresponding to each barcode image, which is then used to form a sample image set. The location information of the barcode in each sample image in the sample image set is labeled to obtain the first true location information of each sample image; In addition, the position information and type of each line and space in each sample image are labeled to obtain multiple second true position information of each sample image. The first true position information and multiple second true position information of each sample image are combined to form a label information set. A barcode prediction set is formed using the sample image set and the annotation information set, and the barcode prediction set is split into a training set and a test set. The pre-built deep learning model is trained based on the training set to obtain a barcode prediction model; The barcode prediction model is tested according to the test set and preset encoding rules. If the preset test conditions are met, training is stopped and the target barcode prediction model is obtained. The system receives a barcode image to be decoded, inputs the barcode image to be decoded into the target barcode prediction model, and obtains a first prediction box and multiple second prediction boxes; wherein, the first prediction box is used to indicate the predicted position of the barcode in the barcode image to be decoded, and the second prediction boxes are used to indicate the predicted position of each bar or empty space in the barcode image to be decoded. Based on the position information and bar / space type of the plurality of second prediction boxes, and the preset encoding rule, the decoded characters and the total number of decoded characters of the barcode image to be decoded are obtained, and based on the position information of the first prediction box and the preset encoding rule, the total number of characters of the barcode image to be decoded is obtained; If the total number of decoded characters in the barcode image to be decoded is equal to the total number of characters in the barcode image to be decoded, then the decoded characters of the barcode image to be decoded are output.

2. The deep learning-based barcode decoding method according to claim 1, characterized in that, The process of acquiring a set of barcode images and preprocessing each barcode image in the set to obtain sample images corresponding to each barcode image, thereby forming a sample image set, includes: The barcode image set is composed of multiple barcode images acquired by the image acquisition device. Each barcode image is subjected to noise reduction filtering to obtain each first intermediate image; Each first intermediate image is converted to grayscale to obtain each second intermediate image; Each second intermediate image is binarized to obtain each third intermediate image; each third intermediate image includes a barcode area and a background area. The barcode area of ​​each third intermediate image is cropped to a preset size to obtain each fourth intermediate image; Each fourth intermediate image is angled to obtain a sample image corresponding to each barcode image; Each sample image is combined to form the sample image set.

3. The deep learning-based barcode decoding method according to claim 1, characterized in that, The deep learning model includes a feature extraction module, a generator network, a first regression network, and a second regression network; the step of training the pre-constructed deep learning model based on the training set to obtain a barcode prediction model includes: Each sample image in the training set is input into the feature extraction module to obtain the feature map corresponding to each sample image; The generator network determines a first initial detection box in each feature map according to each first real location information in the training set, and determines multiple second initial detection boxes in each feature map according to each second real location information in the training set. The location information of each first initial detection box is obtained through the first regression network, the first error between the location information of each first initial detection box and the corresponding first true location information is calculated, and the network parameters of the first regression network are adjusted according to the first error. The location information of each second initial detection box is obtained through the second regression network, the second error between the location information of each second initial detection box and the corresponding second true location information is calculated, and the network parameters of the second regression network are adjusted according to the second error. If both the first error and the second error are less than the preset value, the network parameters of the first regression network and the network parameters of the second regression network will no longer be adjusted. The model obtained from the current training is determined as the barcode prediction model.

4. The deep learning-based barcode decoding method according to claim 1, characterized in that, The step of testing the barcode prediction model according to the test set and preset encoding rules, and stopping training if the preset test conditions are met to obtain the target barcode prediction model, includes: For each sample image in the test set, the sample image is input into the barcode prediction model to obtain multiple first candidate boxes corresponding to the first true location information of the sample image, and multiple second candidate boxes corresponding to each second true location information of the sample image. Calculate the first overlap between the position information of each first candidate box and the first real position information, and filter out the first overlap greater than the first preset value from the first overlap as the first candidate overlap. Calculate the second overlap between the position information of each second candidate box and the corresponding second real position information, and filter out the second overlap greater than the second preset value from the second overlap corresponding to each second candidate box as the second candidate overlap corresponding to each second candidate box; The largest first candidate overlap is selected from the first candidate overlap, and the first candidate box corresponding to the selected first candidate overlap is determined as the first prediction box of the sample image. The largest second candidate overlap is selected from the second candidate overlap corresponding to each second candidate box, and each second candidate box corresponding to the selected second candidate overlap is determined as multiple second prediction boxes of the sample image. Based on the position information and bar / space type of the multiple second prediction boxes in the sample image, and the preset encoding rule, the corresponding decoded characters and the total number of decoded characters are obtained, and based on the position information of the first prediction box in the sample image and the preset encoding rule, the corresponding total number of characters is obtained; If the total number of decoded characters corresponding to each sample image in the test set is equal to the total number of characters, then training stops, and the target barcode prediction model is obtained.

5. The deep learning-based barcode decoding method according to claim 4, characterized in that, The step of obtaining the corresponding decoded characters and the total number of decoded characters based on the position information and bar / space type of multiple second prediction boxes in the sample image and the preset encoding rule, and obtaining the corresponding total number of characters based on the position information of the first prediction box in the sample image and the preset encoding rule, includes: Based on the position information of each second prediction box in the multiple second prediction boxes of the sample image, the width information of each second prediction box is obtained; Based on the width information of each second prediction box, the bar / space type of each second prediction box, and the preset encoding rule, the width information is converted into the corresponding decoded character, and the number of the decoded characters is obtained as the total number of the corresponding decoded characters; The width information of the first prediction box is obtained based on the position information of the first prediction box in the sample image; The total number of characters is calculated based on the width information of the first prediction box and the preset encoding rules.

6. The deep learning-based barcode decoding method according to claim 4, characterized in that, The method further includes: In the test set, when the number of sample images whose total number of corresponding decoded characters is not equal to the total number of corresponding characters reaches a preset number, a supplementary test set is constructed using all sample images whose total number of corresponding decoded characters is not equal to the total number of corresponding characters, and a new training set and a new test set are obtained. The new training set is used to train the currently obtained barcode prediction model, and the new test set is used to test the trained new barcode prediction model. When the total number of decoded characters corresponding to each sample image in the new test set is equal to the total number of characters, the new barcode prediction model is supplemented and tested using the supplementary test set. When the total number of decoded characters corresponding to each sample image in the supplementary test set is equal to the total number of characters, training stops, and the target barcode prediction model is obtained.

7. The deep learning-based barcode decoding method according to claim 1, characterized in that, After obtaining the decoded characters and the total number of decoded characters of the barcode image to be decoded based on the position information and bar / space type of the plurality of second prediction boxes and the preset encoding rule, and obtaining the total number of characters of the barcode image to be decoded based on the position information of the first prediction box and the preset encoding rule, the method further includes: If the total number of characters to be decoded in the barcode image is not equal to the total number of characters in the barcode image, a barcode decoding failure message will be output.

8. A barcode decoding device based on deep learning, characterized in that, include: The sample image acquisition unit is used to acquire a set of barcode images and perform image preprocessing on each barcode image in the set of barcode images to obtain sample images corresponding to each barcode image, so as to form a sample image set. The information annotation unit is used to annotate the position information of the barcode of each sample image in the sample image set to obtain the first true position information of each sample image. In addition, the position information and type of each line and space in each sample image are labeled to obtain multiple second true position information of each sample image. The first true position information and multiple second true position information of each sample image are combined to form a label information set. The prediction set splitting unit is used to form a barcode prediction set using the sample image set and the annotation information set, and to split the barcode prediction set into a training set and a test set. The model training unit is used to train a pre-built deep learning model based on the training set to obtain a barcode prediction model. The model testing unit is used to test the barcode prediction model according to the test set and preset encoding rules. If the preset test conditions are met, the training is stopped and the target barcode prediction model is obtained. A barcode receiving unit is used to receive a barcode image to be decoded, input the barcode image to be decoded into the target barcode prediction model, and obtain a first prediction box and a plurality of second prediction boxes; wherein, the first prediction box is used to indicate the predicted position of the barcode in the barcode image to be decoded, and the second prediction box is used to indicate the predicted position of each bar or empty space in the barcode image to be decoded. The barcode decoding unit is used to obtain the decoded characters and the total number of decoded characters of the barcode image to be decoded based on the position information and bar / space type of the plurality of second prediction boxes and the preset encoding rules, and to obtain the total number of characters of the barcode image to be decoded based on the position information of the first prediction box and the preset encoding rules. The decoding character output unit is used to output the decoded characters of the barcode image to be decoded if the total number of decoded characters of the barcode image to be decoded is equal to the total number of characters of the barcode image to be decoded.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the deep learning-based barcode decoding method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, causes the processor to perform the deep learning-based barcode decoding method as described in any one of claims 1 to 7.

Images