Barcode decoding method and device

Abstract

The embodiment of the invention provides a bar code decoding method and device. In the embodiment of the invention, under the condition that decoding of the bar code in the image to be identified fails, the bar code text box can be decoded based on the position information of the bar code box in the image to be identified and the mapping relation between the bar code box and the corresponding bar code text box in the image similar to the image to be identified. Determining a region of interest containing a bar code textbox in the image to be recognized; performing text recognition on a region of interest containing a bar code textbox in the to-be-recognized image to obtain a text recognition result; and determining a barcode decoding result of the to-be-recognized image based on the text recognition result.

Description

Technical Field

[0001] This application relates to the field of deep learning technology, and in particular, to a barcode decoding method and device. Background Art

[0002] At present, barcode localization and decoding technologies have been widely applied in many fields. Traditional barcode decoding methods usually rely on image processing technologies, locate the barcode area through morphological operations or deep learning, etc., and extract the barcode content using specific decoding algorithms. Such barcode decoding methods often require high image quality and the barcode in the image needs to be in a clear and recognizable state. However, in practical applications, due to factors such as shooting angle, illumination, occlusion, barcode damage (deformation or blur), etc., the barcode in the image may have low quality, resulting in decoding errors or even decoding failures.

[0003] In this case, related technologies usually need to rely on manual assistance, such as manually assisting in the decoding process, thus consuming a large amount of labor and time costs, and then reducing the barcode decoding efficiency.

[0004] Therefore, how to improve the barcode decoding efficiency has become a difficult challenge in the barcode recognition process. Summary of the Invention

[0005] Multiple aspects of this application provide a barcode decoding method and device for improving barcode decoding efficiency.

[0006] An embodiment of this application provides a barcode decoding method, including: In the case where barcode decoding in the image to be recognized fails, based on the position information of the barcode frame in the image to be recognized and the mapping relationship between the barcode frame and the corresponding barcode text frame in the same type of image as the image to be recognized, determine the region of interest containing the barcode text frame in the image to be recognized; Perform text recognition on the region of interest containing the barcode text frame in the image to be recognized to obtain the text recognition result; Based on the text recognition result, determine the barcode decoding result of the image to be recognized.

[0007] An embodiment of this application also provides a barcode decoding device, including: A determination module, configured to, in the case where barcode decoding in the image to be recognized fails, based on the position information of the barcode frame in the image to be recognized and the mapping relationship between the barcode frame and the corresponding barcode text frame in the same type of image as the image to be recognized, determine the region of interest containing the barcode text frame in the image to be recognized; A text recognition module, configured to perform text recognition on a region of interest including a barcode text box in the image to be recognized, so as to obtain the text recognition result; The determination module is further configured to determine a barcode decoding result of the image to be recognized based on the text recognition result.

[0008] An embodiment of the present application further provides an electronic device, including: a processor, a memory, and a bus. The memory stores machine-readable instructions executable by the processor. The processor communicates with the memory through the bus. When the machine-readable instructions are executed by the processor, the steps in the barcode decoding method as described above are performed.

[0009] An embodiment of the present application further provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, the processor is caused to implement the steps in the barcode decoding method provided by the embodiment of the present application.

[0010] In the barcode decoding method provided by the embodiment of the present application, since the barcode frame usually prints the content actually included in the barcode in text form near the barcode, even if the barcode decoding in the image to be recognized fails, the region of interest including the barcode text box in the image to be recognized can be determined according to the position information of the barcode frame in the image to be recognized and the mapping relationship between the barcode frame and the corresponding barcode text box in the same type of image as the image to be recognized. Therefore, based on the text recognition result of the region of interest including the barcode text box in the image to be recognized, its barcode decoding result is determined, avoiding manual-assisted decoding and effectively improving the barcode decoding efficiency in the case of barcode decoding failure. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The schematic embodiments and descriptions thereof of the present application are used to explain the present application and do not constitute an improper limitation to the present application. In the drawings: Figure 1 is a schematic flowchart of a barcode decoding method provided by an exemplary embodiment of the present application; Figure 2 is a schematic diagram of importing an image to be recognized into barcode decoding software in an exemplary embodiment of the present application; FIG. 3(a) is a schematic diagram of segmenting the image in the barcode frame of the image to be recognized to obtain separate bars and spaces; FIG. 3(b) is a schematic diagram of a barcode segmentation map obtained by segmenting the image in the barcode frame of the image to be recognized; Figure 4 is a schematic diagram of establishing a mapping relationship between a barcode frame and a barcode text box in an image to implement RoI following of the barcode text box in an exemplary embodiment of the present application; Figure 5 Schematic diagram for adjusting the text line direction in the region of interest containing the barcode text box in the image to be recognized to the preset reading direction provided by the exemplary embodiment of the present application; Figure 6 Schematic diagram of the string output by the text recognition model based on the input text image provided by the exemplary embodiment of the present application; Figure 7 Schematic diagram of the process of applying the barcode decoding method provided by the exemplary embodiment of the present application to the actual scenario; FIG. 8(a) is a schematic diagram of the overall flow of the barcode decoding method provided by the exemplary embodiment of the present application; FIG. 8(b) is a schematic diagram of the implementation process of decoding the image by text recognition based on RoI following in the barcode decoding method provided by the exemplary embodiment of the present application; Figure 9 Schematic diagram of the structure of a barcode decoding device provided by the exemplary embodiment of the present application; Figure 10 Schematic diagram of the structure of an electronic device provided by the exemplary embodiment of the present application. Detailed implementation manners

[0012] To make the objectives, technical solutions and advantages of the present application clearer, the technical solutions of the present application will be clearly and completely described below in conjunction with the specific embodiments of the present application and the corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

[0013] First, the noun terms involved in one or more embodiments of the present application are explained.

[0014] Region of interest: The full English name is region of interest, and the English abbreviation is ROI. In machine vision and image processing, the area that needs to be processed is outlined from the processed image by means of a rectangle, circle, ellipse, irregular polygon, etc., which is called the region of interest, abbreviated as ROI. These regions usually contain key information and are used for tasks such as image analysis, object detection, image segmentation, and object tracking. Various operators (such as sliding window, selective search, region proposal network, etc.) and functions (for example) are commonly used in machine vision software such as Halcon, OpenCV, and Matlab to obtain the region of interest ROI, corresponding to the possible object positions in the image. These ROIs will then be input into subsequent convolutional neural networks or other machine learning models to further determine whether the ROI contains objects of a specific category, as well as the precise positions and boundaries of the objects.

[0015] A mask, with the English name "mask", uses a selected image, graphic or object to block (entirely or partially) the image being processed, thereby controlling the area or process of image processing. The specific image or object used for covering is called a mask or template. A mask is a binary image composed of 0s and 1s. When a mask is applied in a certain function, the area with 1 values is processed, and the masked area with 0 values is not included in the calculation. An image mask is defined by specified data values, data ranges, finite or infinite values, regions of interest, and annotation files. It is also possible to use any combination of the above options as input to create a mask. A mask is a template for an image filter. Remote sensing images are often processed using masks. When extracting roads, rivers, or buildings, an N*N matrix is used to filter the pixels of the image, and then the features or signs we need are highlighted. This matrix is a type of mask.

[0016] Image segmentation: Image segmentation is the technology and process of dividing an image into several specific regions with unique properties and extracting the objects of interest. It is a key step from image processing to image analysis. Existing image segmentation methods are mainly classified into the following categories: threshold-based segmentation methods, region-based segmentation methods, edge-based segmentation methods, and segmentation methods based on specific theories, etc. From a mathematical perspective, image segmentation is the process of dividing a digital image into non-overlapping regions. The process of image segmentation is also a labeling process, that is, pixels belonging to the same region are assigned the same number.

[0017] Semantic segmentation: Semantic segmentation is classification at the pixel level. Pixels belonging to the same class should be grouped into one class. Therefore, semantic segmentation understands the image from the pixel level. For example, in the following photo, pixels belonging to people should be grouped into one class, pixels belonging to motorcycles should be grouped into one class, and in addition, background pixels are also grouped into one class.

[0018] As described in the background art, when a traditional barcode decoding system encounters a situation where it cannot directly decode, it usually requires manual assistance, which will consume a large amount of labor and time costs. In this case, in relevant industrial barcode printing scenarios, the content actually contained in the barcode is printed near the barcode in text form, which provides an idea for barcode decoding in the form of text recognition. Moreover, the quality requirements for images in text recognition are lower than those for direct barcode decoding, which is more conducive to barcode recognition in low-image-quality scenarios.

[0019] Based on this, embodiments of the present application propose a barcode decoding method and apparatus to improve the barcode decoding efficiency. Since the barcode frame usually prints the content actually contained in the barcode in text form near the barcode, even if the barcode decoding in the image to be recognized fails, the region of interest containing the barcode text box in the image to be recognized can be determined based on the position information of the barcode frame in the image to be recognized and the mapping relationship between the barcode frame and the corresponding barcode text box in images of the same type as the image to be recognized. Then, based on the text recognition result of the region of interest containing the barcode text box in the image to be recognized, the barcode decoding result can be determined, avoiding the need for manual-assisted decoding and effectively improving the barcode decoding efficiency in the case of barcode decoding failure.

[0020] The following will detail the technical solutions provided by each embodiment of the present application in conjunction with the accompanying drawings.

[0021] Figure 1 It is a flowchart of a barcode decoding method provided for an exemplary embodiment of the present application. As Figure 1 shown, the method includes: Step 110, in the case where the barcode decoding in the image to be recognized fails, based on the position information of the barcode frame in the image to be recognized and the mapping relationship between the barcode frame and the corresponding barcode text box in images of the same type as the image to be recognized, determine the region of interest containing the barcode text box in the image to be recognized.

[0022] After obtaining the image to be recognized, the barcode in the image to be recognized can first be decoded using a conventional decoding method. Specifically, the decoding process may include: S1, obtain the image to be recognized containing the barcode in a conventional industrial printing scenario and import the image to be recognized into barcode decoding software. As Figure 2 shown, it is a schematic diagram of importing the image to be recognized into barcode decoding software in an exemplary embodiment of the present application. The image to be recognized includes a barcode and the corresponding barcode text content.

[0023] Among them, the image to be recognized imported into the barcode decoding software may contain multiple or single barcodes and their corresponding barcode text content to ensure that the image quality meets the requirements of subsequent image processing.

[0024] S2, after preprocessing the image to be recognized, a barcode positioning strategy can be used to position the barcode in the image to be recognized and obtain the barcode frame in the image to be recognized. Specifically, this step S2 may include S21 and S22.

[0025] S21. Perform preprocessing operations on the image to be recognized. The preprocessing operations may include roughly delimiting the RoI region in the image to be recognized, and performing one or more operations among adjusting the contrast of the image to be recognized, noise elimination, and grayscale conversion. These operations are aimed at removing irrelevant background information in the image to be recognized, thereby significantly improving the ease of barcode detection in the image to be recognized and further ensuring the accuracy and reliability of barcode features in the image to be recognized.

[0026] S22. For the image to be recognized after the preprocessing operations, first apply an edge detection or contour detection algorithm to find the possible barcode regions in the image to be recognized, and then use geometric analysis and positioning algorithms to determine the specific positions of the barcodes in the image to be recognized. Among them, edge detection is a technique in image processing and computer vision, which is used to identify the points with obvious brightness changes in a digital image, so as to extract the important structural information of the image. Contour detection refers to the process of ignoring the influence of the texture and noise interference inside the background and the target in a digital image containing the target and the background, and using certain techniques and methods to achieve the extraction of the target contour.

[0027] S3. Use an image segmentation algorithm to segment the barcode frame in the image to be recognized into individual bars and spaces, and convert them into corresponding numbers and characters according to the coding rules, that is, perform barcode decoding. Specifically, this step S3 may include S31 and S32.

[0028] S31. Use an image segmentation algorithm to segment the image inside the barcode frame in the image to be recognized to obtain individual bars and spaces, preparing for subsequent decoding. FIG. 3 is a schematic diagram of image segmentation of the barcode frame in the image to be recognized provided by an exemplary embodiment of the present application. Among them, (a) in FIG. 3 is a schematic diagram of segmenting the image inside the barcode frame in the image to be recognized to obtain individual bars and spaces. 300 is used to indicate a barcode, 301 is an individual space in the barcode, and 302 is an individual bar in the barcode. (b) in FIG. 3 is a schematic diagram of the barcode segmentation map obtained by segmenting the image inside the barcode frame in the image to be recognized. The white in the square is the bar of the barcode, and the black strip between two adjacent white bars in the square is the space. Exemplarily, the embodiments of the present application can flexibly select appropriate image segmentation techniques according to specific requirements. Either traditional machine learning methods such as edge detection or threshold segmentation can be used, or lightweight neural network models such as U-Net can be deployed to ensure the accuracy and efficiency of image segmentation.

[0029] S32. After successfully identifying the width and spacing patterns of the barcodes in the image to be recognized and accurately determining the barcode types, according to the corresponding coding rules, the bars and blank units of the barcodes in the image to be recognized obtained by segmentation can be converted into corresponding numbers and characters. This conversion process ensures an accurate mapping from visual elements to data information, ultimately achieving the decoding of the barcode content in the image to be recognized, thereby obtaining the decoded data output, that is, the barcode decoding result in the image to be recognized.

[0030] S4. Verify the barcode decoding result in the image to be recognized. Exemplarily, the barcode decoding result in the image to be recognized can be verified from the following aspects: ① Determine whether the format of the barcode decoding result in the image to be recognized is correct. Specifically, it can be determined whether the length of the string in the barcode decoding result in the image to be recognized conforms to the preset string length, and whether there are illegal characters. For example, if a character appears in a pure digital barcode, it indicates that there are illegal characters in the barcode decoding result.

[0031] ② Determine whether the logic of the barcode decoding result in the image to be recognized is reasonable. Specifically, it can be determined whether there are invalid strings in the barcode decoding result in the image to be recognized, such as the string "000000" or garbled characters.

[0032] ③ Determine whether the check digits of the barcode decoding result in the image to be recognized match. Among them, one-dimensional barcodes themselves have check digits, and it can be determined whether there is an error in the barcode decoding result based on the matching situation of the check digits in the barcode decoding result.

[0033] S5. If the barcode decoding result in the image to be recognized passes the verification, directly output the barcode decoding result of the image to be recognized.

[0034] In some exemplary embodiments, since in industrial production, the barcode styles in a group of images are the same, when the barcode position changes, the embodiments of the present application can determine the position of the barcode text line corresponding to the position of the located barcode frame according to the position of the located barcode frame and the mapping relationship between the pre-constructed barcode and its corresponding barcode text line, so as to recognize the text in the barcode text line. Specifically, based on the position information of the barcode frame in the image to be recognized and the mapping relationship between the barcode frame and the corresponding barcode text frame in the images of the same type as the image to be recognized, determine the region of interest containing the barcode text frame in the image to be recognized, including: Based on the mapping relationship between the barcode frame and the corresponding barcode text frame in the images of the same type as the image to be recognized, determine the affine transformation matrix between the barcode frame and the corresponding barcode text frame in the images of the same type as the image to be recognized; Based on the position information of the barcode frame in the image to be recognized and the affine transformation matrix, determine the position information of the barcode text frame in the image to be recognized; Based on the position information of the barcode text box in the image to be recognized, determine the region of interest in the image to be recognized that contains the barcode text box.

[0035] In an industrial production environment, since the barcode styles in the same set of images usually remain consistent, the embodiments of the present application can effectively handle the changes in barcode positions based on this. Regardless of how the barcode specifically changes its position in the image, this method can accurately determine the location of the corresponding barcode text according to the located barcode position and perform efficient recognition. This function greatly improves the flexibility and adaptability of the automatic recognition system, providing reliable technical support for efficient operation on the industrial production line.

[0036] Specifically, the images of the same type as the image to be recognized are images with the same barcode format as the image to be recognized, that is, images with the same mapping relationship between the barcode frame and the barcode text box in the image. It should be understood that in this type of image with the same mapping relationship between the barcode frame and the barcode text box in the image, the affine transformation matrix between the barcode frame and the barcode text box in the image is also the same. Based on this, the embodiments of the present application can pre-determine the affine transformation matrix between the barcode frame and the barcode text box in this type of image according to the mapping relationship between the barcode frame and the barcode text box in this type of image with the same mapping relationship between the barcode frame and the barcode text box in the image. After establishing the mapping relationship between the barcode frame and the barcode text box in the image, RoI following of the barcode text box in the image to be recognized can be achieved. In this way, even if the barcode position in the image changes, the position of the barcode text box in the image can be determined based on the position of the located barcode frame in the image and the mapping relationship between the barcode frame and the barcode text box in the image.

[0037] In some exemplary embodiments, the position of the barcode text box in the image to be recognized can be determined by the positions of any three vertices of the barcode frame in the image to be recognized and the affine transformation matrix determined by the position of the barcode frame in the image and the mapping relationship between the barcode frame and the barcode text box in the image. Specifically, based on the position information of the barcode frame in the image to be recognized and the affine transformation matrix, determining the position information of the barcode text box in the image to be recognized includes: Based on the coordinates of the three vertices in the position information of the barcode frame in the image to be recognized and the affine transformation matrix, determine the coordinates of the corresponding three vertices in the barcode text box in the image to be recognized; Based on the coordinates of the corresponding three vertices in the barcode text box in the image to be recognized, determine the position information of the barcode text box in the image to be recognized.

[0038] Among them, the three vertices in the position information of the barcode frame in the image to be recognized can be the upper left corner, the upper right corner, and the lower left corner of the barcode frame in the image to be recognized. Correspondingly, the three vertices corresponding to the barcode text frame in the image to be recognized are also the upper left corner, the upper right corner, and the lower left corner of the barcode text frame in the image to be recognized.

[0039] Exemplarily, the barcode text rectangle corresponding to the barcode frame in the image to be recognized can be drawn according to the above step S2 in the embodiments of the present application, as the region of interest in the image to be recognized that includes the barcode text frame, and the RoI following function for this region of interest can be implemented based on this.

[0040] Among them, the RoI following function for the region of interest in the image to be recognized that includes the barcode text frame can be implemented based on the principle of image affine transformation. Through coordinate mapping technology, this function enables the region of interest in the image to be recognized that includes the barcode text frame to closely follow the barcode frame in the image to be recognized, thereby achieving an accurate coordinate following effect. This application ensures that in the continuous image processing process, the text content in the barcode text frame corresponding to the position of the barcode in the image to be recognized can be accurately located each time, improving the stability and accuracy of the recognition of the barcode text frame in the image to be recognized.

[0041] As Figure 4 shown, it is a schematic diagram for establishing a mapping relationship between the barcode frame and the barcode text frame in the image to achieve RoI following of the barcode text frame provided by the exemplary embodiment of the present application. In this specific implementation interface, first, the barcode frame in the image can be accurately drawn using the oblique rectangle tool. Next, by clicking the "Add Benchmark" button, the user maps and binds the barcode frame and the barcode text frame. This operation cleverly realizes the real-time RoI following of the barcode text frame based on the position of the barcode frame, ensuring the accuracy of the positioning of the barcode text frame during the recognition process. Specifically, after determining the position of the barcode frame, the position of the barcode text frame in the image can be determined according to the mapping relationship between the barcode frame and the barcode text frame, and this function can be called the "RoI following function".

[0042] Exemplarily, mapping and binding a barcode frame and a barcode text box can be achieved by means of an affine transformation. Among them, the barcode frame and the barcode text box are usually rectangles. Three vertices corresponding to the two boxes (such as the upper left corner, the upper right corner, and the lower left corner) can be selected respectively, and the mapping relationship between the two boxes can be obtained by calculating the affine transformation matrix of the three vertices. For any vertex of the barcode frame, the corresponding position of the barcode text box can be obtained through this mapping relationship. During the continuous image processing process, the position of the barcode frame may change. After automatically obtaining the position of the barcode frame, according to the vertex coordinates of the new barcode frame and the affine transformation matrix, the vertex coordinates of the corresponding barcode text box are calculated, so as to realize the RoI following of the region of interest containing the barcode text box in the image to be recognized.

[0043] Step 120: Perform text recognition on the region of interest in the image to be recognized that contains a barcode text box, and obtain a text recognition result.

[0044] Among them, before performing text recognition on the region of interest in the image to be recognized that contains a barcode text box, the region of interest in the image to be recognized that contains a barcode text box can be cropped to obtain a more accurate region of interest. Since the text image with RoI can be manually annotated, or obtained through the RoI region obtained by RoI following and then calculating the angle of the RoI box, or obtained through a powerful text recognition algorithm, etc. If it is manually annotated, then the coordinates of the RoI can be directly read according to the annotation file and cropped on the original image. If it is in other ways, it may be necessary to check and fine-tune the obtained RoI box to obtain a better RoI region, and then crop according to the coordinates to obtain a more accurate RoI image.

[0045] In some exemplary embodiments, the embodiment of the present application can also adjust the recognized text direction to make the corresponding text recognition result more in line with the user's text viewing habit, and avoid the problem that the recognized text appears in the opposite direction and the recognized result is not accurate enough. Specifically, performing text recognition on the region of interest in the image to be recognized that contains a barcode text box to obtain a text recognition result includes: Determine the text line direction in the region of interest in the image to be recognized that contains a barcode text box; In the case where the text line direction does not belong to the preset reading direction, adjust the text line direction in the region of interest in the image to be recognized that contains a barcode text box to the preset reading direction; Perform text recognition on the region of interest in the image to be recognized that contains a barcode text box and is adjusted to the preset reading direction, and obtain a text recognition result.

[0046] In some exemplary embodiments, the text line direction in the region of interest containing the barcode text box in the image to be recognized can be identified by a pre-trained text direction discrimination model. Specifically, determining the text line direction in the region of interest containing the barcode text box in the image to be recognized includes: Input the region of interest containing the barcode text box in the image to be recognized into the text direction discrimination model, and determine the text line direction in the region of interest containing the barcode text box in the image to be recognized through the text direction discrimination model; Wherein, the text direction discrimination model is trained based on multiple images of regions of interest containing barcode text boxes and corresponding region of interest labels and text line direction labels, and the text line direction labels include angle labels for indicating the text direction.

[0047] Exemplarily, the above text direction discrimination model can be trained in the following manner: First, collect a batch of text images with RoI labels, where the text images with RoI labels are images of regions of interest containing barcode text boxes and corresponding region of interest labels and text line direction labels, and the text line direction labels can be marked in the form of the angle between the text line direction and the reference coordinate axis. To further improve the performance of the model and optimize the efficiency of data processing, the embodiments of the present application effectively reduce the size and complexity of the input image by precisely cropping the RoI region, significantly reducing the consumption of computing resources, enabling the model to process data more quickly during the training and inference phases. That is, enabling the model to focus on the RoI region itself during the training and inference phases to enhance the recognition accuracy of the model, making it more focused on learning the features of the target object, thereby effectively reducing the interference of background noise in the image where the RoI region is located and significantly improving the overall accuracy of the target recognition and classification tasks. And the correctly written direction annotation is performed on the cropped RoI text image for training the text direction discrimination model.

[0048] Next, the embodiments of the present application can design a lightweight text direction discrimination network, and the text direction discrimination network can adopt industry-recognized lightweight network architectures such as VGG, ResNet, MobileNet, etc. to automatically extract image features. These architectures are optimized to reduce the complexity and computational burden of the model while maintaining a high recognition accuracy. At the end of the network structure, a fully connected layer is added, or a classifier (such as Softmax) is used to output the category prediction of the text line direction. Such a design aims to ensure that the network can efficiently identify and classify the writing direction of the text, providing strong support for the overall performance of the text recognition system.

[0049] Finally, the embodiment of the present application introduces a cross-entropy loss function into the text direction discrimination network for multi-class classification, that is, classifying the categories of different directions of the text line, selecting a suitable optimizer to iteratively optimize the network weights multiple times, and finally obtaining a text direction discrimination model with excellent performance, and deploying it into the barcode decoding software.

[0050] In an actual application scenario, when the region of interest containing the barcode text box in the image to be recognized is input into the text direction discrimination model, first, the text direction discrimination model will accurately crop the image to be recognized, focusing on the RoI region, that is, obtaining the region of interest containing the barcode text box in the image to be recognized. Subsequently, for the cropped region of interest, the text direction discrimination model will discriminate its text direction. This algorithm can accurately identify whether the text is arranged horizontally, vertically or obliquely by deeply analyzing the text features. As Figure 5 shown, it is a schematic diagram of adjusting the text line direction in the region of interest containing the barcode text box in the image to be recognized to the preset reading direction provided by the exemplary embodiment of the present application. After determining the text line direction in the region of interest, the barcode decoding software can use technologies such as image rotation or affine transformation to automatically adjust the text line direction in the region of interest to the correct reading direction, that is, the preset reading direction, thereby laying an accurate foundation for the subsequent processing flow and ensuring the accuracy of text recognition.

[0051] Step 130, based on the text recognition result, determine the barcode decoding result of the image to be recognized.

[0052] After obtaining the text recognition result, the text recognition result can be verified from aspects such as the number of characters and the character type in the text recognition result, so as to determine whether the number of characters, the character prefix, and the character suffix in the text recognition result meet the requirements of the preset format, and whether there are invalid recognition results such as empty output or invalid characters in the text recognition result.

[0053] In some exemplary embodiments, in order to ensure the integrity of the barcode text in the barcode text box in the determined region of interest, avoid the loss of barcode text features caused by boundary cutting during the determination process of the region of interest, and at the same time improve the accuracy and robustness of segmentation, so that the barcode recognition system better meets the requirements of the actual application scenario, the embodiment of the present application can verify the text recognition result, and when the verification is successful, expand the region of interest, and then perform text recognition on the expanded region of interest. Specifically, determining the barcode decoding result of the image to be recognized based on the text recognition result includes: Verifying the text recognition result based on the number of characters and the character type in the text recognition result; In the case where the verification of the text recognition result fails, expand the boundaries of each region of interest containing the barcode text box in the image to be recognized by a specified number of pixels according to a preset ratio, to obtain an expanded region of interest; Perform text recognition on the expanded region of interest to obtain a text recognition result; Based on the text recognition result, determine the barcode decoding result of the image to be recognized.

[0054] Exemplarily, based on the number of characters and the character types in the text recognition result, verify the text recognition result, and it can be determined whether the number of characters in the text recognition result is a preset quantity, whether there are invalid characters or garbled characters in the text recognition result, and / or whether the check digit in the text recognition result is correct, etc.

[0055] Exemplarily, expand the boundaries of each region of interest containing the barcode text box in the image to be recognized by a specified number of pixels according to a preset ratio, to obtain an expanded region of interest. For example, expand the boundaries of each region of interest containing the barcode text box in the image to be recognized by 0.1 times the corresponding boundary length in terms of pixels at a ratio of 1.1 times, to obtain an expanded region of interest.

[0056] It should be understood that in the case where the verification of the text recognition result fails, the process of expanding the boundaries of each region of interest containing the barcode text box in the image to be recognized by a specified number of pixels according to a preset ratio to obtain an expanded region of interest can set a specified number of repetitions until the number of repetitions is used up. If the text recognition result in the recognized expanded region of interest still cannot pass the verification, then decode and recognize, and transfer it to manual assistance for decoding.

[0057] In some exemplary embodiments, the embodiments of the present application can use a pre-trained text localization model to determine the text position in the expanded region of interest. Specifically, performing text recognition on the expanded region of interest to obtain a text recognition result includes: Input the expanded region of interest into the text localization model, and determine the segmentation mask corresponding to the expanded region of interest through the text localization model. The segmentation mask is used to indicate the text position in the expanded region of interest; wherein, the text localization model is trained based on multiple images and corresponding pixel-level text region annotations; Based on the text position in the expanded region of interest indicated by the segmentation mask, draw a text box in the expanded region of interest; Perform text recognition on the text box in the expanded region of interest to obtain a text recognition result.

[0058] Among them, text recognition of the region of interest containing the barcode text box in the image to be recognized, as well as text recognition of the text boxes in the extended region of interest can both be achieved through a text recognition model. The training process of the text recognition model may include: First, a large number of text images covering various fonts, sizes, styles, and languages can be collected and sorted out, and accurate text box annotations are provided for each image. Then, a deep learning model, such as object detection networks like Faster R-CNN, YOLO, SSD, etc., is used to construct an efficient text detection model. This model can accurately identify the text regions in the text images, extract the image features of text lines or each character in the text from the original images according to the detected text boundaries, and output the corresponding text content using the detection head. Finally, the text recognition model is trained by applying a loss function and a suitable optimization algorithm is selected for multiple iterations to optimize the weights of the network, and finally a character recognition model is trained. This trained text recognition model will be deployed into the barcode decoding software for actual application.

[0059] Exemplarily, in the actual application process, the rectified text image (i.e., the region of interest that adjusts the text line direction in the region of interest containing the barcode text box in the image to be recognized to the preset reading direction) can be passed as input to the character recognition model. This text recognition model will analyze the character features in the rectified text image and recognize the text content contained therein. Finally, the text recognition model will output the recognition result in the form of a string, presenting all the text information in the text image, such as Figure 6 shown in the schematic diagram of the string (i.e., the text recognition result) output by the text recognition model provided by the exemplary embodiment of the present application based on the input text image.

[0060] After text recognition of the region of interest containing the barcode text box in the image to be recognized, the text recognition result can be verified, such as whether the character length or character type in the text recognition result meets the requirements. If the text recognition result passes the verification, the text recognition result is output as the barcode decoding result.

[0061] If the text recognition result fails to pass the verification, the boundaries of the region of interest containing the barcode text box in the image to be recognized can be expanded by a specified number of pixels according to a preset ratio to obtain an extended region of interest, and the text recognition model is called to perform text recognition on the extended region of interest, and the steps of verifying the text recognition result and expanding the boundaries of the region of interest containing the barcode text box in the image to be recognized by a specified number of pixels according to a preset ratio to obtain an extended region of interest and calling the text recognition model to perform text recognition on the extended region of interest are repeatedly executed until the number of repetitions is used up or a text recognition result that passes the verification is obtained as the barcode decoding result.

[0062] Exemplarily, the training process of the text localization model includes: First, text images covering various fonts, sizes, styles, and languages can be widely collected, and each image can be accurately annotated with text regions at the pixel level. Next, a deep learning model based on segmentation, such as Mask R-CNN or U-Net, which can generate pixel-level text region predictions to obtain a fine segmentation mask, is used. Finally, a loss function suitable for the segmentation task, such as cross-entropy loss, is adopted to evaluate the difference between the predicted mask and the true annotation. By selecting an appropriate optimizer and performing multiple iterations, the network weights are continuously optimized, and finally, a text localization model is trained and deployed into the barcode decoding software.

[0063] Exemplarily, in the actual application process, first, the region of interest containing the barcode text box in the image to be recognized can be processed, that is, a certain proportion of expansion is performed on each of its boundaries. The specific operation is to expand the specified number of pixels along each boundary on the basis of the original region of interest. The purpose of this is to ensure the integrity of the barcode text box in the image to be recognized, avoid the loss of text features in the barcode text box caused by boundary cutting, and at the same time improve the accuracy and robustness of segmentation, making text recognition more adaptable to the requirements of the actual application scenario. Second, this expanded region of interest is used as the input and input into the trained text localization model. This text localization model has the ability to recognize the region where the text is located in the image after deep learning training. After the text localization model finishes processing, a segmentation mask will be output, and this mask can accurately mark the specific position of the text in the image. Subsequently, according to the position information of this segmentation mask, a circumscribed rectangle can be drawn, and this rectangle tightly encloses the text region represented by the segmentation mask of the expanded region of interest, thus forming a new region of interest. This new region of interest will more accurately locate the text box in the image to be recognized, facilitating further text recognition and processing.

[0064] Figure 7The process schematic diagram of the barcode decoding method provided by the exemplary embodiment of the present application when applied to an actual scenario. The specific implementation process is as follows: First, import the image to be recognized and add a barcode symbology for barcode localization and decoding. Specifically, the image to be recognized can be imported into the barcode decoding software, and information such as the type of barcode (e.g., Code128) and the number of barcodes can be selected. Then, run the software for barcode localization and direct decoding. This step can also be completed through the automatic recognition function. Then, draw the region of interest (RoI) that contains the barcode text box in the image to be recognized and add a reference. Specifically, the oblique rectangle tool can be selected to draw the RoI that contains the barcode text box in the image to be recognized. Click to add a reference to map and bind the RoI and the barcode box, so as to achieve RoI following of the position of the barcode box by the RoI. Finally, output the result. Specifically, the Optical Character Recognition (OCR) option can be selected, and clicking to run can obtain the result. As can be seen from Figure 7 this, the decoding result of the barcode and the result of OCR recognition can be obtained through this method. When the barcode in the image to be recognized cannot be directly decoded, the result of OCR recognition can be used as the decoding result of the barcode in the image to be recognized.

[0065] Figure 8(a) is the overall flowchart of the barcode decoding method provided by the exemplary embodiment of the present application. In Figure 8(a), first, barcode localization and barcode decoding are performed on the acquired image to obtain a decoding result. When it is determined that there is a decoding result, the decoding result is directly output. When it is determined that there is no decoding result, the image can be decoded based on the text recognition with RoI following to obtain a recognition result as the output result.

[0066] FIG. 8(b) is a schematic flowchart of implementing image decoding based on RoI following in the barcode decoding method provided by an exemplary embodiment of the present application. In FIG. 8(b), the implementation process of the method may include: First, import an image and locate the barcode in the image to determine the position of the barcode frame; then define the recognition RoI and enable RoI following, that is, establish a mapping relationship between the position of the barcode frame and the position of the barcode text frame in the image; next, crop the RoI region in the image that contains the barcode text frame determined based on the mapping relationship between the position of the barcode frame and the position of the barcode text frame in the image, and perform text direction identification on this RoI region. When it is determined that the text direction is not the preset reading direction, adjust it to the preset reading direction; then, input the RoI adjusted to the preset reading direction into the text recognition model for text recognition; finally, verify the recognition result to determine whether the recognition result passes the verification (that is, determine whether the recognition result is OK?), and when the verification passes, output the recognition result. If the recognition result fails to pass the verification, expand the existing recognition RoI, call the text localization algorithm to obtain the text localization RoI; then perform RoI cropping and text direction identification; finally, perform text recognition and verify the recognition result, and when the verification passes, output the recognition result. As can be seen from FIG. 8(a) and FIG. 8(b), through this method, the barcode decoding result and the text recognition result can be obtained. When the barcode cannot be directly decoded, the text recognition result is used as the barcode decoding result.

[0067] By using the barcode decoding method provided by the embodiment of the present application, since the barcode frame usually prints the content actually contained in the barcode in text form near the barcode, even in the case where the barcode decoding in the image to be recognized fails, it is possible to determine the region of interest containing the barcode text frame in the image to be recognized based on the position information of the barcode frame in the image to be recognized and the mapping relationship between the barcode frame and the corresponding barcode text frame in the image of the same type as the image to be recognized. Thus, based on the text recognition result of the region of interest containing the barcode text frame in the image to be recognized, its barcode decoding result is determined, avoiding the need for manual-assisted decoding and effectively improving the barcode decoding efficiency in the case of barcode decoding failure. [[ID=?]]

[0068] Figure 9 FIG. is a schematic structural diagram of a barcode decoding device 900 provided by an exemplary embodiment of the present application. As Figure 9 shown, the device 900 includes: a determination module 910 and a text recognition module 920, where: A determination module 910, configured to, when barcode decoding in a to-be-recognized image fails, determine a region of interest containing a barcode text box in the to-be-recognized image based on the position information of the barcode box in the to-be-recognized image and the mapping relationship between the barcode box and the corresponding barcode text box in images of the same type as the to-be-recognized image; A text recognition module 920, configured to perform text recognition on the region of interest containing the barcode text box in the to-be-recognized image to obtain the text recognition result; The determination module 910 is further configured to determine the barcode decoding result of the to-be-recognized image based on the text recognition result.

[0069] In the barcode decoding device provided in the embodiment of the present application, since the barcode box usually prints the content actually contained in the barcode in text form near the barcode, even when barcode decoding in the to-be-recognized image fails, it is possible to determine the region of interest containing the barcode text box in the to-be-recognized image based on the position information of the barcode box in the to-be-recognized image and the mapping relationship between the barcode box and the corresponding barcode text box in images of the same type as the to-be-recognized image. Thus, based on the text recognition result of the region of interest containing the barcode text box in the to-be-recognized image, the barcode decoding result can be determined, avoiding the need for manual-assisted decoding and effectively improving the barcode decoding efficiency in the case of barcode decoding failure.

[0070] Optionally, when the determination module 910 determines the region of interest containing the barcode text box in the to-be-recognized image based on the position information of the barcode box in the to-be-recognized image and the mapping relationship between the barcode box and the corresponding barcode text box in images of the same type as the to-be-recognized image, the specific operations are as follows: Determine an affine transformation matrix between the barcode box and the corresponding barcode text box in images of the same type as the to-be-recognized image based on the mapping relationship between the barcode box and the corresponding barcode text box in images of the same type as the to-be-recognized image; Determine the position information of the barcode text box in the to-be-recognized image based on the position information of the barcode box in the to-be-recognized image and the affine transformation matrix; Determine the region of interest containing the barcode text box in the to-be-recognized image based on the position information of the barcode text box in the to-be-recognized image.

[0071] Optionally, when the text recognition module 920 performs text recognition on the region of interest containing the barcode text box in the to-be-recognized image to obtain the text recognition result, the specific operations are as follows: Determine the text line direction in the region of interest containing the barcode text box in the to-be-recognized image; In the case where the text line direction does not belong to the preset reading direction, adjust the text line direction in the region of interest containing the barcode text box in the image to be recognized to the preset reading direction; Perform text recognition on the region of interest containing the barcode text box in the image to be recognized that has been adjusted to the preset reading direction to obtain the text recognition result.

[0072] Optionally, when the determination module 910 determines the barcode decoding result of the image to be recognized based on the text recognition result, it specifically is used for: Verify the text recognition result based on the number of characters and character types in the text recognition result; In the case where the verification of the text recognition result fails, expand each boundary of the region of interest containing the barcode text box in the image to be recognized by a specified number of pixels according to a preset ratio to obtain an expanded region of interest; Perform text recognition on the expanded region of interest to obtain the text recognition result; Determine the barcode decoding result of the image to be recognized based on the text recognition result.

[0073] Optionally, when the determination module 910 performs text recognition on the expanded region of interest to obtain the text recognition result, it specifically is used for: Input the expanded region of interest into a text localization model, and determine a segmentation mask corresponding to the expanded region of interest through the text localization model, where the segmentation mask is used to indicate the text position in the expanded region of interest; wherein, the text localization model is trained based on multiple images and corresponding pixel-level text region annotations; Draw a text box in the expanded region of interest based on the text position in the expanded region of interest indicated by the segmentation mask; Perform text recognition on the text box in the expanded region of interest to obtain the text recognition result.

[0074] Optionally, when the text recognition module 920 determines the text line direction in the region of interest containing the barcode text box in the image to be recognized, it specifically is used for: Input the region of interest containing the barcode text box in the image to be recognized into a text direction discrimination model, and determine the text line direction in the region of interest containing the barcode text box in the image to be recognized through the text direction discrimination model; Wherein, the text direction discrimination model is trained based on multiple images of regions of interest containing barcode text boxes and corresponding region of interest labels and text line direction labels, and the text line direction labels include angle labels for indicating the text direction.

[0075] Optionally, when the determining module 910 determines the position information of the barcode text box in the image to be recognized based on the position information of the barcode box in the image to be recognized and the affine transformation matrix, it specifically is used for: Based on the coordinates of three vertices in the position information of the barcode box in the image to be recognized and the affine transformation matrix, determining the coordinates of the corresponding three vertices in the barcode text box in the image to be recognized; Based on the coordinates of the corresponding three vertices in the barcode text box in the image to be recognized, determining the position information of the barcode text box in the image to be recognized.

[0076] The barcode decoding device 900 can implement Figure 1 ~ the method of the method embodiment in FIG. 8. For details, reference can be made to Figure 1 ~ the barcode decoding method shown in the embodiment of FIG. 8, which will not be elaborated here.

[0077] Figure 10 FIG. [not shown] is a schematic structural diagram of an electronic device provided by an exemplary embodiment of the present application. As Figure 10 shown, the device includes: a memory 101 and a processor 102.

[0078] The memory 101 is used for storing computer programs and can be configured to store various other data to support operations on the computing device. Examples of such data include instructions for any application program or method for operating on the computing device, contact data, phone book data, messages, images, videos, etc.

[0079] The processor 102 is coupled to the memory 101 and is used for executing the computer program in the memory 101 to: in the case where barcode decoding in the image to be recognized fails, based on the position information of the barcode box in the image to be recognized and the mapping relationship between the barcode box and the corresponding barcode text box in the same type of image as the image to be recognized, determining the region of interest containing the barcode text box in the image to be recognized; performing text recognition on the region of interest containing the barcode text box in the image to be recognized to obtain the text recognition result; and based on the text recognition result, determining the barcode decoding result of the image to be recognized.

[0080] The electronic device provided by the embodiment of the present application. Since the barcode frame usually prints the content actually contained in the barcode in text form near the barcode, even if the barcode decoding in the image to be recognized fails, the region of interest containing the barcode text box in the image to be recognized can be determined according to the position information of the barcode frame in the image to be recognized and the mapping relationship between the barcode frame and the corresponding barcode text box in the images of the same type as the image to be recognized. Thus, based on the text recognition result of the region of interest containing the barcode text box in the image to be recognized, its barcode decoding result can be determined, avoiding the need for manual-assisted decoding and effectively improving the barcode decoding efficiency in the case of barcode decoding failure.

[0081] Further, as Figure 10 shown, the electronic device further includes: other components such as a communication component 103, a display 104, a power supply component 105, an audio component 106, etc. Figure 10 Only some components are schematically shown, and it does not mean that the electronic device only includes Figure 10 the components shown. Additionally, according to the different implementation forms of the traffic playback device, Figure 10 the components within the dashed box are optional components, not essential components. For example, when the electronic device is implemented as a terminal device such as a smart phone, a tablet computer, or a desktop computer, it may include Figure 10 the components within the dashed box; when the electronic device is implemented as a server-side device such as a conventional server, a cloud server, a data center, or a server array, it may not include Figure 10 the components within the dashed box.

[0082] Correspondingly, the embodiment of the present application also provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, the processor is enabled to implement the steps in the above barcode decoding method embodiment.

[0083] The above Figure 10 The communication component is configured to facilitate communication between the device where the communication component is located and other devices in a wired or wireless manner. The device where the communication component is located can access a wireless network based on a communication standard, such as WiFi, 2G, or 3G, or a combination thereof. In an exemplary embodiment, the communication component receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component may further include a near field communication (NFC) module, radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, etc.

[0084] The above Figure 10The memory therein can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0085] The above-mentioned Figure 10 The display therein includes a screen, and the screen can include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen can be implemented as a touch screen to receive input signals from a user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The touch sensors can sense not only the boundaries of touch or swipe actions but also detect the duration and pressure associated with the touch or swipe operation.

[0086] The above-mentioned Figure 10 The power supply component therein provides power for various components of the device where the power supply component is located. The power supply component can include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the device where the power supply component is located.

[0087] The above-mentioned Figure 10 The audio component therein can be configured to output and / or input audio signals. For example, the audio component includes a microphone (MIC), and when the device where the audio component is located is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode, the microphone is configured to receive external audio signals. The received audio signals can be further stored in the memory or transmitted via the communication component. In some embodiments, the audio component further includes a speaker for outputting audio signals.

[0088] Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system, or a computer program product. Therefore, the present application can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0089] This application is described with reference to the flowcharts and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the present application. It should be understood that each flow and / or block in the flowchart and / or block diagram can be implemented by computer program instructions, and the combination of the flows and / or blocks in the flowchart and / or block diagram can also be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing devices generate means for implementing the functions specified in one process Figure 1 one process or multiple processes and / or blocks Figure 1 or means for implementing the functions specified in multiple blocks.

[0090] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory generate a manufactured article including instruction means, and the instruction means implements the functions specified in one process Figure 1 one process or multiple processes and / or blocks Figure 1 or means for implementing the functions specified in multiple blocks.

[0091] These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to generate a computer-implemented process, and thus the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in one process Figure 1 one process or multiple processes and / or blocks Figure 1 or means for implementing the functions specified in multiple blocks.

[0092] In a typical configuration, a computing device includes one or more processors (CPUs), an input / output interface, a network interface, and memory.

[0093] The memory may include non-permanent memory in the form of computer-readable media, random access memory (RAM), and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. The memory is an example of computer-readable media.

[0094] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can be implemented using any method or technology for information storage. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change RAM (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory computer-readable media such as modulated data signals and carrier waves.

[0095] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a ..." does not exclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.

[0096] The foregoing is merely an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application should all be included within the scope of the claims of the present application.

Claims

1. A barcode decoding method, characterized in that, Including: In the case where barcode decoding in the image to be recognized fails, based on the position information of the barcode frame in the image to be recognized and the mapping relationship between the barcode frame and the corresponding barcode text frame in images of the same type as the image to be recognized, determine the region of interest containing the barcode text frame in the image to be recognized; Perform text recognition on the region of interest containing the barcode text frame in the image to be recognized to obtain the text recognition result; Based on the text recognition result, determine the barcode decoding result of the image to be recognized.

2. The method according to claim 1, wherein Based on the position information of the barcode frame in the image to be recognized and the mapping relationship between the barcode frame and the corresponding barcode text frame in images of the same type as the image to be recognized, determining the region of interest containing the barcode text frame in the image to be recognized includes: Based on the mapping relationship between the barcode frame and the corresponding barcode text frame in images of the same type as the image to be recognized, determine the affine transformation matrix between the barcode frame and the corresponding barcode text frame in images of the same type as the image to be recognized; Based on the position information of the barcode frame in the image to be recognized and the affine transformation matrix, determine the position information of the barcode text frame in the image to be recognized; Based on the position information of the barcode text frame in the image to be recognized, determine the region of interest containing the barcode text frame in the image to be recognized.

3. The method according to claim 1 or 2, characterized in that, Performing text recognition on the region of interest containing the barcode text frame in the image to be recognized to obtain the text recognition result includes: Determine the text line direction in the region of interest containing the barcode text frame in the image to be recognized; In the case where the text line direction does not belong to the preset reading direction, adjust the text line direction in the region of interest containing the barcode text frame in the image to be recognized to the preset reading direction; Perform text recognition on the region of interest containing the barcode text frame in the image to be recognized that is adjusted to the preset reading direction to obtain the text recognition result.

4. The method according to claim 1, wherein Based on the text recognition result, determining the barcode decoding result of the image to be recognized includes: Based on the number of characters and character types in the text recognition result, verify the text recognition result; In the case where the verification of the text recognition result fails, expand each boundary of the region of interest containing the barcode text frame in the image to be recognized by a specified number of pixels according to a preset ratio to obtain an expanded region of interest; Perform text recognition on the expanded region of interest to obtain the text recognition result; Based on the text recognition result, determine the barcode decoding result of the image to be recognized.

5. The method according to claim 4, characterized in that, Performing text recognition on the expanded region of interest to obtain the text recognition result includes: Input the expanded region of interest into a text localization model, and determine the segmentation mask corresponding to the expanded region of interest through the text localization model, where the segmentation mask is used to indicate the text position in the expanded region of interest; wherein, the text localization model is trained based on multiple images and corresponding pixel-level text region annotations. Draw a text box in the extended region of interest based on the text positions in the extended region of interest indicated by the segmentation mask; Perform text recognition on the text box in the extended region of interest to obtain the text recognition result.

6. The method according to claim 3, wherein The determining the text line direction in the region of interest in the image to be recognized that contains a bar code text box includes: Input the region of interest in the image to be recognized that contains a bar code text box into a text direction discrimination model, and determine the text line direction in the region of interest in the image to be recognized that contains a bar code text box through the text direction discrimination model; Wherein, the text direction discrimination model is trained based on multiple images of regions of interest containing bar code text boxes and corresponding region of interest labels and text line direction labels, and the text line direction labels include angle labels for indicating the text direction.

7. The method according to claim 2, wherein The determining the position information of the bar code text box in the image to be recognized based on the position information of the bar code box in the image to be recognized and the affine transformation matrix includes: Based on the coordinates of three vertices in the position information of the bar code box in the image to be recognized and the affine transformation matrix, determine the coordinates of the corresponding three vertices in the bar code text box in the image to be recognized; Based on the coordinates of the corresponding three vertices in the bar code text box in the image to be recognized, determine the position information of the bar code text box in the image to be recognized.

8. A bar code decoding device, characterized in that, Includes: A determination module, configured to, when the bar code decoding in the image to be recognized fails, determine the region of interest in the image to be recognized that contains a bar code text box based on the position information of the bar code box in the image to be recognized and the mapping relationship between the bar code box and the corresponding bar code text box in images of the same type as the image to be recognized; A text recognition module, configured to perform text recognition on the region of interest in the image to be recognized that contains a bar code text box to obtain the text recognition result; The determination module is further configured to determine the bar code decoding result of the image to be recognized based on the text recognition result.

9. An electronic device, characterized in that, Includes: A processor, a memory, and a bus, where the memory stores machine-readable instructions executable by the processor, the processor communicates with the memory through the bus, and when the machine-readable instructions are executed by the processor, the steps in the method according to any one of claims 1 to 7 are performed.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, the processor is caused to implement the steps in the method according to any one of claims 1 to 7.

Images