Barcode positioning method based on deep learning and electronic equipment

Abstract

The invention relates to the technical field of bar code recognition, and discloses a bar code positioning method based on deep learning and electronic equipment, and the method comprises the steps: obtaining a bar code image, segmenting a bar code through employing an image segmentation model, generating a start character positioning map, a data region positioning map, and a termination character positioning map, determining upper and lower boundary detection starting points, a data area left boundary detection area and a data area right boundary detection area according to the positioning map, performing corrosion operation on the bar code image in the horizontal direction, traversing pixel points on the corrosion image from the upper and lower boundary detection starting points, and determining an upper boundary and a lower boundary of the bar code; and traversing pixel points in the data area left boundary detection area and the data area right boundary detection area on the barcode image, determining a left boundary and a right boundary of the data area, and determining positioning coordinate points of the data area in the barcode according to the boundaries. Through the above mode, the bar code positioning efficiency and accuracy are improved, and the decoding capability of the electronic equipment in a complex scene is improved.

Description

Technical Field

[0001] This application relates to the field of barcode recognition technology, specifically to a barcode positioning method and electronic device based on deep learning. Background Technology

[0002] A barcode is a graphical symbol system used to represent data, consisting of a series of bars and spaces of varying widths arranged according to specific encoding rules. Taking the PDF417 barcode as an example, it is a high-density, high-capacity two-dimensional stacked barcode with powerful information storage capabilities. It can encode text, images, and biometric data, and is widely used in various fields. Currently, PDF417 barcode recognition primarily employs steps such as image preprocessing, edge detection, contour extraction, and barcode region filtering. This involves locating the barcode region within the image using contour information, and then decoding the barcode region to retrieve the information stored within it.

[0003] However, existing technologies typically locate the start and stop symbols in a PDF417 barcode by detecting the presence of connected rectangles in the barcode image using contour information. Finally, the barcode region is located based on the start and stop symbols. When the start or stop symbol of a PDF417 barcode is missing (or incomplete), finding connected rectangles cannot accurately pinpoint their location. This leads to electronic devices failing to accurately recognize the barcode due to barcode region localization failure. Although deep learning algorithms such as image recognition and object recognition are used to find the start and stop symbols, these algorithms can only determine their approximate positions within the barcode and cannot precisely locate the boundaries of the start symbol, data area, and stop symbol. Summary of the Invention

[0004] In view of the above problems, this application provides a barcode positioning method and electronic device based on deep learning to solve the problem that electronic devices in the prior art cannot accurately identify barcodes due to barcode area positioning failure.

[0005] According to one aspect of the embodiments of this application, a deep learning-based barcode localization method is provided. The method includes: acquiring a barcode image and segmenting the barcode in the barcode image using a pre-trained image segmentation model to generate a start symbol localization map, a data area localization map, and a stop symbol localization map; determining the upper and lower boundary detection start points based on the start symbol localization map, the data area localization map, and the stop symbol localization map; performing a horizontal erosion operation on the barcode image to obtain an eroded image; starting from the upper and lower boundary detection start points, traversing the pixels of the eroded image upwards sequentially to determine the upper boundary of the barcode; starting from the upper and lower boundary detection start points, traversing the pixels of the eroded image downwards sequentially to determine the lower boundary of the barcode; determining the left boundary detection area and the right boundary detection area of ​​the data area based on the data area localization map; traversing the pixels located in the left boundary detection area of ​​the data area on the barcode image to determine the left boundary of the data area in the barcode; traversing the pixels located in the right boundary detection area of ​​the data area on the barcode image to determine the right boundary of the data area in the barcode; and determining the positioning coordinates of the data area in the barcode based on the upper boundary, the lower boundary, the left boundary of the data area, and the right boundary of the data area.

[0006] According to another aspect of the embodiments of this application, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the above-described deep learning-based barcode positioning method.

[0007] In this embodiment, on the one hand, the start symbol, data area, and stop symbol can be found at once through the image segmentation model. Therefore, when performing precise positioning of the barcode data area, only the area initially located by the image segmentation model needs to be calculated, effectively improving the efficiency and accuracy of barcode positioning. On the other hand, this embodiment achieves precise positioning of the barcode data area based on the upper and lower boundaries of the barcode and the left and right boundaries of the data area. Compared with positioning based on the start and stop symbols, this method can accurately locate the barcode data area even when the start or stop symbol is partially or completely missing, the barcode is too blurry, or there is too much background interference. This improves the decoding capability of electronic devices in complex scenarios.

[0008] The above description is merely an overview of the technical solutions of the embodiments of this application. In order to better understand the technical means of the embodiments of this application and to implement them in accordance with the contents of the specification, and to make the above and other objects, features and advantages of the embodiments of this application more obvious and understandable, specific implementation methods of this application are described below. Attached Figure Description

[0009] The accompanying drawings are for illustrative purposes only and are not intended to limit the scope of this application. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings: Figure 1A schematic diagram of the structure of the first type of barcode according to an embodiment of this application is shown; Figure 2 A flowchart illustrating the deep learning-based barcode localization method provided in an embodiment of this application is shown. Figure 3 A schematic diagram of the structure of the second type of barcode involved in an embodiment of this application is shown; Figure 4 A schematic diagram of the structure of the third type of barcode involved in the embodiments of this application is shown; Figure 5 This paper shows a schematic diagram of the structure of the start symbol positioning diagram provided in an embodiment of this application; Figure 6 This illustration shows a structural schematic diagram of the data area positioning map provided in an embodiment of this application; Figure 7 This paper shows a schematic diagram of the structure of the terminator positioning diagram provided in an embodiment of this application; Figure 8 A schematic diagram of the structure of the image segmentation model provided in an embodiment of this application is shown; Figure 9 A schematic diagram of the structure of the corrosion image provided in an embodiment of this application is shown; Figure 10 This illustration shows a structural diagram of a barcode image after its boundary has been determined, as provided in an embodiment of this application. Figure 11 This invention provides a schematic diagram of another data area location map according to an embodiment of the present application. Figure 12 This paper shows a schematic diagram of another terminator positioning diagram provided in an embodiment of this application; Figure 13 This illustration shows a structural schematic diagram of another corrosion image provided in an embodiment of this application; Figure 14 This illustration shows a structural diagram of another barcode image after boundary determination, provided in an embodiment of this application. Figure 15 A schematic diagram of the structure of an electronic device provided in an embodiment of this application is shown. Detailed Implementation

[0010] Exemplary embodiments of the present application will now be described in more detail with reference to the accompanying drawings. Although exemplary embodiments of the present application are shown in the drawings, it should be understood that the present application may be implemented in various forms and should not be limited to the embodiments set forth herein.

[0011] by Figure 1 Taking the barcode shown as an example, Figure 1The diagram illustrates the structure of a barcode. A barcode primarily includes a start symbol (area a in the diagram), data areas (areas b, c, and d in the diagram), and a stop symbol (area e in the diagram). The start and stop symbols mark the beginning and end of the barcode, helping the decoder determine its location. The data is mainly stored in the data area. Therefore, even if the start or stop symbol is missing (or incomplete), as long as the data area is intact, the electronic device can still decode the data area to obtain the information carried by the barcode.

[0012] However, current barcode recognition mainly locates the barcode region in the barcode image by identifying the positions of the start and stop symbols. Specifically, after the electronic device acquires the barcode image, it uses an edge detection algorithm to detect the edge information in the barcode image, generates an edge image, performs an erosion operation on the edge image to remove small edge noise, then performs a dilation operation on the edge image to enhance edge features, extracts all contour information from the processed edge image, and obtains connected rectangles in the edge image based on the contour information to filter out possible barcode regions. Finally, the positions of the start and stop symbols in the barcode image are located by judging the validity of the barcode region, thereby achieving the location of the barcode in the barcode image.

[0013] When the start or stop symbol in a barcode is missing (or incomplete), the electronic device cannot obtain the position of the start and stop symbols in the barcode image by recognizing connected rectangles in the edge image. As a result, the electronic device cannot locate the barcode area in the barcode image, and consequently, the electronic device cannot accurately decode the barcode in the barcode image, resulting in a low barcode decoding rate.

[0014] Based on this, in order to more accurately locate and identify barcodes, this application provides a deep learning-based barcode localization method. After the electronic device acquires the barcode image, it first uses a pre-trained image segmentation model to segment the barcode in the barcode image, obtaining the localization maps corresponding to the start symbol, data area, and stop symbol, respectively. Then, it detects the barcode image based on the localization maps to accurately find the upper and lower boundaries of the barcode and the left and right boundaries of the data area in the barcode image. Then, it determines the localization coordinates of the data area in the barcode image through these boundaries. Finally, it uses the localization coordinates of the data area for decoding to obtain the data information carried by the barcode in the barcode image.

[0015] In this approach, electronic devices can initially locate the start symbol, data area, and stop symbol from a barcode image using a pre-trained image segmentation model, and generate corresponding localization maps. Then, within the initially located areas of the localization maps, calculations are performed on the barcode image to accurately locate the upper and lower boundaries of the barcode and the left and right boundaries of the data area. Based on these boundaries, precise localization of the data area in the barcode image is achieved, allowing the electronic device to decode the barcode using the location coordinates of the data area. This accurate localization of the data area is possible regardless of whether the start and stop symbols are complete, thus improving the barcode decoding rate. Furthermore, this application employs a fusion of deep learning and traditional algorithms. First, deep learning algorithms are used to initially locate the start symbol, data area, and stop symbol, and then traditional algorithms are used to further refine the localization within the areas found by the deep learning algorithm. This approach demonstrates excellent robustness under complex backgrounds, uneven lighting, and conditions of excessive darkness or overexposure.

[0016] According to a first aspect of this application, a barcode localization method based on deep learning is provided, such as... Figure 2 As shown, Figure 1 A flowchart of a deep learning-based barcode localization method provided in an embodiment of this application is shown. This method is executed by an electronic device used to identify and decode barcodes. Specifically, the device can be a barcode scanner, mobile device (such as a smartphone, tablet, etc.), barcode reader, POS scanner, data collector, etc. The method includes the following steps: Step S110: Obtain the barcode image and use a pre-trained image segmentation model to segment the barcode in the barcode image to generate a start symbol location map, a data area location map, and a stop symbol location map.

[0017] A barcode image is an image containing a barcode. For example, a PDF417 barcode might look like this: Figure 1 The barcode includes five regions: a, b, c, d, and e. Region a is the start symbol, used to identify the beginning of the barcode. It usually consists of specific bars and spaces to help identify the start of the barcode. Region b is the left-hand indicator, used to provide information about the data rows in the barcode to help the electronic device determine the arrangement and reading order of the data rows. Region c is the data area of ​​the barcode, used to store the actual encoded data information, which can be text, images, or biometric data. This data area consists of multiple data symbol characters, each representing a certain data information. Region d is the right-hand indicator, similar to the left-hand indicator, used to provide information about the data rows in the barcode to help the electronic device determine the arrangement and reading order of the data rows. Region e is the stop symbol, used to identify the end of the barcode. It usually consists of specific bars and spaces to help identify the end of the barcode.

[0018] like Figure 1 As shown, since the left-row indicator, data area, and right-row indicator have the same structure, and decoding the barcode requires the left-row indicator, data area, and right-row indicator, in this embodiment, these three areas are uniformly classified as the data area. That is, all data areas mentioned below include... Figure 1 The three regions are b, c, and d.

[0019] In addition, barcode images can be like Figure 1 The image shown includes a complete barcode, or you can see it as follows: Figure 3 The examples shown include barcodes with missing start or stop characters, or such as... Figure 4 The barcode shown includes a missing start and / or stop character. Figure 3 and Figure 4 Two other schematic diagrams of barcode structures are shown.

[0020] Image segmentation models are used to segment barcodes in barcode images, dividing them into start symbol, data area, and stop symbol regions to locate the positions of these regions. Figure 4 Taking the barcode structure shown as an example, after the electronic device segments it using an image segmentation model, it can output something like... Figure 5 , Figure 6 and Figure 7 The location map shown indicates that the white area represents... Figure 4 The area where the barcode is located.

[0021] in, Figure 5 For the start symbol positioning diagram, Figure 5 The white area is Figure 4 The area containing the start character of the barcode; Figure 6 Location map of the data area. Figure 6 The white area is Figure 4 The area where the data area of ​​the barcode is located; Figure 7 For the terminator positioning diagram, Figure 7 The white area is Figure 4 The area where the terminator of the barcode is located.

[0022] Regarding the structure of the image segmentation model, this application provides one possible implementation method, such as... Figure 8 As shown, Figure 8The diagram illustrates the structure of an image segmentation model, which consists of an encoder and a decoder. The encoder's core objective is to extract multi-level features from the input training image and compress the spatial resolution through downsampling, while simultaneously improving the semantic abstraction of the features. The decoder's core objective is to restore the spatial resolution of the high-level semantic features compressed by the encoder and combine them with the high-resolution details of the encoder's first level to ultimately generate a pixel-level segmentation result of the same size as the input.

[0023] Among them, the SEModule is a channel attention-focused module. Its core idea is to learn the importance weight of each channel and dynamically adjust the response intensity of each channel in the feature map, thereby making the network focus more on key features useful for image segmentation tasks and suppressing redundant or irrelevant channel information. The MobileNetV1Block module is the core building block of the MobileNetV1 network. Its goal is to significantly reduce the number of parameters and computational cost of convolution operations while maintaining feature representation capabilities, making it particularly suitable for lightweight models.

[0024] Of course, other image segmentation models can also be used, including but not limited to U-Net, Mask R-CNN, and FCN.

[0025] Regarding the training process of the image segmentation model, this application provides an implementation method, such as... Figure 1 As shown, training images in the training dataset are labeled into three categories: category 1 represents start symbols, category 2 represents data regions, and category 3 represents stop symbols. The training images from the training dataset are then input into the image segmentation model to extract features from the training images. The model then outputs localization maps corresponding to the start symbols, data regions, and stop symbols. For example, the image segmentation model outputs a 3-channel localization map, where the first channel represents the start symbol segmentation result, i.e., the start symbol localization map (e.g., ...). Figure 5 As shown), the second channel is the data segmentation result, i.e., the data area location map (as shown). Figure 6 As shown), the third channel is the result of the terminator segmentation, i.e., the terminator location map (as shown). Figure 7 (As shown).

[0026] Finally, the localization map output by the image segmentation model is verified by using the annotation information on the training images to calculate the loss value of the loss function (such as cross-entropy loss, Dice loss, etc.) to measure the difference between the prediction result of the image segmentation model and the real label. Finally, based on the calculated loss value, the model parameters are updated by the backpropagation algorithm to reduce the loss. The above steps are repeated until the image segmentation model converges or reaches the preset training rounds, thus completing the training of the image segmentation model.

[0027] In addition, the start symbol location map, data area location map, and end symbol location map, besides... Figure 5 , Figure 6 and Figure 7 As shown, in addition to setting the pixels in the areas corresponding to the start symbol, data area, and end symbol in the figure to white and the pixels in other areas to black, you can also set the pixels in the areas corresponding to the start symbol, data area, and end symbol in the figure to black and the pixels in other areas to white.

[0028] Step S120: Determine the starting point for upper and lower boundary detection based on the start symbol positioning map, data area positioning map, and end symbol positioning map.

[0029] The upper and lower boundary detection start point is the position where the electronic device begins traversing the barcode image when detecting the upper and lower boundaries of the barcode. The upper and lower boundary detection start point can be the middle position of the barcode in the image, for example, traversing... Figure 5 , Figure 6 or Figure 7 Find any column of white pixels and calculate the average value of the vertical coordinates of the white pixels. Use this average value as the starting point for the upper and lower boundary detection.

[0030] Of course, the start symbol positioning map, data area positioning map, and end symbol positioning map can all be detected to determine the middle position of the barcode in the image, and this middle position can be determined as the starting point for the upper and lower boundary detection. Specifically, step S120 may include the following steps (steps S121 to S122): Step S121: Traverse the start symbol region in the start symbol positioning map, the data region in the data region positioning map, and the end symbol region in the end symbol positioning map, and calculate the average value of the ordinate of all pixels.

[0031] Step S122: Determine the pixels in the erosion image whose vertical coordinate is equal to the average vertical coordinate as the starting point for upper and lower boundary detection.

[0032] by Figure 5 , Figure 6 and Figure 7 For example, an electronic device can analyze the pixel values ​​of each pixel in the start symbol positioning map, data area positioning map, and stop symbol positioning map. The positions with pixel values ​​greater than 0 are the start symbol region in the start symbol positioning map, the data region in the data area positioning map, and the stop symbol region in the stop symbol positioning map. By calculating the average of the ordinates of all pixels in the start symbol region, data region, and stop symbol region, the center position of the barcode in the barcode image can be determined. Specifically, the upper and lower boundary detection starting points can be calculated using the following formula:

[0033] in, This represents the ordinate of all pixels in the start symbol region, data region, and end symbol region. Figure 5 , Figure 6 and Figure 7 The ordinate of the white pixel in the middle, where N is the number of pixels in the start region, data region, and end region. Figure 5 , Figure 6 and Figure 7 The number of white pixels.

[0034] By going through steps S121 to S122, the middle position of the barcode in the barcode image is determined according to the start symbol area in the start symbol positioning map, the data area in the data area positioning map, and the end symbol area in the end symbol positioning map. Then, starting from the middle position of the barcode, the pixels in the barcode image are traversed upwards or downwards, and the position where the bars and spaces in the barcode image are connected is found, so that the upper and lower boundaries of the barcode can be accurately found.

[0035] After step S120, step S130 is executed: horizontal erosion is performed on the barcode image to obtain an eroded image.

[0036] Among them, such as Figure 1 As shown, the data area of ​​a barcode consists of 1 to 30 data symbol characters, arranged from left to right and top to bottom, with no gaps between them. Each data symbol character consists of 4 bars and 4 spaces, for a total of 17 modules (bars and spaces each occupy half). The width of each bar or space can be 1 to 6 module widths. The bars and spaces of the data symbol characters are arranged according to specific encoding rules to represent different data information, and the width of the bars and spaces of each data symbol character varies depending on the encoded data information.

[0037] Therefore, the data area of ​​a barcode is formed by multiple layers of bars and empty stacks. If you directly traverse the pixels in the barcode image and determine whether a pixel is a pixel at the edge of the barcode based on whether the pixel value changes, then the pixels in the middle of the data area where multiple layers of bars and empty stacks are located will also be considered pixels at the edge of the barcode. This will affect the positioning of the upper and lower boundaries of the barcode.

[0038] To ensure the accuracy of upper and lower boundary detection, a horizontal erosion operation can be performed on the barcode image before detecting the upper and lower boundaries to eliminate empty areas within the barcode region. Specifically, the horizontal erosion of the barcode image can be performed using the following formula:

[0039] Where A is the barcode image, and B represents the structuring element (also known as the kernel or template), used to define the method of erosion operation. This represents the resulting image of the erosion operation, i.e., the eroded image, where z is a pixel position. This indicates that the structuring element B is translated to position Z.

[0040] by Figure 4 Taking the barcode structure shown as an example, for Figure 4 After performing a horizontal etching operation, the product can be generated. Figure 9 The eroded image shown in the figure has the upper and lower boundaries of the barcode in it. Figure 4 The upper and lower boundaries of the barcode image shown are basically coincident, while the spaces inside the barcode in the eroded image are eroded and eliminated. This can effectively avoid mistaking the position where the bars and spaces inside the barcode are connected as the edge position of the barcode image, and ensure the accuracy of upper and lower boundary recognition.

[0041] It should be noted that steps S120 and S130 do not have a fixed execution order. In addition to the example shown in this application, where step S120 is executed first to determine the starting point of the upper and lower boundary detection, and then step S130 is executed to perform horizontal erosion operation on the barcode image, step S130 can also be executed first to perform horizontal erosion operation on the barcode image, and then step S120 is executed to determine the starting point of the upper and lower boundary detection.

[0042] Step S140: Starting from the upper and lower boundary detection starting point, traverse the pixels of the eroded image upwards to determine the upper boundary of the barcode.

[0043] The process begins by detecting the upper and lower boundaries, then sequentially traverses the pixels of the eroded image upwards, determining the upper boundary of the barcode based on changes in pixel values. Figure 9 Taking the eroded image shown as an example, assuming the origin of the coordinate system is located at the top left corner of the image and the image is located in the first quadrant of the coordinate system, the vertical coordinate of the barcode's upper boundary position is determined as the target vertical coordinate, taking the pixel located at the upper boundary position as the boundary. In the eroded image, among the two pixels adjacent to the pixel at the upper boundary position in the Y-axis direction, the pixel with a vertical coordinate less than the target vertical coordinate is grayish-white, and the pixel with a vertical coordinate greater than the target vertical coordinate is blackish-gray. Furthermore, the pixel value corresponding to the grayish-white pixel is larger than the pixel value corresponding to the blackish-gray pixel.

[0044] Electronic devices can determine the pixels at the edge of a barcode based on the pixel differences between pixels in the vertical direction. Then, a straight line is fitted based on these edge pixels. Specifically, algorithms such as least squares, robust regression, and principal component analysis can be used to fit the straight line to the edge pixels. The fitted line is the upper boundary of the barcode. Figure 10 Line L1 in the middle, Figure 10 A schematic diagram of the structure of a barcode image after its boundaries have been determined is shown.

[0045] Furthermore, when detecting the upper boundary, the electronic device can traverse all pixels in the etched image, or it can use the start symbol positioning map, data area positioning map, and end symbol positioning map to initially determine the barcode position. When traversing the etched image, it only traverses the barcode area initially determined by the start symbol positioning map, data area positioning map, and end symbol positioning map to speed up the determination of the upper boundary. Specifically, step S140 may include the following steps (steps S141 to S145): Step S141: Based on the coordinates of the currently traversed pixel, obtain the first pixel value of the pixel at the same coordinate on the start symbol positioning map, data area positioning map, and end symbol positioning map respectively.

[0046] Step S142: If there is a first pixel value within the preset threshold range, then obtain the second pixel value of the two pixels adjacent to the currently traversed pixel in the vertical direction in the eroded image.

[0047] If any of the first pixel values ​​obtained from the start symbol positioning map, data area positioning map, and end symbol positioning map are within the preset threshold range, it indicates that the currently traversed pixel is a pixel within the barcode area. It is necessary to detect whether the pixel is located at the edge of the barcode and execute step S142. If the first pixel value obtained from the start symbol positioning map, data area positioning map, and end symbol positioning map is not within the preset threshold range, then the currently traversed pixel is a pixel outside the barcode area. It can be determined that the pixel cannot be a pixel at the edge of the barcode, and the pixel can be left undetected to speed up the image traversal.

[0048] Step S143: If the difference between the second pixel values ​​is greater than the first preset threshold, then the currently traversed pixel is determined as the upper boundary point.

[0049] Step S144: Within the first preset angle range, calculate the second straight-line voting matrix based on the upper boundary point.

[0050] Step S145: Determine the line with the highest number of votes in the second line voting matrix as the upper boundary of the barcode.

[0051] Specifically, the start symbol location map, data area location map, and end symbol location map are used as masks. Within the area covered by the masks in the eroded image, starting from the upper and lower boundary detection points, each pixel in the eroded image is traversed, and the pixel difference between each pixel is calculated. Finally, the pixel points at the barcode edge positions are determined based on the pixel differences. Figure 5 , Figure 6 , Figure 7 and Figure 9 Taking the barcode structure shown as an example, the pixel positions at the barcode edge are determined using the following formula:

[0052] Where I represents Figure 9 The erosion image shown, and M represents the second pixel value of the two pixels vertically adjacent to the currently traversed pixel in the eroded image. Figure 5 The starting symbol positioning diagram shown Figure 6 The data area location map shown and Figure 7 The mask shown, which serves as the terminating symbol location map, indicates that if the pixel value (x, y) of any pixel in the starting symbol location map, data area location map, or terminating symbol location map is greater than 0, then... If the pixel value of the (x, y) point in the start symbol positioning map, data area positioning map, and end symbol positioning map is all equal to 0, then it indicates... , The difference between the second pixel values ​​is the value between the two pixels. If the difference between the second pixel values ​​is greater than the first preset threshold, then the currently traversed pixel point, i.e. the pixel point (x, y) in the eroded image, is determined as the upper boundary point.

[0053] It should be noted that the preset threshold range can be set based on the pixel values ​​of the barcode areas in the start symbol positioning map, data area positioning map, and end symbol positioning map. In the above formula, since the pixel value of the barcode areas in the start symbol positioning map, data area positioning map, and end symbol positioning map is 1, the preset threshold range can be set from 0 to positive infinity, as long as the first pixel value exists. A value greater than 0 indicates that the currently traversed pixel is located within the barcode area. If the pixel values ​​of the barcode area in the start symbol positioning map, data area positioning map, and end symbol positioning map are set to other data, the preset threshold range can also be adjusted according to the actual pixel values ​​of the barcode area.

[0054] Similarly, the setting of the first preset threshold can also be determined based on the pixel value of each pixel in the eroded image. If the eroded image is a binary image, the first preset threshold can be set to 0. If the difference between the second pixel values ​​is greater than 0, the currently traversed pixel is determined as the upper boundary point. If the eroded image is a grayscale image, the first preset threshold can be set to a slightly larger value, for example, the first preset threshold can be set to 20, 30, etc.

[0055] After traversing the eroded image, the upper boundary of the barcode can be determined based on the obtained upper boundary points. Specifically, algorithms such as Hough line detection, LSD line detection, FLD line detection, and CannyLines can be used. A second line voting matrix is ​​calculated based on the upper boundary points, and the line with the highest voting score is determined as the upper boundary of the barcode. Specifically, the second line voting matrix can be calculated using the following formula:

[0056] Where x and y are the coordinate values ​​corresponding to each upper boundary point.

[0057] The first preset angle range can be set to 0 to 180 degrees. If the barcode image has been rotated to the desired angle before determining the upper and lower boundaries of the barcode, then... Figure 1 The positive direction shown can also be set to the first preset angle range of 0 to 10 degrees and 170 to 180 degrees to speed up the determination of the upper boundary and reduce the amount of calculation.

[0058] Through steps S141 to S142, the barcode area is initially determined based on the start symbol positioning map, data area positioning map, and end area positioning map. Then, the difference between pixels located within the barcode area on the eroded image is calculated, and the upper boundary point of the barcode is found based on the difference. Finally, the straight line corresponding to the upper boundary is determined based on the upper boundary point, thus accurately obtaining the upper boundary of the barcode. This eliminates the need to calculate the difference between all pixels in the eroded image, effectively reducing the computational load and improving the speed of upper boundary determination. Furthermore, the barcode area is initially located using the start symbol positioning map, data area positioning map, and end area positioning map, ensuring the accuracy of upper boundary detection.

[0059] After step S140, step S150 is executed: starting from the upper and lower boundary detection starting point, the pixels of the eroded image are traversed downwards in sequence to determine the lower boundary of the barcode.

[0060] The process begins by detecting the upper and lower boundaries, then sequentially traverses the pixels of the eroded image downwards, determining the lower boundary of the barcode based on changes in pixel values. Figure 9 Taking the eroded image shown as an example, assuming the origin of the coordinate system is located at the top left corner of the image and the image is located in the first quadrant of the coordinate system, the vertical coordinate of the lower boundary of the barcode is determined as the target vertical coordinate, taking the pixel at the lower boundary of the barcode as the boundary. In the eroded image, among the two pixels adjacent to the pixel at the lower boundary in the direction shown by the Y-axis, the pixel with a vertical coordinate less than the target vertical coordinate is blackish-gray, and the pixel with a vertical coordinate greater than the target vertical coordinate is grayish-white. Furthermore, the pixel value corresponding to the blackish-gray is smaller than the pixel value corresponding to the grayish-white.

[0061] Electronic devices can determine the pixels at the edge of a barcode based on the pixel differences between pixels in the vertical direction. Then, a straight line is fitted based on these edge pixels. Specifically, algorithms such as least squares, robust regression, and principal component analysis can be used to fit the straight line to the edge pixels. The fitted line is the lower boundary of the barcode. Figure 10 Line L2 in the middle.

[0062] In addition, when detecting the lower boundary, the electronic device can traverse all pixels in the eroded image, or it can use the start symbol positioning map, data area positioning map, and end symbol positioning map to preliminarily determine the position of the barcode. When traversing the eroded image, it can only traverse the barcode area preliminarily determined by the start symbol positioning map, data area positioning map, and end symbol positioning map to speed up the determination of the upper boundary.

[0063] The process of determining the lower boundary in step S150 is similar to that of determining the upper boundary in step S140. Therefore, step S150 can also be performed in the same way as steps S141 to S145 to determine the lower boundary of the barcode. It only requires that the difference between the second pixel values ​​be calculated based on the characteristics of the lower boundary of the barcode. The calculation formula was adjusted to... Figure 5 , Figure 6 , Figure 7 and Figure 9 Taking the barcode structure shown as an example, the difference between the second pixel values ​​can be used to... The calculation formula is adjusted as follows:

[0064] Step S160: Determine the left boundary detection area and the right boundary detection area of ​​the data area based on the data area positioning map.

[0065] Among them, such as Figure 6 Taking the data area positioning map shown as an example, the white area in the map is the data area of ​​the barcode in the barcode image. The position where the white area and the black area in the map are connected can be initially determined as the place where the data area in the barcode image is connected to other areas. It is only necessary to determine the detection area of ​​the left and right boundaries based on the position where the white area and the black area are connected in the data area positioning map, and then perform detailed detection on the corresponding pixels in the barcode image based on the detection area to accurately obtain the left and right boundaries of the barcode.

[0066] Electronic devices can Figure 6 The pixel values ​​of any row of pixels within the white area are detected to find the horizontal coordinates of the left and right edges of the white area. Based on these horizontal coordinates, the left and right boundary detection areas of the data area are determined. As an example, assume... Figure 6 The x-coordinate of the left edge of the white area is Then the range of the detection area at the left boundary of the data area can be determined as follows: The horizontal axis in the barcode image is The pixels within the range are those located within the detection area on the left boundary of the data area.

[0067] Of course, electronic devices can also be used. Figure 6 The horizontal coordinates of all pixels located at the left and right edges of the white area are used to determine the left and right boundary detection areas of the data area. Specifically, step S160 may include the following steps (steps S161 to S163): Step S161: Traverse all pixels in the data area positioning map, calculate the pixel difference between two pixels adjacent to the currently traversed pixel in the horizontal direction, and determine the left edge point and right edge point in the data area positioning map based on the pixel difference.

[0068] Step S162: Calculate the average abscissa of the left edge points, and determine the left boundary detection area of ​​the data area based on the average abscissa of the left edge points.

[0069] Step S163: Calculate the average abscissa of the right edge points, and determine the right boundary detection area of ​​the data area based on the average abscissa of the right edge points.

[0070] This involves traversing all pixels in the data area location map and calculating the pixel difference between two adjacent pixels in the horizontal direction. Specifically, as shown... Figure 6 As shown, when the area containing the barcode is white, the pixels at the left edge of the data area... Figure 9 The positive X-axis changes from black to white, while the pixels at the right edge of the data area change along... Figure 9 The positive X-axis changes from white to black, so the difference between pixels can be used to determine whether the currently traversed pixel is a left edge or a right edge.

[0071] As an example, a threshold can be set if the pixel difference... If the value is greater than this threshold, then the currently traversed pixel will be... If the pixel difference is determined to be the left edge point in the data area location map, then... If the value is less than this threshold, then the currently traversed pixel will be... It was identified as the right edge point in the data area location map. For Figure 6 For the binary image shown, the threshold can be set to 0.

[0072] Next, calculate the average x-coordinate of the left and right edge points respectively, which can be done using the following formula:

[0073] in, This represents the average x-coordinate of either the left or right edge point. The x-coordinate represents the left or right edge point, and N represents the number of left or right edge points.

[0074] Finally, the left boundary detection area of ​​the data area is determined based on the average x-coordinate of the left edge points, and the right boundary detection area is determined based on the average x-coordinate of the right edge points. Specifically, a range value 'step' can be set based on experience, and then the interval range can be further defined. It is determined to be either the left boundary detection area or the right boundary detection area of ​​the data area.

[0075] Through steps S161 to S163, the left and right boundary detection areas of the data area are determined by the average of the abscissas of the left and right edge points, so as to ensure the accuracy of the left and right boundary detection areas of the data area, thereby ensuring the accuracy of the left and right boundary detection structure of the data area.

[0076] After step S160, step S170 is executed: traverse the pixels located in the left boundary detection area of ​​the data area on the barcode image to determine the left boundary of the data area in the barcode.

[0077] Among them, such as Figure 1 As shown, assume the barcode in the barcode image is black, and the origin of the coordinate system is located at the top left corner of the image (the specific coordinate system is as follows). Figure 9 As shown in the figure, the x-coordinate of the left boundary of the data area is determined as the target x-coordinate, with the pixel located at the left boundary of the data area as the boundary. In the barcode image, among the two pixels adjacent to the pixel at the left boundary of the data area in the direction shown by the X-axis, the pixel with the x-coordinate less than the target x-coordinate is white, and the pixel with the x-coordinate greater than the target x-coordinate is black.

[0078] Electronic devices can determine the pixels at the left edge of the data area based on the pixel differences between pixels in the horizontal direction within the detection region of the left boundary of the data area. Then, a straight line is fitted based on the pixels at the edge position. Specifically, algorithms such as least squares, robust regression, and principal component analysis can be used to fit the straight line to the pixels at the edge position. The fitted straight line is the left boundary of the data area. Figure 10 Line L3 in the middle.

[0079] As an example, the left boundary of the data area can be determined through the following steps (steps S171 to S173): Step S171: If the pixel difference between two pixels adjacent to the currently traversed pixel in the horizontal direction in the barcode image is greater than the second preset threshold, then the currently traversed pixel is determined as the left boundary point of the data area.

[0080] Step S172: Within the second preset angle range, calculate the third straight-line voting matrix based on the left boundary point of the data area.

[0081] Step S173: Determine the line with the highest number of votes in the third line voting matrix as the left boundary of the data area in the barcode.

[0082] Specifically, assuming the origin of the coordinate system is at the top left corner of the image and the image is located in the first quadrant of the coordinate system, the electronic device can calculate the pixel difference using the following formula:

[0083] Where K represents the barcode image, x and y are the coordinates of the currently visited pixel, and x satisfies , This represents the detection region at the left boundary of the data area, where y satisfies... , The ordinate of the lower boundary obtained in step S150 and The ordinate of the upper boundary obtained in step S140 is y-coordinate.

[0084] If pixel difference If the value is greater than the second preset threshold, it indicates a pixel. The pixel located at the left edge of the data area can be designated as the left boundary point of the data area. The second preset threshold can also be determined based on the pixel values ​​of each pixel in the barcode image. If the barcode image is a binary image, the second preset threshold can be set to 0. If the pixel difference between two pixels horizontally adjacent to the currently traversed pixel in the barcode image is greater than 0, then the currently traversed pixel is designated as the left boundary point of the data area. If the barcode image is a grayscale image, the second preset threshold can be set slightly larger, for example, to 20 or 30.

[0085] After traversing all pixels within the left boundary detection area of ​​the data area in the barcode image, the left boundary of the data area in the barcode can be determined based on the obtained left boundary points. Specifically, the Hough line detection algorithm can be used to calculate the third line voting matrix based on the left boundary points, and the line with the highest voting score is determined as the left boundary of the barcode. The second preset angle range can be set to 0 to 180 degrees. If the barcode image has been rotated to such an angle before determining the barcode boundary... Figure 1As shown in the positive direction, the second preset angle range can also be set to 80 to 100 degrees to speed up the determination of the left boundary and reduce the amount of calculation.

[0086] Through steps S171 to S173, the pixels located in the left boundary detection area of ​​the data area in the barcode image are traversed to find the left boundary point of the data area. Then, the straight line corresponding to the left boundary of the data area is accurately found through the left boundary point, providing an accurate data basis for the subsequent positioning of the data area.

[0087] After step S170, step S180 is executed: traverse the pixels located in the right boundary detection area of ​​the data area on the barcode image to determine the right boundary of the data area in the barcode.

[0088] Among them, such as Figure 1 As shown, the structure of the left boundary of the data area in the barcode is the same as the structure of the right boundary. The specific process of obtaining the right boundary of the data area in the barcode by traversing the pixels of the right boundary detection area is also the same as the specific process of obtaining the left boundary of the data area in step S170, and will not be repeated here. The right boundary of the data area in the barcode is... Figure 10 Line L4 in the middle.

[0089] Step S190: Determine the positioning coordinates of the data area in the barcode based on the upper boundary, lower boundary, left boundary of the data area, and right boundary of the data area.

[0090] Among them, such as Figure 10 As shown, after adding the upper boundary (i.e., line L1), the lower boundary (i.e., line L2), the left boundary of the data area (i.e., line L3), and the right boundary of the data area (i.e., line L4) to the barcode image, it can be found that the area enclosed by lines L1, L2, L3, and L4 is the precise location of the data area in the barcode.

[0091] Therefore, electronic devices can obtain the location coordinates of the data area in the barcode by calculating the intersection points P1 and P2 between line L1 and lines L3 and L4, and the intersection points P3 and P4 between line L2 and lines L3 and L4. Specifically, taking the intersection point P1 between lines L1 and L3 as an example, the x-coordinate and y-coordinate of intersection point P1 can be calculated using the following formula: ; ; Among them, coordinate points and coordinate points Let L1 be two points with coordinates. and coordinate points Let P2, P3, and P4 be two points on line L3. The coordinates of these points can be calculated using the two formulas described above.

[0092] Furthermore, this application also provides an example of a case where the start or stop character is completely missing; please refer to [link to example]. Figure 3 , Figure 11 , Figure 12 , Figure 13 and Figure 14 , Figure 11 It shows Figure 3 The data area location map corresponding to the barcode image shown. Figure 12 It shows Figure 3 The image shown is a map showing the location of the terminator corresponding to the barcode image. Figure 13 It shows Figure 3 The erosion image corresponding to the barcode image shown. Figure 14 It shows Figure 3 The structure of the barcode image after its boundaries have been determined.

[0093] like Figure 3 As shown, the barcode image contains a barcode with a missing start symbol. First, the electronic device segments the data area and the stop symbol in the barcode image using a pre-trained image segmentation model, and can then generate a barcode like this. Figure 11 The data area location map shown and Figure 12 The diagram showing the location of the terminator is used to... Figure 3 The data area and terminator of the barcode in the barcode image shown are initially located.

[0094] Next, the electronic device can... Figure 11 The data area location map shown and Figure 12 The system iterates through the pixels on the terminal marker localization map shown, determines the upper and lower boundary detection start points based on the pixel values ​​of each pixel, and performs... Figure 3 The barcode image shown undergoes a horizontal erosion operation to generate... Figure 13 The erosion image is shown. Then, starting from the detection start point at the upper and lower boundaries, the erosion is... Figure 13 The pixels of the eroded image shown are traversed to determine the upper boundary of the barcode (i.e., Figure 14 The line L1 in the middle) and the lower boundary (i.e. Figure 14 (Line L2 in the middle).

[0095] Then, the electronic device according to Figure 11 The data area location map shown initially locates the area, determines the left boundary detection area and the right boundary detection area of ​​the data area, and then performs separate operations on... Figure 3 The pixels located in the left and right boundary detection areas of the data area on the barcode image shown are traversed to determine the left boundary of the data area in the barcode (i.e., Figure 14 The straight line L3 in the middle) and the right boundary of the data area (i.e. Figure 14 (Line L4 in the middle).

[0096] Finally, the electronic device determines the boundaries based on the upper boundary, lower boundary, left boundary of the data area, and right boundary of the data area. Figure 3 The location coordinates of the barcode data area. Specifically, such as... Figure 14 As shown, the electronic device can obtain the positioning coordinates P1, P2, P3 and P4 of the data area in the barcode by calculating the intersection points P1 and P2 between line L1 and line L3 and line L4, and the intersection points P3 and P4 between line L2 and line L3 and line L4.

[0097] In the above embodiments, a pre-trained image segmentation model is used to initially locate the start symbol, data area, and end symbol in the barcode image, and generate a start symbol location map, a data area location map, and an end symbol location map. Then, the image after horizontal erosion is traversed within the initially located area to accurately obtain the upper and lower boundaries of the barcode in the barcode image. Then, the detection areas of the left and right boundaries are determined based on the data area location map. Then, the pixels located within the detection areas of the left and right boundaries in the barcode image are traversed to accurately find the left and right boundaries of the barcode data area in the barcode image. Finally, the data area in the barcode is accurately located based on the upper and lower boundaries of the barcode and the left and right boundaries of the barcode data area.

[0098] On the one hand, image segmentation models can locate the start symbol, data area, and stop symbol all at once. Therefore, when precisely locating the data area of ​​a barcode, only the area initially located by the image segmentation model needs to be calculated, effectively improving the efficiency and accuracy of barcode positioning. On the other hand, the embodiments of this application achieve precise positioning of the barcode data area based on the upper and lower boundaries of the barcode and the left and right boundaries of the data area. Compared with positioning based on start and stop symbols, this method can accurately locate the data area of ​​the barcode even when the start or stop symbol is partially or completely missing, the barcode is too blurry, or there is too much background interference. This improves the decoding capability of electronic devices in complex scenarios.

[0099] Moreover, this application adopts a fusion of deep learning and traditional algorithms. First, the start symbol, data area and end symbol are initially found through deep learning algorithm, and then the traditional algorithm is used to further accurately locate the region found by the deep learning algorithm. It has good robustness in complex background, uneven lighting, dark and overexposed conditions.

[0100] Furthermore, to facilitate barcode positioning, before determining the upper and lower boundaries of the barcode, it is necessary to detect the angle and orientation of the barcode in the barcode image, and adjust the barcode image based on the detection results, adjusting the barcode image to, as shown... Figure 1The positive direction is shown. Specifically, before step S120, the method further includes: Step S210: Detect the start symbol positioning map, data area positioning map, and stop symbol positioning map to obtain the edge points of the start symbol, data area, and stop symbol in the barcode.

[0101] Among them, with Figure 5 , Figure 6 and Figure 7 Taking the structure shown as an example, the position where the black area and the white area are connected is the edge point of the start symbol, data area and end symbol in the barcode. Therefore, electronic devices can determine the edge point of the start symbol, data area and end symbol in the barcode by traversing the pixel values ​​of each pixel in the start symbol positioning map, data area positioning map and end symbol positioning map and based on the pixel difference between each pixel.

[0102] Specifically, for each pixel in the start symbol positioning map, data area positioning map, and end symbol positioning map, the pixel difference in the horizontal and vertical directions is calculated separately, using the following formulas: ; ; in, Represents pixels in an image The pixel value, only the calculation result in the horizontal direction. Calculation results in the vertical direction If there is a result greater than 0, then the pixel is... Save them as edge points to the set Vecl.

[0103] Step S220: Calculate the first straight line voting matrix based on the coordinates of the edge points, and determine the angle of the straight line with the highest number of votes in the first straight line voting matrix as the angle of the barcode.

[0104] The electronic device can use algorithms such as Hough line detection, LSD line detection, FLD line detection, and CannyLines to calculate the first line voting matrix based on the coordinates of the edge points. The specific formula is as follows:

[0105] Where t is the angle, ranging from 0 to 180 degrees, N is the total number of elements in the set Vecl, S is the first line voting matrix, and the angle corresponding to the highest voting score in S is the angle of the barcode in the barcode image.

[0106] Step S230: Rotate the barcode image, start symbol positioning image, data area positioning image, and stop symbol positioning image according to the angle of the barcode.

[0107] Specifically, based on the barcode angle calculated in step S230, the barcode image, start symbol positioning image, data area positioning image, and end symbol positioning image are rotated respectively to straighten the tilted barcode.

[0108] Step S240: Determine whether the barcode is reversed based on the start symbol positioning map and the end symbol positioning map.

[0109] Among them, such as Figure 5 , Figure 6 and Figure 7 As shown, since the area where the barcode is located in the start symbol positioning map, data area positioning map, and end symbol positioning map is uniformly white, and other areas are uniformly black, the angle of the barcode calculated by the edge points can only represent the angle of the barcode tilt in the barcode image, and cannot determine whether the barcode in the barcode image is reversed.

[0110] Therefore, after aligning the barcode image, start symbol positioning map, data area positioning map, and stop symbol positioning map according to the barcode's angle, further detection is needed to determine whether the barcode in the barcode image is facing forward or backward. For example... Figure 1 As shown, when the barcode is oriented forward, the start symbol is located on the left side of the data area, and the stop symbol is located on the right side. Assuming the origin is located at the top left corner of the image and the barcode image is in the first quadrant of the coordinate system, when the barcode is oriented forward, the x-coordinates of the pixels on the start symbol will be smaller than the x-coordinates of the pixels on the stop symbol. Conversely, when the barcode is oriented backward, the x-coordinates of the pixels on the start symbol will be larger than the x-coordinates of the pixels on the stop symbol. Therefore, electronic devices can determine the barcode orientation based on the start and stop symbol positions located in the start and stop symbol positioning maps.

[0111] As an example, step S240 may specifically include the following steps (steps S241 to S246): Step S241: Calculate the average horizontal coordinate of each pixel in the start symbol region of the barcode image based on the start symbol positioning map to obtain the first average value.

[0112] The electronic device can traverse all pixels on the start symbol positioning map and determine whether a traversed pixel is within the start symbol region of the barcode image based on its pixel value. Figure 5 Taking the structure shown as an example, if the pixel value of a pixel is equal to 1, it means that the pixel is a pixel in the start symbol area of ​​the barcode image. Record the horizontal coordinate of the pixel, and after traversing all the pixels on the start symbol positioning map, calculate the average horizontal coordinate of each pixel in the start symbol area of ​​the barcode image.

[0113] Step S242: Calculate the average horizontal coordinate of each pixel in the terminator region of the barcode image based on the terminator positioning map to obtain the second average value.

[0114] The electronic device can traverse all pixels on the terminator positioning map and determine whether a traversed pixel is within the terminator region of the barcode image based on its pixel value. Figure 7 Taking the structure shown as an example, if the pixel value of a pixel is equal to 1, it means that the pixel is a pixel in the terminator region of the barcode image. Record the horizontal coordinate of the pixel, and after traversing all the pixels on the terminator positioning map, calculate the average horizontal coordinate of each pixel in the terminator region of the barcode image.

[0115] Step S243: If both the first average value and the second average value are not equal to zero, and the first average value is greater than the second average value, then the barcode is determined to be reversed.

[0116] If both the first and second average values ​​are not equal to zero, it indicates that both the start and stop symbols of the barcode are present in the barcode image. The electronic device can directly determine whether the barcode is reversed by comparing the magnitude of the first and second average values. Specifically, when the first average value is greater than the second average value, that is, when the horizontal coordinates of each pixel on the start symbol are greater than the horizontal coordinates of each pixel on the stop symbol, the electronic device can determine that the barcode in the barcode image is reversed.

[0117] Step S244: If the first average value or the second average value is equal to zero, obtain the image width of the barcode image and calculate the position threshold based on the image width.

[0118] If the first average value is zero, it indicates that the start symbol of the barcode is missing in the barcode image. If the second average value is zero, it indicates that the end symbol of the barcode is missing in the barcode image. In this case, the electronic device cannot directly compare the magnitudes of the first and second average values ​​to determine whether the barcode is reversed.

[0119] like Figure 3 As shown, when the barcode is facing forward, even if the start character is missing, the stop character will be located on the right side of the image; similarly, even if the stop character is missing, the start character will be located on the left side of the image. Therefore, the electronic device can determine the positions of the start and stop characters on the image based on the first and second average values, and determine whether the barcode is reversed based on these positions.

[0120] Specifically, the image width W of the barcode image can be obtained, and a position threshold can be calculated based on the image width. For example, the position threshold can be set to... This is used to determine the positions of the start and stop symbols on the barcode image by using the position threshold.

[0121] Step S245: If the first average value is equal to zero and the second average value is greater than zero and less than the position threshold, then the barcode is determined to be reversed.

[0122] Step S246: If the second average value is equal to zero and the first average value is greater than the position threshold, then the barcode is determined to be reversed.

[0123] If the start symbol of the barcode is missing in the barcode image, a second average value is used to determine whether the stop symbol is located on the left or right side of the barcode image, thus determining whether the barcode is reversed. When the second average value is greater than zero and less than a position threshold, the stop symbol is determined to be on the left side of the barcode image, and the electronic device can then determine that the barcode is reversed.

[0124] If the terminator of the barcode is missing in the barcode image, a second average value is used to determine whether the start character is located on the left or right side of the barcode image, thus determining whether the barcode is reversed. When the first average value is greater than a position threshold, the start character is determined to be on the right side of the barcode image, and the electronic device can then determine that the barcode is reversed.

[0125] Through steps S241 to S246, the electronic device can determine whether the barcode in the barcode image is reversed by determining the positions of the start and stop symbols in the barcode image. Furthermore, when the start or stop symbol of the barcode in the barcode image is missing, the position of the start or stop symbol in the barcode image can be determined by a position threshold, thereby determining whether the barcode in the barcode image is reversed. In this way, regardless of whether the start and stop symbols in the barcode image are missing, the electronic device can accurately determine whether the barcode in the barcode image is reversed, thereby accurately correcting the barcode in the barcode image to ensure the accuracy of barcode positioning.

[0126] After step S240, proceed to step S250: if the barcode is reversed, rotate the rotated barcode image, start symbol positioning image, data area positioning image and end symbol positioning image by 180 degrees.

[0127] If the barcode is reversed, the rotated barcode image, start symbol positioning image, data area positioning image, and stop symbol positioning image will be rotated another 180 degrees to ensure that the start symbol is on the left side of the barcode image and the barcode is in the forward orientation.

[0128] In the above embodiments, the electronic device can determine the angle of the barcode by the edge points of the barcode, and then rotate the barcode image, start symbol positioning image, data area positioning image, and stop symbol positioning image to be upright according to the angle of the barcode so that the barcode is placed horizontally. Furthermore, the electronic device can also determine whether the barcode is reversed based on the position of the start symbol and stop symbol in the barcode image. If the barcode is reversed, the uprighted barcode image, start symbol positioning image, data area positioning image, and stop symbol positioning image are rotated another 180 degrees to ensure that the barcode in the barcode image is upright, thereby ensuring the accuracy of subsequent barcode positioning.

[0129] Furthermore, the start and stop symbols may also contain barcode-specific information, such as the number of rows and data blocks. This information helps the electronic device to more accurately parse the data area. Therefore, in addition to locating the data area in the barcode, the electronic device can also locate the start symbol to provide a data basis for subsequent barcode decoding. Specifically, the method includes the following steps: Step S310: Determine the left boundary detection area of ​​the start symbol based on the start symbol positioning map.

[0130] Step S320: Traverse the pixels located within the detection area of ​​the left boundary of the start character on the barcode image to determine the left boundary of the start character in the barcode.

[0131] Step S330: Determine the positioning coordinates of the start character in the barcode based on the upper boundary, lower boundary, left boundary of the start character, and left boundary of the data area.

[0132] The specific process of determining the left boundary of the start symbol in steps S310 to S320 is the same as the specific process of determining the left boundary of the data area in steps S160 to S170, and will not be repeated here. Figure 1 As can be seen, the start symbol and the data area in a barcode are directly connected, and electronic devices can directly determine the left boundary of the data area as the right boundary of the start symbol.

[0133] Next, as Figure 10 As shown, the area enclosed by the upper boundary (i.e., line L1), the lower boundary (i.e., line L2), the left boundary of the start character (i.e., line L5), and the left boundary of the data area (i.e., line L3) is the precise position of the start character in the barcode. Therefore, the electronic device can obtain the positioning coordinates of the start character in the barcode by calculating the intersection points P1 and P5 between line L1 and lines L3 and L5, and the intersection points P3 and P6 between line L2 and lines L3 and L5. The specific method for determining the positioning coordinates of the start character in step S330 is the same as the specific method for determining the positioning coordinates of the data area in step S190, and will not be repeated here.

[0134] In the above embodiments, the electronic device can determine the left boundary detection area of ​​the start symbol based on the area initially located by the start symbol positioning map, so as to improve the efficiency and accuracy of determining the left boundary of the start symbol. Then, based on the upper and lower boundaries of the barcode and the left and right boundaries of the start symbol, the start symbol of the barcode can be accurately positioned, thereby ensuring the accuracy of the start symbol positioning and helping to improve the accuracy of barcode decoding.

[0135] Furthermore, the electronic device can also locate the terminator in the barcode to provide a data basis for subsequent barcode decoding. This method includes the following steps: Step S410: Determine the right boundary detection area of ​​the terminator based on the terminator positioning map.

[0136] Step S420: Traverse the pixels located within the right boundary detection area of ​​the terminator on the barcode image to determine the right boundary of the terminator in the barcode.

[0137] Step S430: Determine the positioning coordinates of the stop symbol in the barcode based on the upper boundary, lower boundary, right boundary of the data area, and right boundary of the stop symbol.

[0138] Among them, by Figure 1 As can be seen, the data area and the terminator in a barcode are directly connected, and electronic devices can directly determine the right boundary of the data area as the left boundary of the terminator. When the barcode in the image is black, and the origin of the coordinate system is located at the upper left corner of the image (specific coordinate system as shown in the image),... Figure 9 As shown in the figure, the x-coordinate of the right boundary of the terminator is determined as the target x-coordinate, with the pixel located at the right boundary of the terminator as the boundary. In the barcode image, among the two pixels adjacent to the pixel at the right boundary of the terminator in the direction shown on the X-axis, the pixel with the x-coordinate less than the target x-coordinate is black, and the pixel with the x-coordinate greater than the target x-coordinate is white.

[0139] Therefore, the electronic device can determine the pixel at the right edge of the terminator based on the pixel difference between pixels in the horizontal direction within the right boundary detection area of ​​the terminator, and then determine the right boundary of the terminator based on the pixel at the edge. The specific process of determining the right boundary of the terminator in steps S410 to S420 is similar to the specific process of determining the left boundary of the data area in steps S160 to S170, except that the specific method of determining the pixel at the right edge of the terminator is different, and will not be described again here.

[0140] Regarding the specific method for determining the pixel position at the right edge of the terminator, this application provides an implementation method. Assuming the origin of the coordinate system is at the upper left corner of the image and the image is located in the first quadrant of the coordinate system, the electronic device can calculate the pixel difference using the following formula:

[0141] Where K represents the barcode image, x and y are the coordinates of the currently visited pixel, and x satisfies , This indicates the right boundary detection region of the terminator, where y satisfies , The ordinate of the lower boundary obtained in step S150 and The ordinate of the upper boundary obtained in step S140 is y-coordinate.

[0142] If pixel difference If the value is greater than the preset threshold, it indicates a pixel. The pixel located at the right edge of the terminator can be designated as the left boundary point of the data area. This preset threshold can also be determined based on the pixel values ​​of each pixel in the barcode image. If the barcode image is a binary image, the preset threshold can be set to 0. If the pixel difference between two pixels horizontally adjacent to the currently traversed pixel is greater than 0, the currently traversed pixel is designated as the right boundary point of the terminator. If the barcode image is a grayscale image, the preset threshold can be set slightly larger, for example, to 20 or 30.

[0143] Next, as Figure 10 As shown, the area enclosed by the upper boundary (line L1), the lower boundary (line L2), the right boundary of the data area (line L4), and the right boundary of the terminator (line L6) is the precise location of the terminator in the barcode. Therefore, the electronic device can obtain the positioning coordinates of the terminator in the barcode by calculating the intersection points P2 and P7 between line L1 and lines L4 and L5, and the intersection points P4 and P8 between line L2 and lines L4 and L6. The specific method for determining the positioning coordinates of the terminator in step S430 is the same as the specific method for determining the positioning coordinates of the data area in step S190, and will not be repeated here.

[0144] In the above embodiments, the electronic device can determine the right boundary detection area of ​​the terminator based on the area initially located by the terminator positioning map, so as to improve the efficiency and accuracy of determining the right boundary of the terminator. Then, based on the upper and lower boundaries of the barcode and the left and right boundaries of the terminator, the precise positioning of the barcode terminator is achieved, thereby ensuring the accuracy of the terminator positioning and helping to improve the accuracy of barcode decoding.

[0145] Furthermore, for different barcode images, the electronic device can employ different positioning methods based on whether the start and stop symbols of the barcode are missing. For barcode images with normal and complete start and stop symbols, or barcode images where the start or stop symbol is partially missing but the left and right boundaries of the start and stop symbols can be located, the electronic device can execute steps S110 to S190, S310 to S330, and S410 to S430 to locate the start symbol, data area, and stop symbol of the barcode in the barcode image, that is, to determine the start symbol, data area, and stop symbol based on the barcode image. Figure 10 The coordinates of the points P1, P2, P3, P4, P5, P6, P7 and P8 are shown.

[0146] For barcode images where the start or stop symbol is completely missing, the electronic device can perform only steps S110 to S190 to locate the data area of ​​the barcode in the barcode image, that is, determine the data area based on the barcode image. Figure 14The coordinates of the points P1, P2, P3, and P4 are shown.

[0147] Regarding the method for determining whether the start and stop symbols of a barcode in a barcode image are missing, this application provides a specific implementation approach. The electronic device can detect the start symbol location map and the stop symbol location map generated by the image segmentation model. If the image segmentation model has not generated a start symbol location map, or if the pixel values ​​of all pixels in the start symbol location map are the same, then it is determined that the start symbol of the barcode in the barcode image is completely missing. If the image segmentation model has not generated a stop symbol location map, or if the pixel values ​​of all pixels in the stop symbol location map are the same, then it is determined that the stop symbol of the barcode in the barcode image is completely missing.

[0148] According to another aspect of the embodiments of this application, an electronic device is also provided. Figure 11 The diagram shows a structural schematic of an electronic device provided in an embodiment of this application. The specific embodiments of this application do not limit the specific implementation of the electronic device.

[0149] like Figure 11 As shown, the electronic device may include a processor 11 and a memory 12.

[0150] The memory 12 is used to store the computer program 13. The memory 12 may include high-speed RAM, and may also include non-volatile memory, such as at least one disk storage device. The computer program 13 may include computer-executable instructions.

[0151] The processor 11 is used to execute the computer program 13 to implement the above-described embodiment of the deep learning-based barcode positioning method.

[0152] Processor 11 may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of this application. The electronic device may include one or more processors of the same type, such as one or more CPUs; or it may include processors of different types, such as one or more CPUs and one or more ASICs.

[0153] This application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described deep learning-based barcode positioning method embodiment.

[0154] This application provides a computer program that can be executed by a processor to implement the above-described deep learning-based barcode positioning method embodiment.

[0155] This application provides a computer program product, which includes a computer program that, when executed by a processor, implements the above-described deep learning-based barcode positioning method embodiment.

[0156] In the several embodiments provided in this application, any function, if implemented as a software functional module / unit and sold or used as an independent product, can be stored in a computer-readable storage medium. Based on this understanding, part or all of the technical solutions of this application can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or other electronic device) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing computer program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0157] The algorithms or displays provided herein are not inherently related to any particular computer, virtual system, or other device. Various general-purpose systems can also be used in conjunction with the teachings herein. The required structure for constructing such systems is apparent from the above description. Furthermore, the embodiments of this application are not directed to any particular programming language. It should be understood that the content of this application described herein can be implemented using various programming languages, and the above description of specific languages ​​is for the purpose of disclosing the best mode of implementation of this application.

[0158] It should be noted that the above embodiments are illustrative of this application and not restrictive, and those skilled in the art can devise alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses should not be construed as limiting the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. This application can be implemented by means of hardware comprising several different elements and by means of a suitably programmed computer. In claims enumerating several means, several units or modules of these means may be embodied by the same item of hardware. The use of the words first, second, and third, etc., does not indicate any order. These words can be interpreted as names. The steps in the above embodiments, unless otherwise specified, should not be construed as limiting the order of execution.

[0159] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A barcode localization method based on deep learning, characterized in that, The method includes: A barcode image is acquired, and a pre-trained image segmentation model is used to segment the barcode in the barcode image to generate a start symbol location map, a data area location map, and a stop symbol location map. The starting points for upper and lower boundary detection are determined based on the start symbol positioning map, the data area positioning map, and the end symbol positioning map. The barcode image is subjected to a horizontal erosion operation to obtain an eroded image; Starting from the upper and lower boundary detection starting point, the pixels of the eroded image are traversed sequentially upwards to determine the upper boundary of the barcode; Starting from the upper and lower boundary detection starting point, the pixels of the eroded image are traversed downwards in sequence to determine the lower boundary of the barcode; The left boundary detection area and the right boundary detection area of ​​the data area are determined based on the data area positioning map. The left boundary of the data area in the barcode is determined by traversing the pixels located within the left boundary detection area of ​​the data area on the barcode image. The right boundary of the data area in the barcode is determined by traversing the pixels located within the right boundary detection area of ​​the data area on the barcode image. The positioning coordinates of the data area in the barcode are determined based on the upper boundary, the lower boundary, the left boundary of the data area, and the right boundary of the data area.

2. The deep learning-based barcode positioning method according to claim 1, characterized in that, Before determining the upper and lower boundary detection start points based on the start symbol positioning map, the data area positioning map, and the end symbol positioning map, the process further includes: The start symbol positioning map, the data area positioning map, and the end symbol positioning map are detected to obtain the edge points of the start symbol, data area, and end symbol in the barcode. Calculate the first straight line voting matrix based on the coordinates of the edge points, and determine the angle of the straight line with the highest number of votes in the first straight line voting matrix as the angle of the barcode. The barcode image, the start symbol positioning map, the data area positioning map, and the end symbol positioning map are rotated according to the angle of the barcode. Determine whether the barcode is reversed based on the start symbol positioning map and the end symbol positioning map; If the barcode is reversed, then the rotated barcode image, the start symbol positioning image, the data area positioning image, and the end symbol positioning image will be rotated 180 degrees.

3. The deep learning-based barcode positioning method according to claim 2, characterized in that, The step of determining whether the barcode is reversed based on the start symbol positioning map and the end symbol positioning map specifically includes: The first average value is obtained by calculating the average horizontal coordinate of each pixel in the start symbol region of the barcode image based on the start symbol positioning map. The average horizontal coordinate of each pixel within the terminator region of the barcode image is calculated based on the terminator positioning map to obtain a second average value. If both the first average value and the second average value are not equal to zero, and the first average value is greater than the second average value, then the barcode is determined to be reversed. If the first average value or the second average value is equal to zero, then obtain the image width of the barcode image and calculate the position threshold based on the image width; If the first average value is equal to zero, and the second average value is greater than zero and less than the position threshold, then the barcode is determined to be reversed. If the second average value is equal to zero and the first average value is greater than the position threshold, then the barcode is determined to be reversed.

4. The deep learning-based barcode positioning method according to claim 1, characterized in that, The method further includes: The left boundary detection area of ​​the start symbol is determined based on the start symbol positioning map; Traverse the pixels on the barcode image located within the detection area of ​​the left boundary of the start character to determine the left boundary of the start character in the barcode; The positioning coordinates of the start character in the barcode are determined based on the upper boundary, the lower boundary, the left boundary of the start character, and the left boundary of the data area.

5. The deep learning-based barcode positioning method according to claim 1, characterized in that, The method further includes: The right boundary detection area of ​​the terminator is determined based on the terminator positioning map; Traverse the pixels on the barcode image located within the right boundary detection area of ​​the terminator to determine the right boundary of the terminator in the barcode; The positioning coordinates of the terminator in the barcode are determined based on the upper boundary, the lower boundary, the right boundary of the data area, and the right boundary of the terminator.

6. The deep learning-based barcode positioning method according to claim 1, characterized in that, The step of determining the upper and lower boundary detection starting points based on the start symbol positioning map, the data area positioning map, and the end symbol positioning map specifically includes: Traverse the start symbol region in the start symbol positioning map, the data region in the data region positioning map, and the end symbol region in the end symbol positioning map, and calculate the average value of the ordinate of all pixels; The pixels in the eroded image whose ordinate is equal to the average ordinate are determined as the starting points for upper and lower boundary detection.

7. The deep learning-based barcode positioning method according to claim 6, characterized in that, The step of determining the upper boundary of the barcode by sequentially traversing the pixels of the eroded image upwards from the upper and lower boundary detection starting point specifically includes: Based on the coordinates of the currently traversed pixel, obtain the first pixel value of the pixel at the same coordinate on the start symbol positioning map, the data area positioning map, and the end symbol positioning map respectively; If the first pixel value is within a preset threshold range, then the second pixel value of the two pixels adjacent to the currently traversed pixel in the vertical direction in the eroded image is obtained; If the difference between the second pixel values ​​is greater than the first preset threshold, then the currently traversed pixel is determined as the upper boundary point; Within the first preset angle range, the second straight-line voting matrix is ​​calculated based on the upper boundary point; The line with the highest number of votes in the second line voting matrix is ​​determined as the upper boundary of the barcode.

8. The barcode positioning method based on deep learning according to claim 1, characterized in that, The step of determining the left boundary detection area and the right boundary detection area of ​​the data area based on the data area positioning map specifically includes: Traverse all pixels in the data area positioning map, calculate the pixel difference between two pixels adjacent to the currently traversed pixel in the horizontal direction, and determine the left edge point and right edge point in the data area positioning map based on the pixel difference. Calculate the average abscissa of the left edge points, and determine the left boundary detection area of ​​the data area based on the average abscissa of the left edge points; Calculate the average horizontal coordinate of the right edge points, and determine the right boundary detection area of ​​the data area based on the average horizontal coordinate of the right edge points.

9. The barcode positioning method based on deep learning according to claim 1, characterized in that, The step of traversing the pixels located within the left boundary detection area of ​​the data area on the barcode image to determine the left boundary of the data area in the barcode specifically includes: If the pixel difference between two pixels in the barcode image that are adjacent to the currently traversed pixel in the horizontal direction is greater than the second preset threshold, then the currently traversed pixel is determined as the left boundary point of the data area. Within the second preset angle range, the third straight-line voting matrix is ​​calculated based on the left boundary point of the data area; The line with the highest number of votes in the third line voting matrix is ​​determined as the left boundary of the data area in the barcode.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory, characterized in that, The processor executes the computer program to implement the deep learning-based barcode positioning method according to any one of claims 1 to 9.

Images