A method for locating a barcode area and a method for fitting a rectangular area of ​​an image

Abstract

A method for locating a barcode region and a method for fitting a rectangular region of an image can very accurately locate and segment a rectangle from a connected region. When applied to barcode region positioning, a rectangle can be further segmented from the region containing the barcode, thereby completing the precise positioning of the barcode.

Description

Technical Field

[0001] The invention relates to a method for positioning a barcode area and a method for fitting a rectangular area of ​​an image. Background Art

[0002] One of the biggest problems in image processing is edge detection, which is to extract the edges of objects. There are many relatively standard shapes or edges that can be detected and extracted, such as circles, triangles, ellipses and rectangles. There are many objects in life that are rectangular when projected on a two-dimensional plane, such as televisions, mobile phones, tablets, bar codes, QR codes, etc. Therefore, this application studies how to further locate the rectangular area in the preliminary contour area that has been determined in the image. Summary of the invention

[0003] The invention discloses a method for positioning a barcode area and a method for fitting a rectangular area of ​​an image.

[0004] According to the first aspect, an embodiment provides a method for locating a barcode area, comprising:

[0005] Get the original image;

[0006] According to the original image, a corresponding labeling image is obtained;

[0007] Tracing the contour of the region in the labeled image to obtain the contour information of the connected region in the image;

[0008] According to the contour information, all contour points on the area contour are obtained;

[0009] constructing a convex polygon according to the contour points;

[0010] The two points with the greatest distance in the constructed convex polygon are determined as the two points of the diagonal of the rectangle;

[0011] Determine two other points of the rectangle according to the two points of the diagonal line of the rectangle to form an initial rectangle;

[0012] Segmenting each contour point according to the initial rectangle;

[0013] Rectangle fitting is performed according to the segmentation result to determine a rectangular area from the connected area in the image as the barcode area.

[0014] According to the second aspect, an embodiment provides a method for fitting a rectangular region of an image, comprising:

[0015] Get the contour information of the connected area in the image;

[0016] According to the contour information, all contour points on the area contour are obtained;

[0017] constructing a convex polygon according to the contour points;

[0018] The two points with the greatest distance in the constructed convex polygon are determined as the two points of the diagonal of the rectangle;

[0019] Determine two other points of the rectangle according to the two points of the diagonal line of the rectangle to form an initial rectangle;

[0020] Segmenting each contour point according to the initial rectangle;

[0021] Rectangle fitting is performed according to the segmentation result to determine a rectangular area from the connected areas in the image.

[0022] In one embodiment, constructing a convex polygon according to the contour points includes:

[0023] Determine an index point from the contour points, and calculate the angles of all other contour points relative to the index point;

[0024] Sort the contour points according to their angles relative to the index point;

[0025] Construct a convex polygon based on the order of contour points.

[0026] In one embodiment, determining the index point from the contour points includes:

[0027] The contour point with the smallest Y coordinate in the image is taken as the index point. If there are multiple contour points with the smallest Y coordinate, the contour point with the smallest or largest X coordinate is taken as the index point; or, the contour point with the largest Y coordinate in the image is taken as the index point. If there are multiple contour points with the largest Y coordinate, the contour point with the smallest or largest X coordinate is taken as the index point; or, the contour point with the largest X coordinate in the image is taken as the index point. If there are multiple contour points with the largest X coordinate, the contour point with the smallest or largest Y coordinate is taken as the index point; or, the contour point with the smallest X coordinate in the image is taken as the index point. If there are multiple contour points with the smallest X coordinate, the contour point with the smallest or largest Y coordinate is taken as the index point;

[0028] The step of sorting the contour points according to the angles of the contour points relative to the index point comprises:

[0029] According to the angle of each contour point relative to the index point, the contour points are sorted from large to small; or, according to the angle of each contour point relative to the index point, the contour points are sorted from small to large;

[0030] If there are multiple contour points with the same angle as the index point, only the contour point farthest from the index point is retained for sorting.

[0031] In one embodiment, constructing a convex polygon according to the order of contour points includes:

[0032] The contour points are sorted from small to large according to the angle relative to the index point, where the index point is the first point in the sort;

[0033] The first and second points are placed in the stack, and the following iterative process is performed until the last point is reached:

[0034] Determine the second top point and the first top point of the current stack, and the current point, wherein the current point is the next point in order relative to the first top point;

[0035] Determine whether the rotation from the second point on the top of the stack to the first point on the top of the stack and from the first point on the top of the stack to the current point is counterclockwise;

[0036] If it is rotating counterclockwise, the current point is pushed into the stack to update the stack;

[0037] If it is not rotating counterclockwise, the first point on the top of the current stack is popped out of the stack, and the current point is pushed into the stack to update the stack;

[0038] Determine whether the first point at the top of the updated stack is the last point in the sort;

[0039] If it is judged to be the last point in the sort, the iteration is stopped, wherein all points in the updated stack can be connected in sequence to complete the construction of the convex polygon;

[0040] If it is determined that it is not the last point in the sort, the next point in the sort relative to the updated first point on the top of the stack will be used as the current point, and it will be determined whether the rotation from the second point on the top of the stack to the first point on the top of the stack and from the first point on the top of the stack to the current point is counterclockwise.

[0041] In one embodiment, determining the other two points of the rectangle according to the two points on the diagonal of the rectangle includes:

[0042] From the remaining contour points or the remaining points of the convex polygon, find the pixel point whose angle difference between the straight line formed by the two points of the diagonal line of the rectangle and 90 degrees is the smallest, as the remaining point in the rectangle;

[0043] Preferably, the remaining points in the rectangle are found by the following formula:

[0044]

[0045] Where (x i ,y i ) represents the coordinates of the i-th contour point, (x f ,y f) and (x t ,y t ) respectively represent the coordinates of two points on the diagonal line of the rectangle; It means that a contour point is identified among the contour points that can maximize the value in the brackets, which is called the third point.

[0046] In one embodiment, determining the other two points of the rectangle according to the two points on the diagonal of the rectangle further includes:

[0047] The pixel point found is positionally corrected so that the angle between the corrected pixel point and the straight lines formed by the two points of the diagonal line of the rectangle is 90 degrees;

[0048] Preferably, the correction is performed by the following formula to obtain the remaining two points:

[0049]

[0050] where d max It means (x f ,y f ) and (x t ,y t )The distance between these two points; (x m ,y m ) means (x f ,y f ) and (x t ,y t )The coordinates of the midpoint of these two points; (x fit ,y fit ) represents the third point; d fit It means (x m ,y m ) and (x fit ,y fit ) is the distance between these two points.

[0051] In one embodiment, segmenting each contour point according to the initial rectangle includes:

[0052] Calculate the distance between each contour point and the four sides of the initial rectangle;

[0053] According to the distance, the contour points are divided into the side with the shortest distance from the four sides, so as to divide the contour points into four groups, one group corresponding to each side.

[0054] In one embodiment, performing rectangle fitting according to the segmentation result includes:

[0055] The corresponding edges are fitted with straight lines through the contour points that are divided into the same group to complete the fitting of the rectangle;

[0056] Preferably, a straight line is fitted by the following formula:

[0057] Construct the lines that form the four sides of the rectangle:

[0058] ax+by+c=0,ax+by+d=0

[0059] bx-ay+e=0,bx-ay+f=0

[0060] And establish its objective function:

[0061]

[0062] where a 2 +b 2 =1; the contour points corresponding to the summation signs corresponding to each edge in the objective function are the contour points segmented to the corresponding edges.

[0063] According to a third aspect, an embodiment provides a computer-readable storage medium, wherein the computer-readable storage medium stores a program, and the program can be executed by a processor to implement a method as described in any embodiment of the present invention.

[0064] According to the barcode region positioning method, image rectangular region fitting method and computer-readable storage medium of the above-mentioned embodiments, a rectangle can be positioned and segmented from a connected region very accurately. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] Figure 1 A flowchart of a rectangular region fitting method for an image according to an embodiment;

[0066] Figure 2 A flowchart of constructing a convex polygon according to an embodiment;

[0067] Figure 3.1 to Figure 3.11 is a schematic diagram of the process involved in the algorithm for constructing convex polygons; and Figure 3.12 is a schematic diagram of a rectangle obtained after correction or an initial rectangle formed;

[0068] Figure 4 A flowchart of a method for locating a barcode area according to an embodiment;

[0069] Figure 5 A flowchart of obtaining a corresponding labeling image according to an original image according to an embodiment;

[0070] Figure 6 An example of a pyramid consisting of 5 layers;

[0071] Figure 7 A flowchart of steps for obtaining a label map of an image set according to an embodiment;

[0072] Figure 8 A flowchart of the preliminary label map generation steps of one embodiment. DETAILED DESCRIPTION

[0073] The present invention is further described in detail below by specific embodiments in conjunction with the accompanying drawings. Wherein similar elements in different embodiments adopt associated similar element numbers. In the following embodiments, many detailed descriptions are for making the present application better understood. However, those skilled in the art can easily recognize that some features can be omitted in different situations, or can be replaced by other elements, materials, methods. In some cases, some operations related to the present application are not shown or described in the specification, this is to avoid the core part of the present application being overwhelmed by too much description, and for those skilled in the art, it is not necessary to describe these related operations in detail, and they can fully understand the related operations according to the description in the specification and the general technical knowledge in the art.

[0074] In addition, the features, operations or characteristics described in the specification can be combined in any appropriate manner to form various implementations. At the same time, the steps or actions in the method description can also be interchanged or adjusted in a manner that is obvious to those skilled in the art. Therefore, the various sequences in the specification and the drawings are only for the purpose of clearly describing a certain embodiment and are not meant to be a required sequence, unless otherwise specified that a certain sequence must be followed.

[0075] The serial numbers of the components in this document, such as "first", "second", etc., are only used to distinguish the objects described and do not have any order or technical meaning. The "connection" and "coupling" mentioned in this application, unless otherwise specified, include direct and indirect connections (couplings).

[0076] Please refer to Figure 1 In some embodiments of the present invention, a rectangular region fitting method for an image is disclosed, comprising the following steps:

[0077] Step 410: Obtain contour information of connected regions in the image.

[0078] In step 410, the boundary or contour information of the connected region in the image is extracted. It can be understood that there can be one or more connected regions in the image. For each connected region, the rectangular region fitting method of the image in the present application can be executed.

[0079] Step 420: According to the above contour information, all contour points on the area contour are obtained.

[0080] Step 420 extracts all the pixels on the outline or boundary.

[0081] Step 430: Construct a convex polygon based on the above contour points.

[0082] Step 430 is to construct a convex polygon for the connected region based on all the contour points of the connected region. The following specifically describes how to construct a convex polygon.

[0083] Please refer to Figure 2 In some embodiments, step 430 of constructing a convex polygon according to the contour points may include the following steps:

[0084] Step 431: Determine the index point from the contour points, and calculate the angles of all other contour points relative to the index point.

[0085] For a connected region, step 431 determines an index point from all contour points of the connected region, and then calculates the angles of all remaining contour points relative to the index point.

[0086] In one embodiment, the contour point with the smallest Y coordinate in the image is used as the index point. If there are multiple contour points with the smallest Y coordinate, the contour point with the smallest or largest X coordinate is used as the index point. In one embodiment, the contour point with the largest Y coordinate in the image is used as the index point. If there are multiple contour points with the largest Y coordinate, the contour point with the smallest or largest X coordinate is used as the index point. In one embodiment, the contour point with the largest X coordinate in the image is used as the index point. If there are multiple contour points with the largest X coordinate, the contour point with the smallest or largest Y coordinate is used as the index point. In one embodiment, the contour point with the smallest X coordinate in the image is used as the index point. If there are multiple contour points with the smallest X coordinate, the contour point with the smallest or largest Y coordinate is used as the index point.

[0087] It should be noted that the above X-coordinate and Y-coordinate are a normal coordinate system, that is, a coordinate system that conforms to the right-hand rule. For example, in terms of the paper direction, the positive X-axis direction is to the right, and the positive Y-axis direction is upward.

[0088] Step 433: Sort the contour points according to the angle of each contour point relative to the index point.

[0089] In one embodiment, the contour points may be sorted from large to small according to the angle of each contour point relative to the index point. In another embodiment, the contour points may be sorted from small to large according to the angle of each contour point relative to the index point.

[0090] During the calculation process, if there are multiple points with the same angle as the index point, only the point farthest from the index point is protected for sorting.

[0091] Step 435: Construct a convex polygon according to the order of the contour points.

[0092] In some embodiments, step 435 of constructing a convex polygon according to the sorting of contour points can be implemented in the following manner or algorithm. It is useful to take the case where the contour points are sorted from small to large according to the angle relative to the index point as an example. It can be understood that the index point is the first point in the sorting, and the index point is determined as follows: the contour point with the smallest Y coordinate is used as the index point. If there are multiple contour points with the smallest Y coordinate, the contour point with the smallest X coordinate is used as the index point.

[0093] Specifically, the first point and the second point are placed in the stack in sequence, and the following iterative process is performed until the last point is reached:

[0094] (1) Determine the second point at the top of the current stack, the first point at the top of the stack, and the current point, where the current point is the next point relative to the first point at the top of the stack in the sorting. For example, if the first point at the top of the stack is the i-th point after the contour points are sorted, then the current point is the i+1-th point after the contour points are sorted.

[0095] The first point on the stack is the point at the top of the stack, and the second point on the stack is the point below the top point on the stack. The current point is the next point relative to the first point on the stack in the sorting. For example, if the first point on the stack is the i-th point after the contour points are sorted, then the current point is the i+1-th point after the contour points are sorted.

[0096] Specifically, after the first point and the second point after the contour points are sorted are placed in the stack in sequence, the first point on the top of the stack is the second point after the contour points are sorted, and the second point on the top of the stack is the first point after the contour points are sorted; and the current point is the third point after the contour points are sorted. In the subsequent iteration process, the second point on the top of the stack, the first point on the top of the stack, and the current point will continue to change and update until the first point on the top of the stack is the last point after the contour points are sorted, and then the iteration stops.

[0097] (2) Determine whether the rotation from the second point on the top of the stack to the first point on the top of the stack and from the first point on the top of the stack to the current point is counterclockwise.

[0098] (3) If the rotation is counterclockwise, the current point is pushed into the stack to update the stack.

[0099] (4) If it is not rotating counterclockwise, pop the first point on the top of the current stack and push the current point into the stack to update the stack.

[0100] In steps (3) and (4), if the three points from the second point at the top of the stack to the first point at the top of the stack and from the first point at the top of the stack to the current point are in a straight line, they are also counted as counterclockwise rotation in (3) in this article, and if (4) is not counterclockwise rotation, it actually refers to clockwise rotation.

[0101] (5) Determine whether the first point at the top of the updated stack is the last point in the sort;

[0102] (6) If it is determined to be the last point in the sort, the iteration is stopped, wherein all points in the updated stack can be connected in sequence to complete the construction of the convex polygon;

[0103] For example, starting from the bottom point of the stack, connect upwards in sequence, and after connecting to the last point (i.e., the vertex of the stack, i.e., the first point on the top of the stack), connect the vertex of the stack with the bottom point of the stack to form a closed shape, which is the constructed convex polygon.

[0104] For another example, starting from the vertex of the stack, that is, the first point on the top of the stack, connect downwards in sequence, and after connecting to the last point (that is, the bottom point of the stack), connect the bottom point of the stack with the vertex of the stack to form a closed shape, which is the constructed convex polygon.

[0105] (7) If it is determined that the point is not the last point in the sort, the next point in the sort relative to the updated first point on the top of the stack is continued to be used as the current point, and it is continued to be determined whether the rotation from the second point on the top of the stack to the first point on the top of the stack and from the first point on the top of the stack to the current point is counterclockwise.

[0106] Of course, the above is an example of taking the contour point with the smallest Y coordinate as the index point. If there are multiple contour points with the smallest Y coordinate, the contour point with the smallest X coordinate is taken as the index point, and the contour points are sorted from small to large according to the angle relative to the index point. Similarly, it can also be an example of taking the contour point with the largest Y coordinate as the index point. If there are multiple contour points with the largest Y coordinate, the contour point with the smallest X coordinate is taken as the index point, and the contour points are sorted from large to small according to the angle relative to the index point (in terms of the coordinate system of 0 to 360 degrees). However, in this case, the judgment of step (2) is whether the second point at the top of the stack to the first point at the top of the stack and the first point at the top of the stack to the current point rotates clockwise (including three points on a straight line). If so, the current point is pushed into the stack to update the stack. If not, the first point at the top of the current stack is popped out of the stack and the current point is pushed into the stack to update the stack. The other methods for determining index points in step 431 and the corresponding sorting methods in step 433 are similar, except that the judgment criteria in (2) are changed, and will not be described in detail here.

[0107] The following is a specific implementation to illustrate the construction process of convex polygons.

[0108] Please refer to Figure 3.1, which are all the contour points of a connected area in the image, a total of 18; the contour point with the smallest Y coordinate is used as the index point. If there are multiple contour points with the smallest Y coordinate, the contour point with the smallest X coordinate is used as the index point, that is, Figure 3.2 Then calculate and sort the contour points from small to large according to the angle of the contour point relative to the index point to form Figure 3.2 Put P0 and P1 into the stack in sequence. At this time, P0 is the second point on the top of the stack, P1 is the first point on the top of the stack, and the current point is P2; Figure 3.3 As shown, since P0 to P1 and P1 to P2 rotate counterclockwise, P2 is pushed into the stack to update the stack; at this time, P1 is the second point on the top of the stack, P2 is the first point on the top of the stack, and the current point is P3; Figure 3.4 As shown, since P1 to P2 and P2 to P3 rotate clockwise, P2 is popped out of the stack and P3 is pushed into the stack to update the stack; at this time, P1 is the second point on the top of the stack, P3 is the first point on the top of the stack, and the current point is P4; as shown Figure 3.5 As shown, since P1 to P3 and P3 to P4 rotate counterclockwise, P4 is pushed into the stack to update the stack; at this time, P3 is the second point on the top of the stack, P4 is the first point on the top of the stack, and the current point is P5; Figure 3.6 As shown, since P3 to P4 and P4 to P5 rotate clockwise, P4 is popped out of the stack and P5 is pushed into the stack to update the stack; at this time, P3 is the second point on the top of the stack, P5 is the first point on the top of the stack, and the current point is P6; similarly, as Figure 3.7 As shown, the judgment is carried out in sequence, P6, P7 and P8 will be pushed into the stack and no contour point will be popped out during this period; then, as Figure 3.8 , when the current point is P9, then judge in sequence, P8 and P7 will be popped out of the stack, and P9 will be pushed into the stack; then, Figure 3.9 , the current point is P10, and then judge, P9, P6, P5 will appear in turn out of the stack, and P10 will be pushed into the stack; then, Figure 3.10 , P11 is pushed into the stack, and then when the current point is P12, after judgment, P11 is popped out of the stack and P12 is pushed into the stack. At this time, P12 is the last point, and the process ends. The current stack contains P0, P1, P3, P10 and P12. Connect these points in sequence to form a convex polygon, for example Figure 3.11 .

[0109] Step 440: Determine the two points with the farthest distance in the constructed convex polygon as the two points of the diagonal line of the rectangle.

[0110] For example Figure 3.11 P3 and P12 are the two points with the farthest distance in the polygon, and these two points are determined as the two points of the diagonal of the rectangle.

[0111] Step 450: Determine two other points of the rectangle based on the two points on the diagonal line of the rectangle to form an initial rectangle.

[0112] By step 440, two points of the two diagonals of the rectangle have been found, and then the remaining two points can be determined.

[0113] Specifically, the pixel point whose angle difference between the straight line formed by the two points of the diagonal line of the rectangle and 90 degrees is the smallest can be found from the remaining contour points or the remaining points of the convex polygon, as the remaining points in the rectangle. For example, the remaining points in the rectangle can be found by the following formula:

[0114]

[0115] Where (x i ,y i ) represents the coordinates of the i-th contour point, (x f ,y f ) and (x t ,y t ) respectively represent the coordinates of two points on the diagonal line of the rectangle; It means that a contour point is identified among the contour points that can maximize the value in the brackets, which is called the third point.

[0116] The pixel point found (such as the third point mentioned above) is positionally corrected so that the angle between the corrected pixel point and the straight lines formed by the two points of the diagonal line of the rectangle is 90 degrees;

[0117] Preferably, the correction is performed by the following formula to obtain the remaining two points:

[0118]

[0119] where d max It means (x f ,y f ) and (x t ,y t )The distance between these two points; (x m ,y m ) means (x f ,y f ) and (x t ,y t )The coordinates of the midpoint of these two points; (x fit ,y fit ) represents the third point; d fit It means (x m ,y m ) and (x fit ,y fit ) is the distance between these two points.

[0120] For example Figure 3.11 is an example, where P10 is the third point found, and its coordinates can be recorded as (x fit ,y fit ), and then perform position correction to obtain the remaining two points in the rectangle except P3 and P12. Figure 3.12 is the rectangle obtained after correction, where the four points of the rectangle are P3, P12, P rect,1 and P rect,2 ; The coordinates corresponding to P3 and P12 in the above formula are (x f ,y f ) and (x t ,y t ), the middle point is denoted as P m , the coordinates are marked as (x m ,y m ).

[0121] Step 460: Segment each contour point according to the initial rectangle.

[0122] The purpose of this step is to divide the contour points of the connected area into four groups, each group is used to fit one side of the rectangle. Specifically, step 460 calculates the distance between each contour point and the four sides of the initial rectangle; according to the distance, the contour point is divided into the side with the shortest distance from the four sides, so as to divide the contour points into four groups, one group corresponding to each side.

[0123] Step 470: Perform rectangle fitting according to the segmentation result to determine a rectangular area from the connected areas in the image.

[0124] Specifically, step 470 performs straight line fitting on the corresponding edges through the contour points that are divided into the same group to complete the fitting of the rectangle. In some embodiments, the straight line fitting can be performed using the following formula:

[0125] Construct the lines that form the four sides of the rectangle:

[0126] ax+by+c=0,ax+by+d=0

[0127] bx-ay+e=0,bx-ay+f=0

[0128] And establish its objective function:

[0129]

[0130] where a 2 +b 2 =1; the contour points corresponding to the summation signs corresponding to each edge in the objective function are the contour points segmented to the corresponding edges.

[0131] After the objective function and constraints are proposed, they are solved to obtain the corresponding parameters a, b, c, d, e, f that minimize E(a, b, c, d, e, f).

[0132] After obtaining a and b, the coordinates of the four points of the rectangle can be obtained as follows:

[0133]

[0134] in are the average values ​​of the points on the corresponding sides.

[0135] The above is the method for rectangular region fitting of an image in some embodiments of the present invention. This method has many practical applications, such as being used in barcode region positioning, which is described below with examples.

[0136] Please refer to Figure 4 In some embodiments, a method for locating a barcode region is disclosed, which comprises the following steps:

[0137] Step 100, namely the image acquisition step, acquires the original image.

[0138] Images can be acquired through devices capable of taking pictures, such as mobile phones, cameras, and camcorders.

[0139] Step 110: Obtain a corresponding labeling image according to the original image.

[0140] Next, the above step 410 is executed, for example, contour tracking of the region is performed on the label image to obtain contour information of the connected region in the image.

[0141] Then continue to execute the above steps 420 to 470 to determine a rectangular area as the barcode area:

[0142] Step 420: Obtain all contour points on the region contour according to the contour information;

[0143] Step 430: constructing a convex polygon according to the contour points;

[0144] Step 440: Determine the two points with the farthest distance in the constructed convex polygon as the two points of the diagonal line of the rectangle;

[0145] Step 450: Determine two other points of the rectangle according to the two points of the diagonal line of the rectangle to form an initial rectangle;

[0146] Step 460: segment each contour point according to the initial rectangle;

[0147] Step 470: Rectangle fitting is performed according to the segmentation result to determine a rectangular area from the connected area in the image as a barcode area. Decoding can then be performed according to the barcode area.

[0148] In the above steps, step 110 can be implemented in a variety of ways, for example, please refer to Figure 5 , which can be achieved through steps 200 to 300.

[0149] The pyramid image generation step 200 generates a set of pyramid images with a preset number of layers according to the original image in step 100 .

[0150] In order to improve the positioning speed of the significant area of ​​the barcode, the pyramid image generation step 200 can be introduced, and the algorithm design can be performed using a hierarchical structure to achieve coarse-to-fine positioning. In some embodiments, the hierarchical structure can be designed using a pyramid structure to reduce the image resolution, reduce the image calculation amount, and improve the calculation speed. For example Figure 6 As shown in the figure, it is an example of a pyramid consisting of 5 layers. Level 0 is the original input image. As each layer goes up, the resolution of the image is reduced to half of the input image. Assuming that the size of the input image is W×H, the image resolution of the Level i layer is

[0151] The step 300 of obtaining the label map of the image set is to process the image set of the pyramid to obtain the label map output by the image set. The label map can be a binary image, in which 1 and 0 represent different information respectively. For example, in the present invention, when the value of a pixel point in the label map is 1, it means that the image has high-frequency information at that location, and when the value of the pixel point is 0, it means that the image has low-frequency information at that location.

[0152] Please refer to Figure 7 In some embodiments, the label map acquisition step 300 of the image set starts from the top layer of the pyramid image set and stops at the bottom layer, and the following operations are performed on each layer of the pyramid image: a suspected barcode area determination step 310, a gradient calculation step 320, a preliminary label map generation step 330, a direction determination step 340, a cluster analysis step 350 and an output step 360, which are described in detail below.

[0153] The suspected barcode area determination step 310 is to determine the suspected barcode area of ​​the pyramid image of the current layer according to the label map output by the pyramid image of the previous layer; when the pyramid image of the current layer is the topmost pyramid image, the entire area is regarded as the suspected barcode area.

[0154] In some specific embodiments, the suspected barcode containing region determining step 310 maps the region set to 1 in the label map output by the pyramid image of the previous layer to the pyramid image of the current layer according to the ratio of the pyramid images of the previous layer and the current layer; and then performs a dilation operation on the region mapped to the pyramid image of the current layer to obtain the dilated region, and the dilated region is used as the suspected barcode containing region of the pyramid image of the current layer.

[0155] In the mapping process, the suspected barcode area determination step 310 is performed, for example, Figure 6 Taking the pyramid image of as an example, when the area set to 1 in the label map output by the pyramid image of the previous layer is mapped to the pyramid image of the current layer, due to the double difference in the coordinate direction of the upper and lower layers, when a pixel point of a label map of the upper layer is mapped to the pyramid image of the lower layer (i.e., the current layer), it corresponds to four pixels.

[0156] The dilation operation involved in step 310 of determining the area suspected of containing a barcode is a morphological operation. Generally, dilation and erosion are a pair of opposite operations. Dilation is to dilate the highlighted part of the image, such as the position of the value 1 in the marker image, which is similar to domain expansion. Specifically, dilation is an operation to find the local maximum value. From a mathematical point of view, a dilation or erosion operation is to convolve an image (or a part of an image) with a kernel. The kernel can be of any shape and size, and it has a separately defined reference point, which we call an anchor point. In most cases, the kernel is a small square or disk with a reference point and a solid center; in fact, the kernel can also be regarded as a template or mask. Dilation is an operation to find the local maximum value. The kernel is convolved with the image, that is, the maximum value of the pixels in the area covered by the kernel is calculated, and this maximum value is assigned to the pixel specified by the reference point, which will gradually increase the highlighted area in the image.

[0157] In the gradient calculation step 320 , for the area suspected to contain the barcode in the pyramid image of the current layer, a gradient image in the first direction and a gradient image in the second direction are calculated.

[0158] In some embodiments, the first direction is perpendicular to the second direction; for example, the first direction may be a 45-degree direction, and the second direction may be a 135-degree direction.

[0159] The preliminary labeling map generation step 330 determines the high- and low-frequency information of the suspected barcode area based on the gradient image in the first direction and the gradient image in the second direction, and sets the label values ​​corresponding to the high-frequency information and the low-frequency information to 1 and 0 respectively, so as to generate a preliminary labeling map of the pyramid image of the current layer.

[0160] Please refer to Figure 8In some embodiments, the preliminary label map generation step 330 may include the following steps:

[0161] Step 331: Generate a gradient image in a comprehensive direction according to the gradient image in the first direction and the gradient image in the second direction.

[0162] Step 332: For each pixel point, construct a covariance matrix of the pixel point according to the values ​​of its gradient image in the first direction, the gradient image in the second direction, and the gradient image in the comprehensive direction.

[0163] In some embodiments, the constructed covariance matrix of the pixels is as follows:

[0164]

[0165] Where (u, v) represents the pixel point of the pyramid image at the i-th layer, I i,x ,I i,y and I i,xy are respectively the gradient image of the pyramid image of the i-th layer in the first direction, the gradient image in the second direction and the gradient image in the comprehensive direction; I i,xy =I i,x I i,y ;

[0166] Since the gradient image may contain noise, before constructing the covariance matrix through the gradient image, the gradient image in the first direction, the gradient image in the second direction and the gradient image in the comprehensive direction may be smoothed by filtering, for example, using a Gaussian convolution kernel for smoothing filtering.

[0167] Step 333: Calculate and determine the high- and low-frequency information of the pixel based on the eigenvalue correlation ratio of the covariance matrix of the pixel, and set the label values ​​corresponding to the high-frequency information and the low-frequency information to 1 and 0 respectively, so as to generate a preliminary label map of the pyramid image of the current layer.

[0168] In some embodiments, step 333 calculates and determines the high-frequency and low-frequency information of the pixel point based on the eigenvalue correlation ratio of the covariance matrix of the pixel point, including: calculating the eigenvalue correlation ratio k of the covariance matrix of the pixel point, where where e 1 =(ab) 2 +4c 2 , e 2 =(a+b); and then determine whether k is within a preset range. If so, it is determined that the pixel has high-frequency information. In some embodiments, the preset range may be 0.3 to 4.

[0169] In the direction determination step 340, a connectivity analysis is performed on the preliminary label map of the pyramid image of the current layer to obtain connected areas; for each connected area, the angles of each pixel point are calculated and counted to obtain at least the main direction of each connected area of ​​the preliminary label map.

[0170] In some specific embodiments, the direction determination step 340 calculates the direction vector of each pixel point in the connected area, calculates the corresponding angle based on the direction vector of the pixel point, and converts the angle of the pixel point to a 180-degree interval, for example, [-90°, 90°], wherein the 180-degree interval is divided into a preset number of small intervals, for example, 8 intervals. If more intervals are divided, the amount of calculation will increase a lot, resulting in a slower speed. If fewer intervals are divided, the accuracy cannot be guaranteed. The number of pixels in each small interval is then counted, and the direction represented by the small interval with the largest number of pixels is used as the main direction of the current connected area. In some embodiments, the direction represented by the small interval with the second largest number of pixels is also used as the secondary direction of the current connected area.

[0171] The cluster analysis step 350 is to perform cluster analysis on the connected regions in the preliminary labeled graph at least according to the main directions of the connected regions in the preliminary labeled graph, so as to obtain a labeled graph after cluster analysis.

[0172] In some embodiments, the clustering analysis step includes: when the boundary distance of the circumscribed rectangles of two regions is less than a preset threshold distance, and the difference in the angle of the main direction is less than a preset threshold angle, the two regions are merged, and the angle of the main direction of the merged region is updated, for example, the area of ​​the region before the merger is used as a weight to calculate the weighted average of the angles of the main direction, and the angle of the main direction of the merged region is obtained.

[0173] In some embodiments, the secondary direction is also considered in the cluster analysis process. Therefore, when the boundary distance of the circumscribed rectangles of two regions is less than a preset threshold distance, and the difference in the angle of the main direction is less than a preset threshold angle, the two regions are merged; or, when the boundary distance of the circumscribed rectangles of two regions is less than a preset threshold distance, and the difference in the angle of the secondary direction is less than a preset threshold angle, the two regions are also merged; then the angle of the main direction and the angle of the secondary direction of the merged region are updated, for example, the area of ​​the region before the merger is used as a weight to calculate the weighted average of the angles of the main direction, and the angle of the main direction of the merged region is obtained; the area of ​​the region before the merger is used as a weight to calculate the weighted average of the angles of the secondary direction, and the angle of the secondary direction of the merged region is obtained.

[0174] In the output step 360, the labeled image after the cluster analysis is output; the labeled image output by the bottom layer of the pyramid image is used as the labeled image output by the pyramid image set.

[0175] In some embodiments, when outputting the labeled image after cluster analysis, the output step 360 first performs an OR operation on the labeled image and the labeled image output by the pyramid image of the previous layer; when the pyramid image of the current layer is the topmost pyramid image, the labeled image after cluster analysis is directly output.

[0176] The labeling image outputted in the output step 360 is the “obtaining a corresponding labeling image based on the original image” involved in the step 110 .

[0177] The above is an explanation of the method for locating the barcode area in some embodiments of the present invention. In some embodiments of the present invention, a positioning process from coarse to fine is implemented, the amount of calculation is greatly reduced, and the real-time performance of the system is improved. In some embodiments of the present invention, a processing process from coarse positioning to precise positioning is implemented, which improves the positioning accuracy and also provides accurate barcode direction for subsequent decoding. In some embodiments of the present invention, the coarse positioning process improves the positioning capability and robustness of complex scenes such as lighting changes and blur. In some embodiments of the present invention, the adaptability of barcode positioning to complex environments is proposed. In some embodiments of the present invention, the accuracy of barcode positioning is improved.

[0178] This document is described with reference to various exemplary embodiments. However, those skilled in the art will recognize that changes and modifications may be made to the exemplary embodiments without departing from the scope of this document. For example, various operating steps and components for performing the operating steps may be implemented in different ways (e.g., one or more steps may be deleted, modified, or incorporated into other steps) depending on the specific application or considering any number of cost functions associated with the operation of the system.

[0179] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. In addition, as understood by those skilled in the art, the principles of this article can be reflected in a computer program product on a computer-readable storage medium, which is pre-installed with a computer-readable program code. Any tangible, non-temporary computer-readable storage medium can be used, including magnetic storage devices (hard disks, floppy disks, etc.), optical storage devices (CD to ROM, DVD, Blue Ray disks, etc.), flash memory and / or the like. These computer program instructions can be loaded onto a general-purpose computer, a special-purpose computer or other programmable data processing device to form a machine, so that these instructions executed on a computer or other programmable data processing device can generate a device that implements a specified function. These computer program instructions can also be stored in a computer-readable memory, which can instruct a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory can form a manufactured product, including an implementation device that implements a specified function. Computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operating steps are performed on a computer or other programmable device to generate a computer-implemented process, so that the instructions executed on a computer or other programmable device can provide steps for implementing a specified function.

[0180] Although the principles of this invention have been shown in various embodiments, many modifications of structures, arrangements, proportions, elements, materials and components particularly suitable for specific environments and operational requirements can be used without departing from the principles and scope of this disclosure. The above modifications and other changes or modifications will be included in the scope of this invention.

[0181] The foregoing specific description has been described with reference to various embodiments. However, those skilled in the art will recognize that various modifications and changes can be made without departing from the scope of the present disclosure. Therefore, the consideration of the present disclosure will be illustrative rather than restrictive, and all these modifications will be included in its scope. Similarly, the advantages, other advantages and solutions to the problems of various embodiments have been described above. However, the benefits, advantages, solutions to the problems and any elements that can produce these, or make them more clear, should not be interpreted as critical, necessary or necessary. The term "include" and any other variants used in this article are all non-exclusive inclusions, so that the process, method, article or device including the list of elements not only includes these elements, but also includes other elements that are not explicitly listed or do not belong to the process, method, system, article or device. In addition, the term "coupled" and any other variants used in this article refer to physical connections, electrical connections, magnetic connections, optical connections, communication connections, functional connections and / or any other connections.

[0182] Those skilled in the art will appreciate that many changes may be made to the details of the above-described embodiments without departing from the basic principles of the invention. Therefore, the scope of the present invention should be determined solely by the claims.

Claims

1. A method for locating a barcode area, characterized in that: include: Get the original image; According to the original image, a corresponding labeling image is obtained; Tracing the contour of the region in the labeled image to obtain the contour information of the connected region in the image; According to the contour information, all contour points on the area contour are obtained; constructing a convex polygon according to the contour points; The two points with the greatest distance in the constructed convex polygon are determined as the two points of the diagonal of the rectangle; Determine two other points of the rectangle according to the two points of the diagonal line of the rectangle to form an initial rectangle; Segmenting each contour point according to the initial rectangle; Rectangle fitting is performed according to the segmentation result to determine a rectangular area from the connected area in the image as the barcode area.

2. A method for fitting a rectangular region of an image, characterized in that: include: Get the contour information of the connected area in the image; According to the contour information, all contour points on the area contour are obtained; constructing a convex polygon according to the contour points; The two points with the greatest distance in the constructed convex polygon are determined as the two points of the diagonal of the rectangle; Determine two other points of the rectangle according to the two points of the diagonal line of the rectangle to form an initial rectangle; Segmenting each contour point according to the initial rectangle; Rectangle fitting is performed according to the segmentation result to determine a rectangular area from the connected areas in the image.

3. The method according to claim 1 or 2, characterized in that The step of constructing a convex polygon according to the contour points comprises: Determine an index point from the contour points, and calculate the angles of all other contour points relative to the index point; Sort the contour points according to their angles relative to the index point; Construct a convex polygon based on the order of contour points.

4. The method according to claim 3, characterized in that The step of determining index points from the contour points comprises: The contour point with the smallest Y coordinate in the image is taken as the index point. If there are multiple contour points with the smallest Y coordinate, the contour point with the smallest or largest X coordinate is taken as the index point; or, the contour point with the largest Y coordinate in the image is taken as the index point. If there are multiple contour points with the largest Y coordinate, the contour point with the smallest or largest X coordinate is taken as the index point; or, the contour point with the largest X coordinate in the image is taken as the index point. If there are multiple contour points with the largest X coordinate, the contour point with the smallest or largest Y coordinate is taken as the index point; or, the contour point with the smallest X coordinate in the image is taken as the index point. If there are multiple contour points with the smallest X coordinate, the contour point with the smallest or largest Y coordinate is taken as the index point; The step of sorting the contour points according to the angles of the contour points relative to the index point comprises: According to the angle of each contour point relative to the index point, the contour points are sorted from large to small; or, according to the angle of each contour point relative to the index point, the contour points are sorted from small to large; If there are multiple contour points with the same angle as the index point, only the contour point farthest from the index point is retained for sorting.

5. The method according to claim 3, characterized in that The step of constructing a convex polygon according to the order of the contour points comprises: The contour points are sorted from small to large according to the angle relative to the index point, where the index point is the first point in the sort; The first and second points are placed in the stack, and the following iterative process is performed until the last point is reached: Determine the second top point and the first top point of the current stack, and the current point, wherein the current point is the next point in order relative to the first top point; Determine whether the rotation from the second point on the top of the stack to the first point on the top of the stack and from the first point on the top of the stack to the current point is counterclockwise; If it is rotating counterclockwise, the current point is pushed into the stack to update the stack; If it is not rotating counterclockwise, the first point on the top of the current stack is popped out of the stack, and the current point is pushed into the stack to update the stack; Determine whether the first point at the top of the updated stack is the last point in the sort; If it is judged to be the last point in the sort, the iteration is stopped, wherein all points in the updated stack can be connected in sequence to complete the construction of the convex polygon; If it is determined that it is not the last point in the sort, the next point in the sort relative to the updated first point on the top of the stack will be used as the current point, and it will be determined whether the rotation from the second point on the top of the stack to the first point on the top of the stack and from the first point on the top of the stack to the current point is counterclockwise.

6. The method according to claim 1 or 2, characterized in that: Determining two other points of the rectangle according to the two points on the diagonal of the rectangle includes: From the remaining contour points or the remaining points of the convex polygon, find the pixel point whose angle difference between the straight line formed by the two points of the diagonal line of the rectangle and 90 degrees is the smallest, as the remaining point in the rectangle.

7. The method according to claim 6, characterized in that Use the following formula to find the remaining points in the rectangle: Where (x i ,y i ) represents the coordinates of the i-th contour point, (x f ,y f ) and (x t ,y t ) respectively represent the coordinates of two points on the diagonal line of the rectangle; It means that a contour point is identified among the contour points that can maximize the value in the brackets, which is called the third point.

8. The method according to claim 6, characterized in that Determining the other two points of the rectangle according to the two points of the diagonal line of the rectangle also includes: The position of the found pixel point is corrected so that the angle between the corrected pixel point and the straight line formed by the two points of the diagonal line of the rectangle is 90 degrees.

9. The method according to claim 7, characterized in that Determining the other two points of the rectangle according to the two points of the diagonal line of the rectangle also includes: The position of the found pixel point is corrected so that the angle between the corrected pixel point and the straight line formed by the two points of the diagonal line of the rectangle is 90 degrees.

10. The method according to claim 9, characterized in that Correction is done by the following formula to obtain the remaining two points: where d max It means (x f ,y f ) and (x t ,y t )The distance between these two points; (x m ,y m ) means (x f ,y f ) and (x t ,y t )The coordinates of the midpoint of these two points; (x fit ,y fit ) represents the third point; d fit It means (x m ,y m ) and (x fit ,y fit ) is the distance between these two points.

11. The method according to claim 1 or 2, characterized in that: The segmenting of each contour point according to the initial rectangle comprises: Calculate the distance between each contour point and the four sides of the initial rectangle; According to the distance, the contour points are divided into the side with the shortest distance from the four sides, so as to divide the contour points into four groups, one group corresponding to each side.

12. The method according to claim 11, characterized in that The performing rectangle fitting according to the segmentation result includes: The corresponding edges are fitted with straight lines through the contour points that are divided into the same group to complete the fitting of the rectangle.

13. The method according to claim 12, characterized in that The straight line fitting is performed using the following formula: Construct the lines that form the four sides of the rectangle: ax+by+c=0,ax+by+d=0 bx-ay+e=0,bx-ay+f=0 And establish its objective function: where a 2 +b 2 =1; the contour points corresponding to the summation signs corresponding to each edge in the objective function are the contour points segmented to the corresponding edges.

14. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a program, which can be executed by a processor to implement the method according to any one of claims 1 to 13.

Images