Bill field matching method based on relative position structure

Abstract

The present invention provides a bill field matching method based on a relative position structure. First, each field value text and field name text in a bill image is detected to obtain multiple text detection frames and their coordinates. Then, the relative position feature codes of each text detection frame are calculated. Next, the feature distances between the relative position feature codes of each field value text detection frame and the relative position feature codes of each field name text detection frame are calculated. Finally, based on the multiple feature distances, the matching relationship between the field value text and the field name text is obtained, and the field value text is matched with the field name text with the smallest feature distance. Therefore, the method of the present invention can match the corresponding field name text and field value text based on the feature distance, that is, based on the relative position structure, effectively eliminating the adverse effects of the rotation and offset of the bill field value text printing on the bill field matching.

Description

Technical Field

[0001] The present invention relates to the technical field of bill information extraction, and in particular to a bill field matching method based on relative position structure. Background Art

[0002] In recent years, thanks to the rapid development of deep learning, text detection and recognition technology based on deep learning has been widely used in various practical scenarios.

[0003] When applying deep learning to bill recognition, the bill image is first fed into a text detection model to generate text detection boxes for each line of text within the bill image. The text recognition model then uses this detection box to identify the text within the boxes. In actual business needs, it's also necessary to match field names and values ​​within the bill. Field names are typically printed uniformly during bill production, while field values ​​are printed by a printer during invoice issuance.

[0004] Existing methods often use the invoice template to determine the location of each field value text, and then compare it with the position of the text detection box for matching. However, in real-world applications, the printed position of the field value text in the invoice image is often rotated or offset, which makes invoice field matching difficult. Summary of the Invention

[0005] To solve the above problems, a bill field matching method based on relative position structure is provided. This method uses the similarity of the relative position structure of field names and field values ​​in bill images to match field names and field values. This method achieves bill field matching without using templates and eliminates the effects of rotation and offset of the printed position of field value text on bill field matching. The present invention adopts the following technical solutions:

[0006] The present invention provides a bill field matching method based on relative position structure, characterized in that it includes: step S1, detecting each field value text and each field name text of the bill image, and obtaining multiple text detection frames and their coordinates, wherein the text detection frame includes a field value text detection frame and a field name text detection frame; step S2, respectively calculating the relative position feature coding of each of the text detection frames; step S3, calculating the feature distance between the relative position feature coding of each of the field value text detection frames and the relative position feature coding of each of the field name text detection frames; step S4, based on each of the feature distances, obtaining the matching relationship between the field value text and the field name text, wherein each of the field value texts is matched with the field name text having the smallest feature distance with the field value text.

[0007] The bill field matching method based on relative position structure provided by the present invention may also have such a technical feature, wherein step S2 includes the following sub-steps: step S2-1, for each of the text detection frames, calculating the vector from the upper left corner endpoint of the text detection frame to the upper right corner endpoint of the text detection frame as the text detection frame's own reference vector; step S2-2, for each of the text detection frames, calculating the vector from the upper left corner endpoint of the text detection frame to the upper left corner endpoint of each other text detection frame as the relative reference vector, thereby obtaining a set of the relative text detection frames. For the reference vector; Step S2-3, record the set of relative reference vectors obtained in Step S2-2 as a set of polar coordinates, wherein the polar angle of each polar coordinate is the counterclockwise angle between the relative reference vector and the corresponding self-reference vector of the text detection frame, and the polar diameter is the length of the relative reference vector; Step S2-4, normalize the polar diameter of each polar coordinate according to a predetermined normalization algorithm; Step S2-5, mark the normalized set of polar coordinates of each text detection frame as the relative position feature code of the text detection frame.

[0008] The bill field matching method based on relative position structure provided by the present invention may also have such a technical feature, wherein, in step S2-4, normalization is performed according to the following normalization formula:

[0009]

[0010] Where, ρ j is the polar radius of the polar coordinate j, ρ′ i is the normalized polar radius of the polar coordinate i, ρ i is the polar diameter of the polar coordinate i before normalization.

[0011] The bill field matching method based on relative position structure provided by the present invention may also have such a technical feature, wherein the two relative position feature codes in step S3 are respectively recorded as relative position feature code a and relative position feature code b, and step S3 includes the following sub-steps: step S3-1, respectively calculating the polar coordinate dissimilarity of each polar coordinate in the relative position feature code a and each polar coordinate in the relative position feature code b; step S3-2, performing bipartite graph matching on each polar coordinate in the relative position feature code a and each polar coordinate in the relative position feature code b according to the polar coordinate dissimilarity, wherein the polar coordinates with smaller polar coordinate dissimilarity are matched first; step S3-3, calculating the feature distance between the relative position feature code a and the relative position feature code b according to the matching result of step S3-2.

[0012] The bill field matching method based on relative position structure provided by the present invention may also have such a technical feature, wherein, in step S3-1, the polar coordinate dissimilarity is calculated according to the following formula:

[0013] vector_distance[i,j]=(1+de[i,j]) 2 +(1+dc[i,j]) 2 -2

[0014]

[0015] dc[i,j]=0.5+0.5*cos(θ i -θ j )

[0016] Wherein, vector_distance[i,j] is the polar coordinate dissimilarity, de[i,j] is the Euclidean distance between the i-th polar coordinate in the relative position feature code a and the j-th polar coordinate in the relative position feature code b, dc[i,j] is the cosine distance between the i-th polar coordinate in the relative position feature code a and the j-th polar coordinate in the relative position feature code b, and ρ i represents the polar diameter of the i-th polar coordinate, θ i represents the polar angle of the i-th polar coordinate.

[0017] The bill field matching method based on relative position structure provided by the present invention may also have such a technical feature, wherein, in step S3-3, the feature distance is calculated according to the following formula:

[0018] code_distance(a,b)=∑matched_distance(a,b,i)

[0019] Wherein, code_distance(a,b) is the feature distance, and matched_distance(a,b,i) is the polar coordinate dissimilarity between the i-th polar coordinate of the position feature code a and the polar coordinate matched in the position feature code b.

[0020] Functions and effects of the invention

[0021] According to the bill field matching method based on relative position structure of the present invention, first, each field value text and field name text in the bill image is detected to obtain multiple text detection frames and their coordinates; then, the relative position feature code of each text detection frame is calculated; then, the feature distance between the relative position feature code of each field value text detection frame and the relative position feature code of each field name text detection frame is calculated; finally, based on multiple feature distances, the matching relationship between the field value text and the field name text is obtained, and the field value text is matched with the field name text with the smallest feature distance. The field value text of the bill is inevitably subject to printing rotation, offset, etc. when the bill is printed, but the similar relationship between the relative position structure of the field name text and the relative position structure of the field value text still exists. Therefore, the method of the present invention can match the corresponding field name text and field value text based on the feature distance, that is, based on the relative position structure, effectively eliminating the adverse effects of the printing rotation and offset of the bill field value text on the bill field matching. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 is a flow chart of a bill field matching method based on a relative position structure in an embodiment of the present invention;

[0023] Figure 2 is a schematic diagram of a bill image in an embodiment of the present invention;

[0024] Figure 3 is a schematic diagram of a bill image with a text detection frame in an embodiment of the present invention;

[0025] Figure 4 Schematic diagram of the distribution position of the field name text detection box in an embodiment of the present invention;

[0026] Figure 5 Schematic diagram of the distribution position of field value text detection boxes in an embodiment of the present invention;

[0027] Figure 6 Schematic diagram of the reference vector of the field name text detection box in an embodiment of the present invention;

[0028] Figure 7 Schematic diagram of the reference vector of the field value text detection box in an embodiment of the present invention;

[0029] Figure 8 1 is a schematic diagram of relative position feature coding of a field name text detection frame in an embodiment of the present invention;

[0030] Figure 9 1 is a schematic diagram of relative position feature coding of a field value text detection box in an embodiment of the present invention;

[0031] Figure 101 is a schematic diagram of relative position feature coding of another field value text detection box in an embodiment of the present invention;

[0032] Figure 11 1 is a schematic diagram of a bipartite graph matching result of polar coordinates in two relative position feature codes in an embodiment of the present invention;

[0033] Figure 12 1 is a schematic diagram of the bipartite graph matching results of polar coordinates in two other relative position feature codes in an embodiment of the present invention;

[0034] Figure 13 It is a feature distance matrix of the relative position features of the field name text and the relative position features of the field value text in the embodiment of the present invention. DETAILED DESCRIPTION

[0035] In order to make the technical means, creative features, objectives and effects achieved by the present invention easy to understand, the following is a detailed description of the bill field matching method based on the relative position structure of the present invention in combination with the embodiments and drawings.

[0036] <Example>

[0037] Figure 1 This is a flow chart of a bill field matching method based on relative position structure in an embodiment of the present invention.

[0038] like Figure 1 As shown, in this embodiment, the bill field matching method based on the relative position structure specifically includes the following steps:

[0039] Step S1: Detect each field value and each field name of the bill image to obtain multiple text detection boxes and their coordinates.

[0040] Figure 2 Schematic diagram of a bill image in an embodiment of the present invention.

[0041] like Figure 2 As shown, the content of the bill image includes multiple texts, wherein the text content includes field name text and corresponding field value text. In this embodiment, the field value text in the bill image is printed rotated and offset, but the similar relationship between the relative position structure of the field name text and the field value text still exists.

[0042] Figure 3 Schematic diagram of a bill image with a text detection frame in an embodiment of the present invention.

[0043] like Figure 3As shown, the bill text detection method in the prior art is used to perform text detection on the bill image, multiple text detection boxes are marked on the bill image, and the coordinates of these text detection boxes are obtained, wherein the text detection boxes include the field name text detection box and the field value text detection box, and the bill text detection method in the prior art can also distinguish them.

[0044] Figure 4 Schematic diagram of the distribution position of the field name text detection box in an embodiment of the present invention.

[0045] Figure 5 Schematic diagram of the distribution position of the field value text detection box in an embodiment of the present invention.

[0046] like Figure 4 and Figure 5 As shown, the field name text is uniformly printed when the bill is produced, so its position is correct. The field value text is printed when the bill is issued, and there is printing rotation and offset. However, the relative position structure of the field value text is similar and corresponding to the relative position structure of the field name text.

[0047] Step S2: Calculate and obtain the relative position feature codes of each text detection frame.

[0048] In this embodiment, step S2 specifically includes the following sub-steps:

[0049] Step S2-1: For each text detection frame, calculate the vector from the upper left corner endpoint of the text detection frame to the upper right corner endpoint of the text detection frame as the reference vector of the text detection frame itself;

[0050] Step S2-2: For each text detection frame, calculate the vector from the upper left corner endpoint of the text detection frame to the upper left corner endpoint of each other text detection frame as a relative reference vector, thereby obtaining a set of relative reference vectors;

[0051] Step S2-3: Record the set of relative reference vectors obtained in step S2-2 as a set of polar coordinates, where the polar angle of each polar coordinate is the counterclockwise angle between the relative reference vector and the corresponding text detection box's own reference vector, and the polar diameter is the length of the relative reference vector;

[0052] Step S2-4: normalize the polar diameter of each polar coordinate according to a predetermined normalization algorithm. Specifically, normalize according to the following formula:

[0053]

[0054] Where, ρ j is the polar radius of the polar coordinate j, ρ′ i is the normalized polar radius of the polar coordinate i, ρ iis the polar diameter of the polar coordinate i before normalization.

[0055] Step S2-5: Mark the normalized set of polar coordinates of each text detection frame as the relative position feature code of the text detection frame.

[0056] Figure 6 Schematic diagram of the reference vector of the field name text detection box in an embodiment of the present invention.

[0057] Figure 7 Schematic diagram of the reference vector of the field value text detection box in an embodiment of the present invention.

[0058] like Figure 6 and Figure 7 As shown in FIG, the self-reference vectors of each text detection box are calculated according to the above method.

[0059] Figure 8 2 is a schematic diagram of relative position feature coding of a field name text detection box in an embodiment of the present invention.

[0060] like Figure 8 As shown in the figure, a set of relative reference vectors of a field name text detection frame is calculated according to the above method. Further, this set of relative reference vectors is recorded as a set of polar coordinates and normalized to obtain a relative position feature code, where the polar angle of the field name text detection frame's own reference vector is 0°, and the relative position feature code is represented as a set of vectors in the form of polar coordinates:

[0061] [(0.731,0.00°),(1.000,4.53°),(0.100,269.46°),(0.285,269.81°),(0.194,269.73°),(0.547,327.72°),(0.496,336.32°),(0.954,342.11°),(0.935,348.18°),(0.738,352.32°)].

[0062] Figure 9 2 is a schematic diagram of relative position feature coding of a field value text detection box in an embodiment of the present invention.

[0063] like Figure 9 As shown, similarly, the relative position feature code of a field value text detection frame is obtained according to the above method, wherein the polar angle of the reference vector of the field value text detection frame itself is -3.27°, and the relative position feature code is represented as a set of vectors in polar coordinate form:

[0064] [(0.963,3.87°),(0.215,260.57°),(0.309,264.06°),(0.603,327.44°),(0.545,335.65°),(1.000,342.02°),(0.988,346.44°),(0.831,351.90°),(0.819,358.35°)].

[0065] Figure 10 This is a schematic diagram of relative position feature coding of another field value text detection box in an embodiment of the present invention.

[0066] like Figure 10 As shown, the polar angle of the reference vector of the other field value text detection box is also -3.27°, and the relative position feature encoding is represented as a set of vectors in polar coordinate form:

[0067] [(0.511,157.74°),(0.473,168.88°),(0.498,179.51°),(0.507,191.09°),(0.093,271.55°),(0.512,33.34°),(0.359,32.37°),(0.327,17.47°),(0.437,0.21°),(0.453,347.86°)]

[0068] Step S3, calculating the feature distance between the relative position feature code of each field value text detection frame and the relative position feature code of each field name text detection frame.

[0069] In this embodiment, the two different relative position feature codes obtained by the above method are respectively recorded as relative position feature code a and relative position feature code b. Step S3 specifically includes the following sub-steps:

[0070] Step S3-1, respectively calculate the polar coordinate dissimilarity vector_distance[i,j] between each polar coordinate in the relative position feature code a and each polar coordinate in the relative position feature code b. Specifically, the calculation is performed according to the following formula:

[0071] vector_distance[i,j]=(1+de[i,j]) 2 +(1+dc[i,j]) 2 -2

[0072]

[0073] dc[i,j]=0.5+0.5*cos(θ i -θ j )

[0074] Where de[i,j] is the Euclidean distance between the i-th polar coordinate in the relative position feature code a and the j-th polar coordinate in the relative position feature code b, dc[i,j] is the cosine distance between the i-th polar coordinate in the relative position feature code a and the j-th polar coordinate in the relative position feature code b, and ρ i represents the polar diameter of the i-th polar coordinate, θ i represents the polar angle of the i-th polar coordinate;

[0075] Step S3-2, performing bipartite graph matching on each polar coordinate in the relative position feature code a and each polar coordinate in the relative position feature code b based on the polar coordinate dissimilarity obtained in step S3-1, wherein polar coordinates with smaller polar coordinate dissimilarity are matched first;

[0076] Step S3-3, calculating the feature distance code_distance(a,b) between the relative position feature code a and the relative position feature code b based on the matching result of step S3-2, specifically, calculating according to the following formula:

[0077] code_distance(a,b)=∑matched_distance(a,b,i)

[0078] Where matched_distance(a,b,i) is the polar coordinate dissimilarity between the i-th polar coordinate of position feature encoding a and the matching polar coordinate in position feature encoding b.

[0079] Figure 11 3 is a schematic diagram of the bipartite graph matching results of polar coordinates in two relative position feature codes in an embodiment of the present invention.

[0080] Figure 11 The left side shows the polar coordinates in the relative position feature encoding of a field name text detection box, and the right side shows the polar coordinates in the relative position feature encoding of a field value text detection box. The relative position feature distance between the field name text detection box and the field value text detection box is the sum of the polar coordinate dissimilarity of all matching polar coordinate pairs, which is 0.123.

[0081] Figure 12 3 is a schematic diagram of the bipartite graph matching results of polar coordinates in the other two relative position feature codes in an embodiment of the present invention.

[0082] Figure 12The left side shows the polar coordinates in the relative position feature encoding of another field name text detection box, and the right side shows the polar coordinates in the relative position feature encoding of another field value text detection box. The relative position feature distance between the field name text detection box and the field value text detection box is the sum of the polar coordinate dissimilarity of all matching polar coordinate pairs, which is 45.947.

[0083] Step S4, based on the characteristic distances obtained in step S3, obtain the matching relationship between the field value text and the field name text, wherein each field value text is matched with the field name text having the smallest characteristic distance with the field value text.

[0084] Figure 13 It is a feature distance matrix of the relative position features of the field name text and the relative position features of the field value text in the embodiment of the present invention.

[0085] like Figure 13 As shown, in this embodiment, the bill field matching result is:

[0086] Date: 2021 / 09 / 14

[0087] Project No.: SH135084321

[0088] Construction unit: A Construction Company

[0089] Project Name: L Road Reconstruction Project FXA68

[0090] Construction location: Ramp W2-7# platform Car number: 153

[0091] Departure time: 13:43

[0092] Vehicle volume: 16.00

[0093] Order number: m488551360905

[0094] Arrival time: 15:56

[0095] Cumulative volume: 34.00

[0096] In this embodiment, the bill fields are matched through the relative position relationship between the field name text detection box and the field value text detection box, which effectively eliminates the adverse effects of the printing rotation and offset of the bill field values ​​on the bill field matching and obtains accurate matching results.

[0097] Example Function and Effect

[0098] According to the bill field matching method based on relative position structure provided by this embodiment, first, each field value text and field name text in the bill image is detected to obtain multiple text detection frames and their coordinates; then, the relative position feature code of each text detection frame is calculated; then, the feature distance between the relative position feature code of each field value text detection frame and the relative position feature code of each field name text detection frame is calculated; finally, based on multiple feature distances, the matching relationship between the field value text and the field name text is obtained, and the field value text is matched with the field name text with the smallest feature distance. The field value text of the bill is inevitably subject to printing rotation, offset, etc. when the bill is printed, but the similarity between the relative position structure of the field name text and the relative position structure of the field value text still exists. Therefore, the method of the present invention can match the corresponding field name text and field value text based on feature distance, that is, based on the relative position structure, effectively eliminating the adverse effects of the printing rotation and offset of the bill field value text on the bill field matching.

[0099] In addition, in the above embodiment, corresponding formulas are given for the specific calculations of each step, so these steps can be fully automated using programming and other methods. Combined with machine learning methods, the bill fields can be fully automatically matched, thereby saving a lot of manpower.

[0100] The above embodiments are only used to illustrate specific implementations of the present invention, and the present invention is not limited to the description scope of the above embodiments.

Claims

1. A bill field matching method based on relative position structure, characterized in that: include: Step S1: Detect each field value text and each field name text of the bill image to obtain multiple text detection frames and their coordinates, wherein the text detection frames include a field value text detection frame and a field name text detection frame; Step S2, respectively calculating and obtaining the relative position feature codes of each of the text detection frames; Step S3, calculating the feature distances between the relative position feature codes a of each field value text detection frame and the relative position feature codes b of each field name text detection frame; Step S4: based on each of the characteristic distances, obtaining a matching relationship between the field value text and the field name text, wherein each of the field value texts is matched with the field name text having the smallest characteristic distance with the field value text. The relative position feature code a and the relative position feature code b are respectively represented as a set of vectors in polar coordinate form, Step S3 includes the following sub-steps: Step S3-1, respectively calculating the polar coordinate dissimilarity between each polar coordinate in the relative position feature code a and each polar coordinate in the relative position feature code b; Step S3-2, performing bipartite graph matching on each polar coordinate in the relative position feature code a and each polar coordinate in the relative position feature code b according to the polar coordinate dissimilarity, wherein polar coordinates with smaller polar coordinate dissimilarity are matched first; Step S3-3, calculating the characteristic distance between the relative position characteristic code a and the relative position characteristic code b according to the matching result of step S3-2, In step S3-1, the polar coordinate dissimilarity is calculated according to the following formula: vcctor_distance[i,j]=(1+de[i,j]) 2 +(1+dc[i,j]) 2 -2 dc[i,j]=0.5+0.5*cos(θ i -θ j ) Wherein, vector_distance[i,j] is the polar coordinate dissimilarity, de[i,j] is the Euclidean distance between the i-th polar coordinate in the relative position feature code a and the j-th polar coordinate in the relative position feature code b, dc[i,j] is the cosine distance between the i-th polar coordinate in the relative position feature code a and the j-th polar coordinate in the relative position feature code b, and ρ i represents the polar diameter of the i-th polar coordinate, θ i represents the polar angle of the i-th polar coordinate, In step S3-3, the characteristic distance is calculated according to the following formula: code_distance(a,b)=∑matched_distance(a,b,i) Wherein, code_distance(a,b) is the feature distance, and matched_distance(a,b,i) is the polar coordinate dissimilarity between the i-th polar coordinate of the position feature code a and the polar coordinate matched in the position feature code b.

2. The bill field matching method based on relative position structure according to claim 1, Its characteristics are: Wherein, step S2 includes the following sub-steps: Step S2-1: For each text detection frame, calculate a vector from the upper left corner endpoint of the text detection frame to the upper right corner endpoint of the text detection frame as the reference vector of the text detection frame itself; Step S2-2: for each text detection frame, calculate a vector from the upper left corner endpoint of the text detection frame to the upper left corner endpoint of each other text detection frame as a relative reference vector, thereby obtaining a set of relative reference vectors; Step S2-3, recording the set of relative reference vectors obtained in step S2-2 as a set of polar coordinates, wherein the polar angle of each polar coordinate is the counterclockwise angle between the relative reference vector and the corresponding self-reference vector of the text detection box, and the polar diameter is the length of the relative reference vector; Step S2-4, normalizing the polar diameter of each polar coordinate according to a predetermined normalization algorithm; Step S2-5: Marking the normalized set of polar coordinates of each text detection frame as the relative position feature code of the text detection frame.

3. The bill field matching method based on relative position structure according to claim 2, characterized in that: in, In step S2-4, normalization is performed according to the following normalization formula: Where, ρ j is the polar radius of coordinate j, ρ' i is the normalized polar radius of the polar coordinate i, ρ i is the polar diameter of the polar coordinate i before normalization.

Images