A multi-branch license key field and content segmentation and extraction method based on a multi-language BERT model

Abstract

The application is a kind of multi-branch license key field and content segmentation and extraction method based on multi-language BERT model.The application relates to the technical field of artificial intelligence, and the application extracts the text information and text position information in the license image through an OCR tool; the text recognition result is divided into lines according to the text position information; a Chinese word segmentation tool is used to perform word segmentation processing on each text line to obtain a word list containing the word information in the line; the word list is divided into two parts for processing a plurality of times, and the two parallel BERT model branches are processed to realize probability prediction; the probability prediction result of the BERT is processed through a full connection layer, and whether it is a correct segmentation position is determined according to the output result.The application realizes the segmentation and extraction of the key field content in the license file only through semantic information, can avoid the influence of different complex and variable layouts of licenses on recognition, has better universality and stronger feasibility.

Description

TECHNICAL FIELD

[0001] The application relates to the technical field of artificial intelligence, and is a multi-branch certificate key field and content segmentation extraction method based on a multi-language BERT model. BACKGROUND

[0002] In government data processing, key field extraction of a certificate image is often required. Key fields and contents in a certificate file exist in the form of key-value and have strong structural information, but the layout is various and has no rules. Direct OCR recognition of the certificate image is often limited by the variable layout, so that automatic extraction of the key field cannot be realized. Common certificate key field extraction is realized through an artificial method. However, the artificial extraction method is time-consuming and laborious, and the efficiency is not high. The timeliness of a large number of file processing is difficult to guarantee. Another method is to pre-register each type of certificate to be processed, mark the relative positions of the corresponding key fields and contents in the certificate, and realize automatic extraction of the contents after one-time registration. However, the method has the defects that the certificate type needs to be determined before recognition, and the unregistered certificate cannot be effectively recognized. SUMMARY

[0003] In order to overcome the defects of the prior art, the application extracts the text information and the text position information in the certificate image through OCR. The text recognition result is divided into text lines according to the position information. The text lines are subjected to word segmentation processing. The word list is segmented, and the probability is predicted through the BERT model. The probability prediction result is processed through the full connection layer, and the segmentation position of the key field and the content is determined. The application provides a multi-branch certificate key field and content segmentation extraction method based on a multi-language BERT model. The application provides the following technical scheme:

[0004] A multi-branch certificate key field and content segmentation extraction method based on a multi-language BERT model, the method comprising the following steps:

[0005] Step 1: extracting the text information and the text position information in the certificate image through an OCR tool;

[0006] Step 2: dividing the text recognition result into lines according to the text position information;

[0007] Step 3: performing word segmentation processing on each text line through a Chinese word segmentation tool to obtain a word list containing the word information in the line;

[0008] Step 4: dividing the word list into two parts for processing multiple times, and processing through two parallel BERT model branches to realize probability prediction;

[0009] Step 5: Process the probability prediction results of BERT through a full connection layer, and determine whether it is a correct segmentation position according to the output results.

[0010] Preferably, the step 1 is specifically:

[0011] The position information corresponding to the character is represented by (x0, y0, x1, y1), wherein x0 and y0 represent the coordinates of the top-left corner point of the character anchor frame in the image, and x1 and y1 represent the coordinates of the bottom-right corner point in the image.

[0012] Preferably, the step 2 is specifically:

[0013] Step 2.1: Sort all characters according to the y coordinate information of the character center position, and take the first character as the first character of the new line;

[0014] Step 2.2: Traverse the remaining characters according to the first character, and when the difference between the y coordinates of the center points is less than 1 / 2 of the height of the first character, the character is added to the line until there is no character that meets the condition, and the character arranged first among the remaining characters is taken as the first character of the new line;

[0015] Step 2.3: Repeat the above process until all characters are added to the line.

[0016] Preferably, after being divided into lines, the characters in each line are sorted from left to right according to the x coordinates of the character center positions, as a continuous character line.

[0017] Preferably, the step 3 further includes:

[0018] A placeholder is added at the beginning and end of the word segmentation list to deal with the case where a line of characters is neither a key field nor the content corresponding to the key field.

[0019] Preferably, the step 4 is specifically:

[0020] For a list containing n words and 2 placeholders, there are n+1 intervals, and n+1 divisions are performed; wherein the division method is: from the first interval, all placeholders or words before the interval are spliced as the first part of the to-be-processed corpus, and all words or placeholders after the interval are spliced as the second part of the to-be-processed corpus.

[0021] Each piece of corpus is obtained by a pre-trained BERT model to obtain a three-dimensional vector, represented as [p o ,p k ,p v ] T , p k represents the probability of being a key field, and pv represents the probability that the content corresponding to the key field, wherein p o represents the probability that the content corresponding to the key field, wherein p

[0022] Preferably, the step 5 is specifically:

[0023] The probability prediction results obtained by the two branches are spliced to obtain a 6-dimensional vector, denoted as [p o1 , p k1 , p v1 , p o2 , p k2 , p v2 ] T ;

[0024] The spliced probability prediction results are processed by three full connection layers; the number of neurons of the three full connection layers is 16, 6, and 2, respectively;

[0025] The output of the last full connection layer is a 2-dimensional vector, which is [0, 1] T or [1, 0] T , respectively corresponding to correct segmentation of the key field and the content and incorrect segmentation of the key field and the content.

[0026] A multi-branch license key field and content segmentation extraction system based on a multi-language BERT model, the system comprising:

[0027] An extraction module, which extracts text information and text position information in a license image by an OCR tool;

[0028] An identification module, which divides the text recognition result in units of lines according to the text position information;

[0029] A word segmentation module, which performs word segmentation processing on each text line by a Chinese word segmentation tool to obtain a word list containing the word information in the line;

[0030] A division prediction module, which divides the word list into two parts for processing multiple times, processes by two parallel BERT model branches, and realizes probability prediction;

[0031] A segmentation module, which determines whether it is a correct segmentation position according to the output result according to the probability prediction result of the BERT processed by the full connection layer.

[0032] A computer readable storage medium having a computer program stored thereon, the program being executed by a processor to implement the multi-branch license key field and content segmentation extraction method based on the multi-language BERT model according to any one of claims 1-7.

[0033] A computer device comprises a memory and a processor, the memory stores a computer program, when the processor runs the computer program stored in the memory, the processor executes the multi-branch license key field and content segmentation extraction method based on the multi-language BERT model according to any one of claims 1-7.

[0034] The present application has the following beneficial effects:

[0035] The present application provides a multi-branch license key field and content segmentation extraction method based on a multi-language BERT model. The present application realizes the segmentation and extraction of key field content in the license file only through semantic information by performing multiple segmentation combinations on the text in the license, judging the category probability by the pre-trained BERT model, and judging whether the segmentation is effective by the full connection layer, which can avoid the influence of different complex and variable layouts of licenses on recognition, has better universality and stronger feasibility. BRIEF DESCRIPTION OF DRAWINGS

[0036] In order to more clearly illustrate the specific embodiments of the present application or the technical solutions in the prior art, the drawings needed to be used in the specific embodiments or prior art description will be briefly introduced below. Obviously, the drawings in the following description are some embodiments of the present application, and those skilled in the art can obtain other drawings according to these drawings without creative labor.

[0037] Figure 1 The flowchart of the multi-branch license key field and content segmentation extraction method based on the multi-language BERT model;

[0038] Figure 2 The result of dividing the text line;

[0039] Figure 3 The result of dividing the text line;

[0040] Figure 4 The segmentation of the word list;

[0041] Figure 5 The probability prediction;

[0042] Figure 6 The full connection layer processing process. DETAILED DESCRIPTION

[0043] The technical solutions of the present application will be described below in conjunction with the drawings. Obviously, the described embodiments are part of the embodiments of the present application, not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0044] In the description of the present application, it should be noted that the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer" and the like indicate the orientation or positional relationship shown in the drawings, which are only for the convenience of describing the present application and simplifying the description, and do not indicate or imply that the devices or elements referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as a limitation of the present application. In addition, the terms "first", "second", "third" are only for descriptive purposes and cannot be understood as indicating or implying relative importance.

[0045] In the description of the present application, it should be noted that unless otherwise explicitly specified and limited, the terms "mounting", "connecting", "connecting" should be understood broadly, for example, it can be fixedly connected, or it can be detachably connected, or integrally connected; it can be mechanically connected, or it can be electrically connected; it can be directly connected, or it can be indirectly connected through an intermediate medium, or it can be the communication inside two elements. For those skilled in the art, the specific meaning of the above terms in the present application can be understood according to the specific circumstances.

[0046] In addition, the technical features involved in the different embodiments of the application described below can be combined with each other as long as there is no conflict.

[0047] The present application is described in detail below in combination with specific embodiments. Specific embodiment one:

[0049] According to Figures 1-6 The specific optimization technical scheme adopted by the present application to solve the above technical problems is: the present application relates to a multi-branch license key field and content segmentation extraction method based on a multi-language BERT model.

[0050] A multi-branch license key field and content segmentation extraction method based on a multi-language BERT model, characterized in that: the method comprises the following steps:

[0051] Step 1: extract the text information and text position information in the license image through the OCR tool;

[0052] Step 2: divide the text recognition result into units according to the text position information;

[0053] Step 3: perform word segmentation processing on each text line through a Chinese word segmentation tool to obtain a word list containing the word information in the line;

[0054] Step 4: divide the word list into two parts for processing multiple times, and process through two parallel BERT model branches to realize probability prediction;

[0055] Step 5: Process the probability prediction results of BERT through a fully connected layer, and determine whether it is the correct segmentation position based on the output results. Specific Implementation Example 2:

[0057] The only difference between Embodiment 2 and Embodiment 1 of this application is that:

[0058] Step 1 specifically involves:

[0059] The position information corresponding to the text is represented by (x0, y0, x1, y1), where x0 and y0 represent the coordinates of the upper left corner of the text anchor box in the image, and x1 and y1 represent the coordinates of the lower right corner in the image. Specific Implementation Example 3:

[0061] The only difference between Embodiment 3 and Embodiment 2 of this application is that:

[0062] Step 2 specifically involves:

[0063] Step 2.1: Sort all the text according to the y-coordinate of the center position of the text, and take the first text as the first text of the new line;

[0064] Step 2.2: Traverse the remaining texts based on the first text. When the merging condition is met that the difference between the y-coordinates of the center points is less than 1 / 2 of the height of the first text, add the text to the line. Continue until there are no more texts that meet the condition, and then take the first text among the remaining texts as the first text of the new line.

[0065] Step 2.3: Repeat the above process until all text has been added to the line, as shown in the image. Figure 2 As shown. Specific Implementation Example 4:

[0067] The only difference between Embodiment 4 and Embodiment 3 of this application is that:

[0068] After dividing the text into rows, within each row, the text is sorted from left to right according to the x-coordinate corresponding to the center position of each character, forming consecutive lines of text. Specific Example 5:

[0069] The only difference between Embodiment 5 and Embodiment 4 of this application is that:

[0070] Step 3 also includes:

[0071] Add a placeholder at the beginning and end of the word segmentation list to handle situations where a line of text is neither a key field nor its corresponding content. The result is as follows: Figure 3 As shown. Specific Implementation Example Six:

[0073] The difference between embodiment six and embodiment five is only that:

[0074] The step 4 is specifically:

[0075] For a list containing n words and 2 placeholders, n+1 intervals are included, and n+1 divisions are performed; wherein, the division method is: from the first interval, all placeholders or words before the interval are spliced as the first part of the to-be-processed corpus, and all words or placeholders after the interval are spliced as the second part of the to-be-processed corpus, and one division case is as shown in Figure 4

[0076] Each piece of corpus obtains a three-dimensional vector through a pre-trained BERT model, and is represented as [p o ,p k ,p v ] T , p k represents the probability of being a key field, p v represents the probability of being a content corresponding to the key field, and p o represents the probability of being other, as shown in Figure 5 Specific embodiment seven:

[0078] The difference between embodiment seven and embodiment six is only that:

[0079] The step 5 is specifically:

[0080] The probability prediction results obtained by the two branches are spliced to obtain a six-dimensional vector, represented as [p o1 ,p k1 ,p v1 ,p o2 ,p k2 ,p v2 ] T ;

[0081] The spliced probability prediction result is processed through three fully connected layers; the neuron numbers of the three fully connected layers are 16, 6, and 2 respectively;

[0082] The output of the last fully connected layer is a two-dimensional vector, which is [0, 1] T or [1, 0] T , respectively corresponding to correct segmentation of the key field and the content, and incorrect segmentation of the key field and the content, and the processing process is as shown in Figure 6 Specific embodiment eight:

[0084] The difference between embodiment eight and embodiment seven is only that:

[0085] ​​​The application provides a multi-branch license key field and content segmentation and extraction system based on a multi-language BERT model.

[0086] An extraction module extracts text information and text position information in a license image through an OCR tool.

[0087] An identification module divides text recognition results in a line unit according to the text position information.

[0088] A word segmentation module performs word segmentation processing on each text line through a Chinese word segmentation tool to obtain a word list containing word information in the line.

[0089] A division prediction module divides the word list into two parts for processing multiple times, and performs processing through two parallel BERT model branches to realize probability prediction.

[0090] A segmentation module determines whether it is a correct segmentation position according to an output result according to the probability prediction result of the BERT processed by the full connection layer. Specific embodiment nine:

[0092] The difference between the ninth embodiment and the eighth embodiment is only that:

[0093] The application provides a computer readable storage medium, which stores a computer program, and the program is executed by a processor to implement a multi-branch license key field and content segmentation and extraction method based on a multi-language BERT model. Specific embodiment ten:

[0095] The difference between the tenth embodiment and the ninth embodiment is only that:

[0096] The application provides a computer device, which comprises a memory and a processor, and the memory stores a computer program, and when the processor runs the computer program stored in the memory, the processor executes a multi-branch license key field and content segmentation and extraction method based on a multi-language BERT model.

[0097] The above is only a preferred embodiment of the multi-branch license key field and content segmentation and extraction method based on the multi-language BERT model, and the protection scope of the multi-branch license key field and content segmentation and extraction method based on the multi-language BERT model is not limited to the above-mentioned embodiments. Any technical solution belonging to the idea is within the protection scope of the application. It should be noted that for those skilled in the art, some improvements and changes without departing from the principles of the application are also considered as the protection scope of the application.

Claims

1. A multi-branch license key field and content segmentation extraction method based on a multi-language BERT model, characterized by: The method comprises the following steps: Step 1: extracting the text information and the text position information in the license image through an OCR tool; Step 2: dividing the text recognition result into lines according to the text position information; The step 2 is specifically: Step 2.1: sorting all the texts according to the y coordinate information of the text center position, and taking the first text as the first text of a new line; Step 2.2: traversing the remaining texts according to the first text, and when the difference between the y coordinates of the center points is less than 1 / 2 of the height of the first text, the text is added to the line until there is no text meeting the condition, and then the text arranged first among the remaining texts is taken as the first text of a new line; Step 2.3: repeating the above process until all the texts are added to the line; Step 3: performing word segmentation processing on each text line through a Chinese word segmentation tool to obtain a word list containing the word information in the line; Step 4: dividing the word list into two parts for processing multiple times, processing through two parallel BERT model branches to realize probability prediction; The step 4 is specifically: For a list containing n words and 2 placeholders, there are n+1 intervals, and a total of n+1 divisions are performed; wherein, the division method is: from the first interval, all the placeholders or words before the interval are spliced as the first part of the processing data, and all the words or placeholders after the interval are spliced as the second part of the processing data; Each piece of corpus gets a three-dimensional vector by pre-trained BERT model, denoted as , denotes the probability of being a key field, denotes the probability of being the content corresponding to the key field, where denotes the probability of being other; Step 5: processing the probability prediction result of BERT through a fully connected layer to determine whether it is a correct segmentation position according to the output result; The step 5 is specifically: The probability prediction results obtained by the two branches are spliced to obtain a 6-dimensional vector, denoted as ; The spliced probability prediction result is processed through three fully connected layers; the number of neurons of the three fully connected layers is 16, 6 and 2 respectively. The output of the last fully connected layer is a 2-dimensional vector, which is or , respectively, corresponding to correct segmentation of the key field and content, and incorrect segmentation of the key field and content.

2. The multi-branch license key field and content segmentation and extraction method based on the multi-language BERT model according to claim 1, characterized in that: The step 1 is specifically: The position information corresponding to the text is represented by (x0, y0, x1, y1), wherein x0 and y0 represent the coordinates of the left upper corner point of the text anchor frame in the image, and x1 and y1 represent the coordinates of the right lower corner point in the image.

3. The multi-branch license key field and content segmentation and extraction method based on the multi-language BERT model according to claim 2, characterized in that: After being divided into lines, the texts in each line are sorted from left to right according to the x coordinates of the text center positions as continuous text lines.

4. The multi-branch license key field and content segmentation extraction method based on the multi-language BERT model according to claim 3, characterized in that: The step 3 further comprises: A placeholder is added at the beginning and end of the word segmentation list to cope with the case that a line of text is neither a key field nor the content corresponding to the key field.

5. A multi-branch license key field and content segmentation and extraction system based on a multi-language BERT model, the system is run based on the multi-branch license key field and content segmentation and extraction method based on a multi-language BERT model according to any one of claims 1-4, characterized in that: The system comprises: An extraction module which extracts the text information and the text position information in the license image through an OCR tool; An identification module which divides the text recognition result into lines according to the text position information; A word segmentation module which performs word segmentation processing on each text line through a Chinese word segmentation tool to obtain a word list containing the word information in the line; A division prediction module which divides the word list into two parts for processing multiple times, processes through two parallel BERT model branches to realize probability prediction; The segmentation module processes the probability prediction result of the BERT according to a full connection layer, and determines whether it is a correct segmentation position according to an output result.

6. A computer-readable storage medium having stored thereon a computer program, characterized in that, The program is executed by a processor to implement the multi-branch license key field and content segmentation extraction method based on the multi-language BERT model.

7. A computer device, comprising: The application discloses a computer readable storage medium, which comprises a memory and a processor, and the memory stores a computer program. When the processor runs the computer program stored in the memory, the processor executes the multi-branch license key field and content segmentation extraction method based on the multi-language BERT model.

Images