Form information extraction method and system

Abstract

The invention discloses a form information extraction method and system, and the method comprises the steps: obtaining a form image, carrying out the character recognition of all character pre-selection boxes on the form image through a preset text recognition technology, obtaining a character recognition result, building a first image according to the character recognition result, carrying out the histogram comparison of the first image and a second image, and obtaining a form information extraction result. The second image is a region image of the whole text region in the form image, if the histogram comparison result meets the requirements of a filter, generating target text data, and if the histogram comparison result does not meet the requirements of the filter, generating target text data; and extracting an image region from all the character pre-selection boxes on the form image, and performing single character detection to output target text data.

Description

Technical Field

[0001] The present application relates to the field of information processing technology, and more specifically, to a form information extraction method and system. Background Art

[0002] Text is an indispensable source of visual information. Compared with other content in images / videos, text often contains stronger semantic information. Therefore, text extraction and recognition in images is of great significance and is a very important research topic that affects practical applications in many fields.

[0003] For example, in actual scenarios, when inputting data based on paper documents from customers, staff need to carefully input and check whether the data is accurate, which takes a long time and requires staff to concentrate highly. Mistakes are prone to occur during long hours of work, especially in financial scenarios such as bank transfers, where the risk is relatively high. In addition, due to the huge number of Chinese characters, a large number of font types, image deformation, similar characters, and rare characters, the recognition ability of Chinese characters is relatively weak. In natural scenes, the lighting is uneven, and the handwriting fonts of customers vary greatly, which affects the accuracy of the text recognition algorithm, thereby reducing the accuracy of form image information extraction.

[0004] Therefore, how to improve the accuracy of form image information extraction is a problem that needs to be solved urgently in this application. Summary of the invention

[0005] In view of this, the present application discloses a form information extraction method and system, aiming to improve the accuracy of text recognition and improve the accuracy of paper form image information extraction.

[0006] In order to achieve the above purpose, the disclosed technical solution is as follows:

[0007] The first aspect of the present application discloses a form information extraction method, the method comprising:

[0008] Get the form image;

[0009] Perform text recognition on all text pre-selected boxes on the form image using a preset text recognition technology to obtain a text recognition result;

[0010] Creating a first image according to the text recognition result;

[0011] Performing a histogram comparison on the first image and the second image to obtain a histogram comparison result; wherein the second image is a region image of the entire text region in the form image;

[0012] If the histogram comparison result meets the filter requirements, generating target text data;

[0013] If the histogram comparison result does not meet the filter requirement, all the text pre-selected boxes on the form image are extracted into image areas and single word detection is performed to output target text data.

[0014] Preferably, the text recognition is performed on all text pre-selection boxes on the form image by using a preset text recognition technology to obtain a text recognition result, including:

[0015] Extracting image features from the form image and enhancing the image features;

[0016] In the target detection stage, each text pre-selection box in the enhanced image features is obtained through the target detection algorithm model, and the position information of each text pre-selection box is stored;

[0017] According to the position information of each text pre-selection box, adjacent text pre-selection boxes in each text pre-selection box are merged to obtain a whole text area;

[0018] In the text recognition stage, the convolutional neural network model is used to recognize the text in the entire text area to obtain the text recognition result.

[0019] Preferably, performing a histogram comparison on the first image and the second image to obtain a histogram comparison result includes:

[0020] calculating a first histogram of the first image, and calculating a second histogram of the second image;

[0021] Normalizing the first histogram and the second histogram to the same scale space;

[0022] The Bhattacharyya distance between the first histogram and the second histogram is calculated in the same scale space to complete the process of obtaining the histogram comparison result.

[0023] Preferably, if the histogram comparison result meets the filter requirement, generating target text data includes:

[0024] If the Bhattacharyya distance in the histogram comparison result is greater than or equal to a preset threshold, determining that the histogram comparison result does not meet the filter requirement;

[0025] If the Bhattacharyya distance in the histogram comparison result is less than the preset threshold, it is determined that the histogram comparison result meets the filter requirement.

[0026] Preferably, if the histogram comparison result does not meet the filter requirement, extracting the image area from all the text pre-selection boxes on the form image and performing single word detection to output the target text data includes:

[0027] If the histogram comparison result does not meet the filter requirement, extracting the corresponding image area from each text pre-selected box in the enhanced image feature;

[0028] For the image area extracted from each text pre-selection box, the image area is input into a convolutional neural network for single word detection and a corresponding serial number is generated; wherein the serial number is determined by the order of the text data in the form image;

[0029] Until the single word detection is completed in the image area of ​​all the text pre-selection boxes, the text data corresponding to all the serial numbers are integrated and the target text data is output.

[0030] The second aspect of the present application discloses a form information extraction system, the system comprising:

[0031] An acquisition unit, used for acquiring a form image;

[0032] The recognition unit is used to perform text recognition on all text pre-selected boxes on the form image by using a preset text recognition technology to obtain a text recognition result;

[0033] A creating unit, used for creating a first image according to the text recognition result;

[0034] A comparison unit, configured to perform a histogram comparison on the first image and the second image to obtain a histogram comparison result; wherein the second image is a region image of the entire text region in the form image;

[0035] A generating unit, configured to generate target text data if the histogram comparison result meets the filter requirement;

[0036] The detection unit is used to extract the image area from all the text pre-selection boxes on the form image and perform single word detection to output the target text data if the histogram comparison result does not meet the filter requirement.

[0037] Preferably, the identification unit comprises:

[0038] An enhancement module, used for extracting image features from the form image and enhancing the image features;

[0039] The first acquisition module is used to acquire each text pre-selection box in the enhanced image features through the target detection algorithm model during the target detection stage, and store the position information of each text pre-selection box;

[0040] A merging module, used for merging adjacent text pre-selection boxes in each text pre-selection box according to the position information of each text pre-selection box to obtain a whole text area;

[0041] The recognition module is used to recognize the text in the entire text area through the convolutional neural network model in the text recognition stage to obtain the text recognition result.

[0042] Preferably, the comparison unit comprises:

[0043] A first calculation module, used for calculating a first histogram of the first image and calculating a second histogram of the second image;

[0044] A normalization module, used for normalizing the first histogram and the second histogram to the same scale space;

[0045] The second calculation module is used to calculate the Bhattacharyya distance between the first histogram and the second histogram in the same scale space to complete the process of obtaining the histogram comparison result.

[0046] Preferably, the generating unit comprises:

[0047] A first determination module, configured to determine that the histogram comparison result does not meet the filter requirement if the Bhattacharyya distance in the histogram comparison result is greater than or equal to a preset threshold;

[0048] The second determination module is used to determine whether the histogram comparison result meets the filter requirement if the Bhattacharyya distance in the histogram comparison result is less than the preset threshold.

[0049] Preferably, the detection unit comprises:

[0050] An extraction module, for extracting a corresponding image region from each text pre-selected box in the enhanced image feature if the histogram comparison result does not meet the filter requirement;

[0051] A detection module, for inputting the image area extracted from each text pre-selection box into a convolutional neural network for single word detection and generating a corresponding serial number; wherein the serial number is determined by the order of the text data in the form image;

[0052] The integration output module is used to complete the single word detection in the image area of ​​all the text pre-selection boxes, integrate the text data corresponding to all the serial numbers obtained, and output the target text data.

[0053] Through the above technical scheme, it can be known that a form information extraction method and system obtains a paper form image, obtains a form image, performs text recognition on all text pre-selection boxes on the form image through a preset text recognition technology, obtains a text recognition result, establishes a first image according to the text recognition result, compares the first image with the second image by histogram, and obtains a histogram comparison result, wherein the second image is a regional image of the entire text area in the form image, if the histogram comparison result meets the filter requirements, generates target text data, if the histogram comparison result does not meet the filter requirements, extracts all text pre-selection boxes on the form image to the image area and performs single word detection to output the target text data. Through the above scheme, a histogram comparison method is added on the basis of the preset text recognition technology, and a secondary judgment is made on the form image, which can reduce the impact of problems such as uneven illumination on the image and improve the accuracy of text recognition. At the same time, a threshold-based filter is added to perform single word detection on the results that do not meet the filter requirements to optimize the text recognition results and improve the accuracy of form image information extraction. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0055] Figure 1 A flowchart of a form information extraction method disclosed in an embodiment of the present application;

[0056] Figure 2 A schematic diagram of similarity processing between a detection image and a template image disclosed in an embodiment of the present application;

[0057] Figure 3 A schematic diagram comparing the effects of the solution disclosed in the embodiment of the present application and the traditional method;

[0058] Figure 4 A flowchart of another form information extraction method disclosed in an embodiment of the present application;

[0059] Figure 5 A diagram of a form information extraction device disclosed in an embodiment of the present application;

[0060] Figure 6 A schematic diagram of the structure of a form information extraction system disclosed in an embodiment of the present application. DETAILED DESCRIPTION

[0061] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0062] In this application, the terms "comprises", "comprising" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprising a ..." does not exclude the presence of other identical elements in the process, method, article or device comprising the element.

[0063] As can be seen from the background technology, in actual scenarios, when inputting data based on the customer's paper documents, the staff needs to carefully input and check whether the data is accurate, which takes a long time and requires the staff to concentrate highly. Mistakes are prone to occur during long working hours, especially in financial scenarios such as bank transfers, where the risk is relatively high. In addition, due to the huge number of Chinese characters, the large number of font types, image deformation, similar characters, and rare characters, the recognition ability of Chinese characters is relatively weak. In natural scenes, the lighting is uneven, and the handwriting fonts of customers vary greatly, which affects the accuracy of the text recognition algorithm, thereby reducing the accuracy of the information extraction of paper form images. Therefore, how to improve the accuracy of information extraction of paper form images is a problem that needs to be solved in this application.

[0064] In order to solve the above problems, the present application discloses a form information extraction method and system. A histogram comparison method is added on the basis of a preset text recognition technology to perform a secondary judgment on the form image, which can reduce the impact of uneven lighting and other problems on the image and improve the accuracy of text recognition. At the same time, a threshold-based filter is added to perform single-word detection on the results that do not meet the filter requirements to optimize the text recognition results and improve the accuracy of form image information extraction. The specific implementation method is specifically described through the following embodiments.

[0065] refer to Figure 1 As shown, a form information extraction method disclosed in an embodiment of the present application is provided, and the form information extraction method mainly includes the following steps:

[0066] S101: Acquire a form image.

[0067] The type of the form image may be a paper form image, a non-paper form image, etc. The type of the form image is not specifically limited in this application.

[0068] S102: Performing text recognition on all text pre-selected boxes on the form image using a preset text recognition technology to obtain a text recognition result.

[0069] The preset text recognition technology includes but is not limited to optical character recognition (OCR).

[0070] Among them, OCR refers to the process in which an electronic device (such as a scanner or digital camera) examines characters printed on paper and uses character recognition methods to translate shapes into computer text. That is, the process of scanning text materials, then analyzing and processing image files, obtaining text and layout information, and returning it in the form of text. It mainly consists of two parts: target detection and text recognition. Target detection is to detect the location, range and layout of the text, and usually also includes layout analysis and text line detection; text recognition is to recognize the text content based on target detection, and convert the text information in the image into text information suitable for computers.

[0071] The preset text recognition technology consists of two parts: object detection and text recognition.

[0072] The target detection algorithm (You Only Look Once version 4, YOLO4) model is used to detect the text pre-selection box of each word in the paper form, merge the adjacent text pre-selection boxes into the same text area (the entire text area), and save the coordinate information of the text pre-selection box and the entire text area.

[0073] The text recognition part uses the Convolutional Neural Networks (CRNN) model to identify the text in the output text area in the target detection. The CRNN model is a convolutional recurrent neural network structure, which is mainly used to recognize text sequences of indefinite length end-to-end. Instead of cutting individual characters first, text recognition is converted into a time-dependent sequence learning problem. Image-based sequence recognition can recognize longer text sequences.

[0074] Specifically, the process of performing text recognition on all text pre-selected boxes on the paper form image by using the preset text recognition technology to obtain the text recognition result is shown in A1-A4.

[0075] A1: Extract image features from the form image and enhance the image features.

[0076] In A1, operations such as grayscale conversion and denoising are performed on the image features to enhance the image features.

[0077] A2: In the target detection stage, each text pre-selection box in the enhanced image features is obtained through the target detection algorithm model (such as the YOLO4 model), and the position information (ie, coordinate information) of each text pre-selection box is stored.

[0078] A3: According to the position information of each text pre-selection box, adjacent text pre-selection boxes in the text pre-selection boxes are merged to obtain a whole text area.

[0079] In A3, the adjacent text pre-selection boxes in each text pre-selection box are merged into a whole text area according to the position information of each text pre-selection box. For example, for text pre-selection box 3 and text pre-selection box 7, the adjacent text pre-selection boxes of text pre-selection box 3 are text pre-selection box 2 and text pre-selection box 4, and the adjacent text pre-selection box of text pre-selection box 2 is text pre-selection box 1; the adjacent text pre-selection boxes of text pre-selection box 7 are text pre-selection box 6 and text pre-selection box 8, and the adjacent text pre-selection boxes of text pre-selection box 6 are text pre-selection box 5 and text pre-selection box 7. And so on, so that the adjacent text pre-selection boxes in each text pre-selection box can be merged into a whole text area.

[0080] A4: In the text recognition stage, a convolutional neural network model (such as a CRNN model) is used to recognize the text in the text area to obtain the text recognition result.

[0081] S103: Create a first image according to the text recognition result.

[0082] The first image is an image created using the text in the text recognition result.

[0083] S104: performing a histogram comparison on the first image and the second image to obtain a histogram comparison result; wherein the second image is a region image of the entire text region in the form image.

[0084] Image histogram is a statistical property describing each gray level of an image. It is a function of the gray value of the image and counts the number or frequency of occurrence of each gray level in the image. From the image point of view, the gray histogram is a two-dimensional image, the horizontal axis is the gray level of each pixel in the image, and the vertical axis represents the number and frequency of occurrence of pixels with each gray level in the image. Histogram comparison is to calculate the similarity between the histograms of two images and then compare the similarity of the images themselves.

[0085] In the process of histogram comparison, based on the first image established by the text recognition result, the regional image of the entire text area in the form image is extracted as the second image, and the histogram (H1) of the first image and the histogram (H2) of the second image are calculated respectively to perform histogram comparison and obtain the histogram comparison result.

[0086] Specifically, the process of performing histogram comparison on the first image and the second image to obtain the histogram comparison result is shown in B1-B3.

[0087] B1: Calculate a first histogram of the first image, and calculate a second histogram of the second image.

[0088] B2: Normalize the first histogram and the second histogram to the same scale space.

[0089] B3: Calculate the Bhattacharyya distance between the first histogram and the second histogram in the same scale space to complete the process of obtaining the histogram comparison result.

[0090] In B3, H1 of the first image and H2 of the second image are calculated respectively, and H1 and H2 are normalized to the same scale space to calculate the similarity d(H1, H2) between H1 and H2. The similarity d(H1, H2) is used to determine whether the text recognition result is similar to the regional image of the entire text area in the form image. The higher the similarity, the more accurate the text recognition result.

[0091] The method of calculating the histogram similarity in this application includes but is not limited to using the Bhattacharyya distance (d BC ) calculation method. The method for calculating the histogram similarity in this application is preferably the Bhattacharyya distance calculation method.

[0092] This scheme is improved on the basis of Bhattacharyya coefficient BC(p, q) to obtain the Bhattacharyya distance calculation method. Bhattacharyya coefficient BC(p, q) describes the similarity of two discrete or continuous probability distributions. In image processing, it can be used to measure the similarity of two image histograms and determine whether the first image and the second image are the same. The pixel value range in the digital image is [0, 255] and they are all integers, so the discrete distribution probability function is used to calculate BC(p, q). The expression of BC(p, q) is shown in formula (1).

[0093]

[0094] Among them, BC(p, q) is the Bhattacharyya coefficient; p(x) is the color distribution of the template image; q(x) is the color distribution of the detection image; z is the pixel value range, the pixel value range is [0, 255] and all are integers.

[0095] Since BC(p, q) does not satisfy the triangle inequality for measuring distance, the calculation result has a large error. Therefore, this scheme improves formula (1) through visual tracking algorithm (Comaniciu) and obtains the method for accurately calculating similarity in this scheme, namely, the Bhattacharyya distance calculation method. The formula of the Bhattacharyya distance calculation method is shown in formula (2).

[0096]

[0097] Among them, d BC (p,q) is the Bhattacharyya distance.

[0098] Substituting H1 of the first image and H2 of the second image into formula (2), we obtain formula (3).

[0099]

[0100] Where H1(I) is the number of pixels I in H1 of the first image; H2(I) is the number of pixels I in H2 of the second image; I∈[0,255]; d BC (H1, H2) is the Bhattacharyya distance between H1 and H2.

[0101] In order to facilitate data calculation and comparison, the histogram data needs to be normalized, as shown in formula (4).

[0102]

[0103] Among them, d BC (H1, H2) is the histogram comparison result; represents a histogram average pixel of the first image; represents the average pixel of the histogram of the second image; N is the total number of pixels, and d BC [0, 1]. The more similar the two images are, the greater the difference between the histograms of the template image and the detection image, d BC The closer it is to 1. Figure 2 As shown, the Figure 2 Just an example.

[0104] Figure 2 In , the detected image is the first histogram and the second histogram.

[0105] S105: Determine whether the histogram comparison result meets the filter requirements. If the histogram comparison result meets the filter requirements, execute S106. If the histogram comparison result does not meet the filter requirements, execute S107.

[0106] S106: Generate target text data.

[0107] Specifically, if the histogram comparison result meets the filter requirements, the process of generating the target text data is shown in C1-C3.

[0108] C1: Get the similarity of the histogram comparison results.

[0109] C2: If the Bhattacharyya distance in the histogram comparison result is greater than or equal to the preset threshold, it is determined that the histogram comparison result does not meet the filter requirement.

[0110] Among them, the preset threshold is set according to the actual situation and this application does not make any specific limitation.

[0111] C3: If the Bhattacharyya distance in the histogram comparison result is less than a preset threshold, it is determined that the histogram comparison result meets the filter requirement.

[0112] The calculated d BC (H1, H2) is sent to a threshold filter with a preset threshold for accuracy determination. BC When (H1, H2) is less than the preset threshold, the text recognition result of the text area is determined to be accurate and meets the filter requirements. BC When (H1, H2) is greater than or equal to a preset threshold, it is determined that the text recognition result of the text area is inaccurate and does not meet the filter requirements.

[0113] S107: Extracting image areas from all text pre-selected boxes on the form image and performing single word detection to output target text data.

[0114] In S107, the image area extracted from all the text pre-selected boxes on the form image is sent to the Convolutional Neural Networks (CNN) model for single word detection, and the detection result is obtained and output. The CNN model can be used for text recognition, especially in single word detection, with a high detection accuracy. The detection result is specifically shown in formula (5).

[0115]

[0116] Among them, δ is the preset threshold.

[0117] Repeat the above process until all text areas detected in the YOLO4 model have completed text recognition. After all the information of the paper form is extracted, the final target text data is output and the process ends.

[0118] The specific process of extracting all the text pre-selected boxes on the form image into the image area and performing single word detection to output the target text data is shown in D1-D3.

[0119] D1: If the histogram comparison result does not meet the filter requirements, extract the corresponding image area from each text pre-selected box (i.e., the box before merging) in the enhanced image feature;

[0120] D2: For the image area extracted from each text pre-selection box, the image area is input into the CNN model for single word detection and a corresponding serial number is generated; wherein the serial number is determined by the order of the text data in the paper form image.

[0121] The serial number is used to ensure that the content of the target text data output subsequently is not disordered, thereby improving accuracy.

[0122] D3: until the single word detection is completed in the image area of ​​all the text pre-selection boxes, the text data corresponding to all the serial numbers are integrated and the target text data is output.

[0123] In order to facilitate the understanding of the process of form information extraction method, combined with Figure 3 Give an example.

[0124] Figure 3 In the YOLO4 model, the text pre-selection box of each text in the form information image can be obtained through the feature extraction stage and the feature enhancement stage. In this stage, the position information of each text pre-selection box is retained at the same time, and the adjacent pre-selection boxes are merged into a whole text area according to the position information of each text pre-selection box;

[0125] Use the CRNN model to identify the text data in the entire text area and obtain the text recognition result;

[0126] Creating a new image, i.e., a first image, according to the text recognition result;

[0127] Extracting a corresponding original image region from a regional image of the entire text region, i.e., a second image;

[0128] Performing a histogram comparison on the first image and the second image to obtain a histogram comparison result;

[0129] Send the histogram comparison result to the threshold filter to determine whether the histogram comparison result passes the filter requirement;

[0130] If the histogram comparison result meets (passes) the filter requirements, the target text data is generated;

[0131] If the histogram comparison result does not meet (does not pass) the filter requirement, all text pre-selected boxes on the form image are extracted to the image area and sent to the CNN model for single word detection to output the result, i.e., the target text data.

[0132] The text recognition accuracy of this application is greatly improved compared with the traditional method. Since the optimization improvement is greater when the amount of training data added to the traditional method is small, as the amount of data increases, the traditional method also has a high accuracy, and the improvement effect decreases. The schematic diagram of the effect comparison between this solution and the traditional method is shown in the figure. Figure 4 shown.

[0133] like Figure 4 As shown in the figure, the YOLO4 model can obtain the pre-selected boxes of each word in the image through the feature extraction stage and the feature enhancement stage. In this stage, the position information of these pre-selected boxes is retained and the adjacent pre-selected boxes are merged into the same text area. Then the text data in these text areas are recognized by the CRNN model.

[0134] The form information extraction method of the present application is applicable to a form information extraction device, such as Figure 5 As shown, the form information extraction device is composed of an image acquisition module and an image processing module, wherein the image acquisition module includes a bracket, a camera, a light source, etc.; the image processing module includes a computer, an image processing program, etc.

[0135] This application adds a histogram comparison method based on OCR text recognition technology to perform a secondary judgment on the OCR text recognition results, which can reduce the impact of uneven lighting and other problems on the image and improve the accuracy of text recognition. At the same time, a threshold-based filter is added, and the CNN model performs single-word detection on the results that do not meet the filter requirements, optimizes the text recognition results, and improves the accuracy of information extraction.

[0136] The scenarios of this application include but are not limited to scenarios where banks need to input data based on paper forms provided by customers. With the help of OCR text recognition technology based on the YOLO4 model and the CRNN model, form information is automatically extracted to improve the input efficiency and accuracy of staff and reduce customer waiting time.

[0137] Based on the OCR text recognition technology based on the YOLO4 model and the CRNN model, this application adds a histogram comparison method and a threshold-based filter to optimize the text recognition results, re-detects the wrong classification results through a single word detection model, adds a CNN model, and re-recognizes single words to improve the correctness of text recognition, thereby improving the accuracy of information extraction from paper forms.

[0138] The beneficial effects of the embodiments of the present application are as follows: by adding a histogram comparison method on the basis of the preset text recognition technology and performing a secondary judgment on the form image, the influence of problems such as uneven illumination on the image can be reduced, and the accuracy of text recognition can be improved. At the same time, a threshold-based filter is added to perform single-word detection on the results that do not meet the filter requirements, so as to optimize the text recognition results and improve the accuracy of form image information extraction.

[0139] Based on the above embodiments Figure 1 A form information extraction method is disclosed, and the embodiment of the present application also discloses a form information extraction system, such as Figure 6 As shown, the form information extraction system includes:

[0140] An acquisition unit 601 is used to acquire a form image;

[0141] The recognition unit 602 is used to perform text recognition on all text pre-selection boxes on the form image by using a preset text recognition technology to obtain a text recognition result;

[0142] A creation unit 603, configured to create a first image according to the text recognition result;

[0143] A comparison unit 604 is used to perform a histogram comparison on the first image and the second image to obtain a histogram comparison result; wherein the second image is a region image of the entire text region in the form image;

[0144] A generating unit 605, for generating target text data if the histogram comparison result meets the filter requirement;

[0145] The detection unit 606 is used to extract the image area from all the text pre-selected boxes on the form image and perform single word detection to output the target text data if the histogram comparison result does not meet the filter requirement.

[0146] Furthermore, the identification unit 602 includes:

[0147] An enhancement module is used to extract image features from the form image and enhance the image features;

[0148] The first acquisition module is used to acquire each text pre-selection box in the enhanced image features through the target detection algorithm model during the target detection stage, and store the position information of each text pre-selection box;

[0149] A merging module, used for merging adjacent text pre-selection boxes in each text pre-selection box according to the position information of each text pre-selection box to obtain a whole text area;

[0150] The recognition module is used to recognize the text in the entire text area through the convolutional neural network model in the text recognition stage to obtain the text recognition result.

[0151] Furthermore, the comparison unit 604 includes:

[0152] A first calculation module, used for calculating a first histogram of the first image and calculating a second histogram of the second image;

[0153] A normalization module, used to normalize the first histogram and the second histogram to the same scale space;

[0154] The second calculation module is used to calculate the Bhattacharyya distance between the first histogram and the second histogram in the same scale space to complete the process of obtaining the histogram comparison result.

[0155] Furthermore, the generating unit 605 includes:

[0156] A first determination module, configured to determine that the histogram comparison result does not meet the filter requirement if the Bhattacharyya distance in the histogram comparison result is greater than or equal to a preset threshold;

[0157] The second determination module is used to determine whether the histogram comparison result meets the filter requirement if the Bhattacharyya distance in the histogram comparison result is less than the preset threshold.

[0158] Furthermore, the detection unit 606 includes:

[0159] An extraction module, for extracting a corresponding image region from each text pre-selected box in the enhanced image feature if the histogram comparison result does not meet the filter requirements;

[0160] A detection module is used to input the image area extracted from each text pre-selection box into a convolutional neural network for single word detection and generate a corresponding serial number; wherein the serial number is determined by the order of the text data in the form image;

[0161] The integration output module is used to complete the single word detection in the image area of ​​all the text pre-selection boxes, integrate the text data corresponding to all the serial numbers obtained, and output the target text data.

[0162] The beneficial effects of the embodiments of the present application are as follows: by adding a histogram comparison method on the basis of the preset text recognition technology and performing a secondary judgment on the form image, the influence of problems such as uneven illumination on the image can be reduced, and the accuracy of text recognition can be improved. At the same time, a threshold-based filter is added to perform single-word detection on the results that do not meet the filter requirements, so as to optimize the text recognition results and improve the accuracy of form image information extraction.

[0163] For the aforementioned method embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the order of the actions described, because according to the present application, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0164] It should be noted that each embodiment in this specification is described in a progressive manner, and each embodiment focuses on the differences from other embodiments, and the same or similar parts between the embodiments can be referred to each other. For system embodiments, since they are basically similar to method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0165] The steps in the methods of the embodiments of the present application can be adjusted in order, combined, and deleted according to actual needs.

[0166] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are merely used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations.

[0167] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

[0168] The above is only a preferred implementation of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims

1. A form information extraction method, characterized in that: The method comprises: Get the form image; Perform text recognition on all text pre-selected boxes on the form image using a preset text recognition technology to obtain a text recognition result; Creating a first image according to the text recognition result; Performing a histogram comparison on the first image and the second image to obtain a histogram comparison result; wherein the second image is a region image of the entire text region in the form image; If the histogram comparison result meets the filter requirements, generating target text data; If the histogram comparison result does not meet the filter requirement, all the text pre-selected boxes on the form image are extracted into image areas and single word detection is performed to output target text data.

2. The method according to claim 1, characterized in that The text recognition is performed on all the text pre-selected boxes on the form image by using the preset text recognition technology to obtain the text recognition results, including: Extracting image features from the form image and enhancing the image features; In the target detection stage, each text pre-selection box in the enhanced image features is obtained through the target detection algorithm model, and the position information of each text pre-selection box is stored; According to the position information of each text pre-selection box, adjacent text pre-selection boxes in each text pre-selection box are merged to obtain a whole text area; In the text recognition stage, the convolutional neural network model is used to recognize the text in the entire text area to obtain the text recognition result.

3. The method according to claim 1, characterized in that The step of performing a histogram comparison on the first image and the second image to obtain a histogram comparison result includes: calculating a first histogram of the first image, and calculating a second histogram of the second image; Normalizing the first histogram and the second histogram to the same scale space; The Bhattacharyya distance between the first histogram and the second histogram is calculated in the same scale space to complete the process of obtaining the histogram comparison result.

4. The method according to claim 1, characterized in that: If the histogram comparison result meets the filter requirement, generating target text data includes: If the Bhattacharyya distance in the histogram comparison result is greater than or equal to a preset threshold, determining that the histogram comparison result does not meet the filter requirement; If the Bhattacharyya distance in the histogram comparison result is less than the preset threshold, it is determined that the histogram comparison result meets the filter requirement.

5. The method according to claim 4, characterized in that If the histogram comparison result does not meet the filter requirement, extracting all the text pre-selection boxes on the form image into the image area and performing single word detection to output the target text data, including: If the histogram comparison result does not meet the filter requirement, extracting the corresponding image area from each text pre-selected box in the enhanced image feature; For the image area extracted from each text pre-selection box, the image area is input into a convolutional neural network for single word detection and a corresponding serial number is generated; wherein the serial number is determined by the order of the text data in the form image; Until the single word detection is completed in the image area of ​​all the text pre-selection boxes, the text data corresponding to all the serial numbers are integrated and the target text data is output.

6. A form information extraction system, characterized in that: The system comprises: An acquisition unit, used for acquiring a form image; The recognition unit is used to perform text recognition on all text pre-selected boxes on the form image by using a preset text recognition technology to obtain a text recognition result; A creating unit, used for creating a first image according to the text recognition result; A comparison unit, configured to perform a histogram comparison on the first image and the second image to obtain a histogram comparison result; wherein the second image is a region image of the entire text region in the form image; A generating unit, configured to generate target text data if the histogram comparison result meets the filter requirement; The detection unit is used to extract the image area from all the text pre-selection boxes on the form image and perform single word detection to output the target text data if the histogram comparison result does not meet the filter requirement.

7. The system according to claim 6, characterized in that The identification unit comprises: An enhancement module, used for extracting image features from the form image and enhancing the image features; The first acquisition module is used to acquire each text pre-selection box in the enhanced image features through the target detection algorithm model during the target detection stage, and store the position information of each text pre-selection box; A merging module, used for merging adjacent text pre-selection boxes in each text pre-selection box according to the position information of each text pre-selection box to obtain a whole text area; The recognition module is used to recognize the text in the entire text area through the convolutional neural network model in the text recognition stage to obtain the text recognition result.

8. The system according to claim 6, characterized in that The comparison unit comprises: A first calculation module, used for calculating a first histogram of the first image and calculating a second histogram of the second image; A normalization module, used for normalizing the first histogram and the second histogram to the same scale space; The second calculation module is used to calculate the Bhattacharyya distance between the first histogram and the second histogram in the same scale space to complete the process of obtaining the histogram comparison result.

9. The system according to claim 6, characterized in that The generating unit comprises: A first determination module, configured to determine that the histogram comparison result does not meet the filter requirement if the Bhattacharyya distance in the histogram comparison result is greater than or equal to a preset threshold; The second determination module is used to determine whether the histogram comparison result meets the filter requirement if the Bhattacharyya distance in the histogram comparison result is less than the preset threshold.

10. The system according to claim 9, characterized in that The detection unit comprises: An extraction module, for extracting a corresponding image region from each text pre-selected box in the enhanced image feature if the histogram comparison result does not meet the filter requirement; A detection module, for inputting the image area extracted from each text pre-selection box into a convolutional neural network for single-word detection and generating a corresponding serial number; wherein the serial number is determined by the order of the text data in the form image; The integration output module is used to complete the single word detection in the image area of ​​all the text pre-selection boxes, integrate the text data corresponding to all the serial numbers obtained, and output the target text data.

Images