A form recognition system based on deep learning

Abstract

This invention discloses a form recognition system based on deep learning, specifically relating to the field of intelligent form recognition. It includes a document format conversion module, a table scanning and confirmation / splitting module, a cell content extraction module, a form basic information collection module, a table information extraction and verification module, an information content classification module, an information content matching module, a recognition capability index calculation module, a recognition result judgment module, and a judgment result feedback module. This invention scans paper documents, converts them into image files, recognizes the image files, locates tables within the image files, and extracts, classifies, and matches the table content. By calculating the completeness and accuracy of table recognition, as well as the data classification and matching capabilities, a recognition capability index is obtained for the form recognition system. The recognition capability index is then judged and the data is fed back to the system maintenance terminal.

Description

Technical Field

[0001] This invention relates to the field of intelligent form recognition technology, and more specifically, to a form recognition system based on deep learning. Background Technology

[0002] Existing form recognition technologies work by identifying application forms, extracting data from the forms, training a recognition model to determine various types of loss values, and adjusting the model's parameters based on these loss values. These technologies aim to improve the accuracy of form location and reduce interference from blank pages by removing them, thereby increasing recognition efficiency. Furthermore, they utilize matching algorithms to calculate the cosine similarity between attributes. The current focus of form recognition technology is on improving accuracy and efficiency, leading to more precise and efficient form recognition results.

[0003] Existing form recognition technologies primarily focus on improving accuracy and efficiency through various algorithms. However, they lack calculations of the system's form recognition capability index and fail to assess whether the system's capabilities are up to standard. For form recognition technologies with adequate capabilities, further improvements in efficiency and accuracy are unnecessary. The absence of a calculation process for the system's recognition capability index hinders accurate quantitative evaluation, causing significant inconvenience for system operation and maintenance. Furthermore, the lack of data classification and matching processes after form recognition, and the absence of calculations for these capabilities, make it impossible to objectively evaluate existing form recognition technologies. Accuracy, completeness, and coverage are crucial factors in evaluating form recognition technology, yet current technologies lack detailed explanations and experimental calculations, making objective evaluation impossible. Summary of the Invention

[0004] To overcome the aforementioned deficiencies of the prior art, the present invention provides a form recognition system based on deep learning, which solves the problems mentioned in the background art by calculating the form recognition capability index of the system.

[0005] To achieve the above objectives, the present invention provides the following technical solution: a form recognition system based on deep learning includes a document format conversion module, a table scanning confirmation and splitting module, a cell content extraction module, a form basic information collection module, a table information extraction and verification module, an information content classification module, an information content matching module, a recognition ability index calculation module, a recognition result judgment module, and a judgment result feedback module.

[0006] Preferably, the paper document is converted into an image file using a scanner, and the text in the image is converted into text format using recognition software. The specific steps for document format conversion are as follows:

[0007] A1. The document format conversion module connects to the scanner, which scans paper documents and converts them into image files for display.

[0008] A2. Use image recognition software to convert the text in the image into text format.

[0009] Preferably, the specific steps for table processing are as follows: First, identify the target table, then detect and determine the table range within the text, and finally split each cell in the identified target table.

[0010] B1. Use deep learning-based object detection methods to detect objects in documents, capture table content, and determine the range of tables.

[0011] B2. Use deep learning technology to detect and split the cells in the table, and distinguish the range of each cell.

[0012] Preferably, the content of the target table is extracted using OCR (Optical Character Recognition) technology.

[0013] Preferably, the basic information of the form contained in the document to be identified is collected. The specific steps for information collection are as follows:

[0014] C1. Collect the original document and the generated data after recognition separately;

[0015] C2. The information collected specifically includes: the number of existing tables (n), and the number of data items contained in each existing table (C). n The number of tables extracted, D n The number of data points in each extracted table is g; the number of correct data points classified as correct data points is a; the number of incorrect data points classified as incorrect data points is b; the number of correct data points classified as incorrect data points is d; and the number of incorrect data points classified as correct data points is e.

[0016] Preferably, the specific steps for detecting data content, calculating the completeness of extracted content, and performing data calculations are as follows:

[0017] D1. Compare the table data contained in the original document with the table data contained in the identified document;

[0018] D2. Calculate the completeness of the extracted table data:

[0019] D3. Calculate the accuracy of the extracted data:

[0020] Preferably, the classification ability index of the system is calculated based on the comparison of the classified data. The specific steps for calculating the classification ability index are as follows:

[0021] E1. Use deep learning-based multimodal text classification technology to classify the content inside the cell;

[0022] E2. Calculate the classification ability index of the system: L = K r *R+K p *P, where P g For the accuracy of classification, R g For the coverage of the category,

[0023]

[0024] Preferably, the collected information is matched with business-related content, and the system's data matching capability index is calculated. The specific steps for calculating the data matching capability index are as follows:

[0025] F1. Match the collected information with business-related content;

[0026] F2. Using the TF-IDF algorithm and data cosine similarity, the data matching ability index X of the system is calculated.

[0027] Preferably, the completeness and accuracy of the extracted content, the system's classification ability index, and the system's matching ability index are obtained, and the data are integrated to calculate the system's recognition ability index; the specific steps for calculating the recognition ability index are as follows:

[0028] G1, the completeness and accuracy of receiving and integrating extracted content, the system's classification ability index, and the system's matching ability index;

[0029] G2, Recognition capability index of the computing system: S n =δ(L+X)*e (W+Z) , where δ is the influence factor.

[0030] Preferably, the specific steps for determining whether the recognition ability index is normal and for data judgment are as follows:

[0031] H1. Receive the system's recognition capability index calculation results and judge the results according to the judgment criteria;

[0032] H2. Determine whether the calculated system recognition capability index is greater than T%. If the system recognition capability index is greater than 80%, the result is normal; if the system recognition capability index is less than T%, the result is abnormal.

[0033] Preferably, the judgment result transmitted by the recognition result judgment module is fed back and corresponding reminders are given.

[0034] J1. The judgment result feedback module is connected to the system database and the system maintenance terminal;

[0035] J2. If the judgment result is normal, the data will be directly fed back to the system maintenance terminal, and a message will be displayed indicating that the system recognition capability index is normal.

[0036] J3. The judgment result feedback module will send the result of the abnormal recognition capability index judgment to the system maintenance terminal and prompt that the system recognition capability index is abnormal, and that the system recognition performance needs to be retested or the system maintenance needs to be performed directly.

[0037] The technical effects and advantages of this invention are as follows:

[0038] This invention employs a table information extraction and verification module to verify the table information extracted by the system, calculating the completeness and accuracy of the extracted information. An information content classification module categorizes the extracted table information and calculates the classification capability index, resulting in the system's table recognition data classification capability index. An information content matching module performs data matching on the form recognition data and calculates the data matching capability index, yielding the form recognition system's data matching capability index. Through these modules, the recognition capability index of the form recognition system is analyzed from multiple perspectives, helping to determine whether the system's recognition capability meets standards and providing an accurate quantitative assessment of the system's recognition ability. This facilitates targeted system maintenance and training to improve the recognition capability of the form recognition system. Attached Figure Description

[0039] Figure 1 This is a system structure block diagram of the present invention.

[0040] Figure 2 This is a system flowchart of the present invention. Detailed Implementation

[0041] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0042] This embodiment provides, for example Figure 1 The system described is a form recognition system based on deep learning, which includes a document format conversion module, a table scanning and confirmation splitting module, a cell content extraction module, a form basic information collection module, a table information extraction and verification module, an information content classification module, an information content matching module, a recognition ability index calculation module, a recognition result judgment module, and a judgment result feedback module.

[0043] The document format conversion module converts paper documents into image files after scanning. The table scanning and splitting module detects tables within the text, determines the table range, and splits each cell in the target table. The cell content extraction module extracts the content from the target table. The form basic information collection module collects basic data and transmits it to the system database. The table information extraction verification module verifies the completeness and accuracy of the extracted information after table recognition and transmits the calculation results to the recognition capability index calculation module. The information content classification module classifies the extracted table content, calculates the system classification capability index, and transmits the data results to the recognition capability index calculation module. The information content matching module matches the form recognition content, calculates the system's data matching degree, and transmits the calculated data matching degree to the recognition capability index calculation module. The recognition capability index calculation module calculates the system's form recognition capability index and connects to the recognition result judgment module. The recognition result judgment module judges whether the system's form recognition capability is qualified and connects to the judgment result feedback module, transmitting the judgment result to the judgment result feedback module. The judgment result feedback module connects to the system maintenance terminal, provides feedback on the judgment result, and gives corresponding prompts.

[0044] The difference between this implementation and existing technologies lies in the table information extraction and verification module, information content classification module, information content matching module, and recognition result judgment module. The table information extraction and verification module adds the function of calculating the completeness and accuracy of extracted information; the information content classification module adds the function of calculating the recognition and extraction content classification ability index; the information content matching module adds the function of calculating the data matching ability after recognition; and the recognition result judgment module adds the function of result judgment. It can calculate the recognition ability of the form recognition system and obtain the recognition ability index of the form recognition system. The whole process is not available in existing technologies.

[0045] like Figure 2 This embodiment provides a system flowchart for a form recognition system based on deep learning, which specifically includes the following steps:

[0046] 101. The specific steps for converting paper documents into image files using a scanner via a document conversion module, and then converting the text in the image into text format using recognition software, are as follows:

[0047] A1. The document format conversion module connects to the scanner, which scans paper documents and converts them into image files for display.

[0048] A2. Use image recognition software to convert the text in the image into text format;

[0049] It should be noted here that the document format conversion module in this embodiment can convert paper documents into image files. This step is beneficial for subsequent secondary processing of the image files, making it easier to test the actual recognition capability of the form recognition system.

[0050] 102. The table scanning and splitting module confirms the target table, detects tables within the text, determines the table range, and splits each cell in the determined target table. The specific steps for table processing are as follows:

[0051] B1. Use deep learning-based object detection methods to detect objects in documents, capture table content, and determine the range of tables.

[0052] B2. Use deep learning technology to detect and split the cells in the table, and distinguish the range of each cell.

[0053] It's worth noting here that defining the range and splitting cells in a table helps to extract content from the table in parts, preventing complex content from causing chaotic extraction results.

[0054] 103. The cell content extraction module uses OCR optical character recognition technology to extract the content from the target table.

[0055] It should be noted here that when using OCR (Optical Character Recognition) technology to extract table content, it is necessary to ensure that the image being recognized is complete and flat, and that the font formats on the recognized images are roughly similar. OCR technology can extract text and characters from tables, and can effectively realize the transformation from paper to image and then to electronic version.

[0056] 104. The specific steps for collecting basic form information from the document to be identified using the form basic information collection module are as follows:

[0057] C1. Collect the original document and the generated data after recognition separately;

[0058] C2. The information collected specifically includes: the number of existing tables (n), and the number of data items contained in each existing table (C). n The number of tables extracted, D n The number of data points in each extracted table is g; the number of correct data points classified as correct data points is a; the number of incorrect data points classified as incorrect data points is b; the number of correct data points classified as incorrect data points is d; and the number of incorrect data points classified as correct data points is e.

[0059] 105. The specific steps for performing data calculations by using the table information extraction and verification module to check the data content and calculate the completeness of the extracted content are as follows:

[0060] D1. Compare the table data contained in the original document with the table data contained in the identified document;

[0061] D2. Calculate the completeness of the extracted table data:

[0062] D3. Calculate the accuracy of the extracted data:

[0063] It should be noted here that the calculation of the completeness of table data extraction is based on two aspects: the completeness of the number of tables extracted and the completeness of the extracted table data. If the number of tables is insufficient, the data will inevitably be incomplete, so the overlapping calculation content of the two parts needs to be subtracted.

[0064] 106. The classification ability index of the system is calculated by comparing the classified data through the information content classification module. The specific steps for calculating the classification ability index are as follows:

[0065] E1. Use deep learning-based multimodal text classification technology to classify the content inside the cell;

[0066] E2. Calculate the classification ability index of the system: L = K r *R+K p *P, where P g For the accuracy of classification, R g For the coverage of the category,

[0067]

[0068] It should be noted here that: P g ,R g These are used to represent the accuracy and coverage after classification, but they are only applicable to the binary classification method. In the multi-class classification method mentioned above, the average of the results calculated by the binary classification method is used. Classification ability is an influencing factor of the system's recognition ability. Therefore, calculating the system's data classification ability helps to calculate the system's recognition ability index.

[0069] 107. The information content matching module matches the collected information with business-related content and calculates the system's data matching capability index. The specific steps for calculating the data matching capability index are as follows:

[0070] F1. Match the collected information with business-related content;

[0071] F2. Calculate the data matching ability index X of the system by combining the TF-IDF algorithm and data cosine similarity.

[0072] It's important to note here that the TF-IDF algorithm is a common weighting technique used in information retrieval and data mining. TF stands for Term Frequency, and IDF stands for Inverse Term Frequency Index. TF-IDF is simply TF*IDF. |D| represents the total number of files in the corpus, and |{j:ti∈dj}| represents the number of files containing the word ti. This calculation process is not an innovation of this invention and will not be described in detail here.

[0073] 108. Obtain the completeness and accuracy of the extracted content, the system's classification ability index, and the system's matching ability index through the recognition ability index calculation module, and integrate the data to calculate the system's recognition ability index; the specific steps for calculating the recognition ability index are as follows:

[0074] G1, the completeness and accuracy of receiving and integrating extracted content, the system's classification ability index, and the system's matching ability index;

[0075] G2, Recognition capability index of the computing system: S n =δ(L+X)*e (W+Z) , where δ is the impact factor;

[0076] It should be noted here that the system's recognition capability index is used to measure the system's form recognition capability. The higher the recognition capability index, the better the system's recognition effect and the higher the accuracy and completeness of the recognized data. The system's form recognition capability is affected by many factors. The calculation of the recognition capability index quantifies the system's form recognition capability, which can provide a more intuitive, objective, and realistic evaluation of the form recognition system.

[0077] 109. The specific steps for judging whether the recognition capability index is normal through the recognition result judgment module are as follows:

[0078] H1. Receive the system's recognition capability index calculation results and judge the results according to the judgment criteria;

[0079] H2. Determine whether the calculated system recognition capability index is greater than T%. If the system recognition capability index is greater than 80%, the result is normal. If the system recognition capability index is less than T%, the result is abnormal.

[0080] It should be noted that the reference value T used for comparison with the system recognition capability index is not a fixed value. The value of T changes according to the different requirements for the system's recognition capability. The higher the value of T, the higher the requirements for the system's form recognition capability and the more accurate the form recognition results are required.

[0081] 110. The judgment result feedback module provides feedback on the judgment result transmitted by the recognition result judgment module and makes corresponding reminders.

[0082] J1. The judgment result feedback module is connected to the system database and the system maintenance terminal;

[0083] J2. If the judgment result is normal, the data will be directly fed back to the system maintenance terminal, and a message will be displayed indicating that the system recognition capability index is normal.

[0084] J3. The judgment result feedback module will feed back the result of the abnormal recognition capability index to the system maintenance terminal and prompt that the system recognition capability index is abnormal, and the system recognition performance needs to be retested or the system maintenance needs to be performed directly.

[0085] It should be noted here that the function of the judgment result feedback module is to transmit the calculated system form recognition capability index to the system maintenance terminal and provide corresponding prompts. Providing prompts after transmitting the data helps in the maintenance of the form recognition system. The judgment results and prompts help to understand whether the form recognition system meets the standards and whether system maintenance is required, thus saving maintenance costs and time.

[0086] In conclusion, the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A form recognition system based on deep learning, characterized in that: include: Document format conversion module: Used to convert paper documents into image files using a scanner, and to convert the text in the images into text format using recognition software; Table scanning and splitting module: Used to confirm the target table. It uses a deep learning-based object detection method to detect tables in the text, determine the table range, and split each cell in the determined target table. Cell content extraction module: This module receives the range information of each cell in the target table and uses OCR technology to extract the content from the target table. The form basic information collection module is used to collect basic information about the forms contained in the document to be identified. Specifically, it includes the number of original tables, the number of data items contained in each original table, the number of extracted tables, the number of data items contained in each extracted table, the number of correct data items classified as correct data, the number of incorrect data items classified as incorrect data, the number of correct data items classified as incorrect data, and the number of incorrect data items classified as correct data. Table Information Extraction and Verification Module: This module receives data from the target table transmitted by the cell content extraction module, verifies the data content, and calculates the completeness and accuracy of the extracted content. Information content classification module: It is used to receive the table content transmitted by the cell content extraction module, classify the collected information, and calculate the system's classification ability index based on the comparison of the classified data. Information content matching module: It is used to receive the table content transmitted by the cell content extraction module, match the collected information with business-related content, and calculate the system's matching ability index by using the TF-IDF algorithm combined with data cosine similarity. Recognition Ability Index Calculation Module: This module receives the completeness and accuracy of the extracted content, the system's classification ability index, and the system's matching ability index from the table information extraction and verification module, the information content classification module, and the information content matching module, and integrates the data to calculate the system's recognition ability index. Recognition Result Judgment Module: This module receives the system recognition capability index transmitted by the recognition capability index calculation module and determines whether the recognition capability index is greater than T%. If the recognition capability index is less than T%, the system recognition is deemed abnormal. Judgment Result Feedback Module: This module receives the judgment result from the recognition result judgment module and transmits it to the system maintenance center. If the system recognition capability index is greater than T%, it provides a normal performance prompt. If the system recognition capability index is less than T%, it provides a system performance abnormal prompt. Here, T is a reference value used for comparison with the system recognition capability index.

2. The form recognition system based on deep learning according to claim 1, characterized in that: The document format conversion module is used to convert paper documents into image files using a scanner, and to convert the text in the image into text format using recognition software. The specific steps for document format conversion are as follows: A1. The document format conversion module connects to the scanner, which scans paper documents and converts them into image files for display. A2. Use image recognition software to convert the text in the image into text format.

3. The form recognition system based on deep learning according to claim 1, characterized in that: The table scanning and splitting module is used to confirm the target table, detect tables within the text, determine the table range, and split each cell in the determined target table. The specific steps for table processing are as follows: B1. Use deep learning-based object detection methods to detect objects in documents, capture table content, and determine the range of tables. B2. Use deep learning technology to detect and split the cells in the table, and distinguish the range of each cell.

4. The form recognition system based on deep learning according to claim 1, characterized in that: The cell content extraction module uses OCR optical character recognition technology to extract the content from the target table.

5. A form recognition system based on deep learning according to claim 4, characterized in that: The form basic information collection module is used to collect basic information about forms contained in the document to be identified. The specific steps for information collection are as follows: C1. Collect the original document and the generated data after recognition separately; C2. The information collected specifically includes: the number of existing tables (n), and the number of data items contained in each existing table (C). n The number of tables extracted, D n The number of data points in each extracted table is g; the number of correct data points classified as correct data points is a; the number of incorrect data points classified as incorrect data points is b; the number of correct data points classified as incorrect data points is d; and the number of incorrect data points classified as correct data points is e.

6. The form recognition system based on deep learning according to claim 5, characterized in that: The table information extraction and verification module is used to detect the data content and calculate the completeness of the extracted content. The specific steps for data calculation are as follows: D1. Compare the table data contained in the original document with the table data contained in the identified document; D2. Calculate the completeness of the extracted table data: D3. Calculate the accuracy of the extracted data:

7. A form recognition system based on deep learning according to claim 6, characterized in that: The information content classification module is used to calculate the system's classification ability index based on the comparison of the classified data. The specific steps for calculating the classification ability index are as follows: E1. Use deep learning-based multimodal text classification technology to classify the content inside the cell; E2. Calculate the classification ability index of the system: L = K r *R+K p *P, where P g For the accuracy of classification, R g For the coverage of the category, Where P and R represent the average of the accuracy and coverage after classification, respectively.

8. A form recognition system based on deep learning according to claim 7, characterized in that: The information content matching module is used to match the collected information with business-related content and calculate the system's data matching capability index. The specific steps for calculating the data matching capability index are as follows: F1. Match the collected information with business-related content; F2. Using the TF-IDF algorithm and data cosine similarity, the data matching ability index X of the system is calculated.

9. A form recognition system based on deep learning according to claim 8, characterized in that: The recognition capability index calculation module is used to obtain the completeness and accuracy of the extracted content, the system's classification capability index, and the system's matching capability index, and integrates the data to calculate the system's recognition capability index; the specific steps for calculating the recognition capability index are as follows: G1, the completeness and accuracy of receiving and integrating extracted content, the system's classification ability index, and the system's matching ability index; G2, Recognition capability index of the computing system: S n =δ(L+X)*e (W+Z) , where δ is the influence factor.

10. A form recognition system based on deep learning according to claim 1, characterized in that: The recognition result judgment module is used to determine whether the recognition capability index is normal. The specific steps for data judgment are as follows: H1. Receive the system's recognition capability index calculation results and judge the results according to the judgment criteria; H2. Determine whether the calculated system recognition capability index is greater than T%. If the system recognition capability index is greater than 80%, the result is normal. If the system recognition capability index is less than T%, the result is abnormal. T is a reference value used for comparison with the system recognition capability index.

Images