Document key information extraction method
By combining regular extraction and deep learning models, and setting extraction rules and models according to file types, the problem of inefficient processing of file documents in traditional technology is solved, efficient and accurate extraction of key information is achieved, and labor costs are reduced.
Patent Information
- Application Number
- CN202411239505.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-05
- Publication Date
- 2024-12-31
AI Technical Summary
Traditional manual processing of file documents is inefficient and prone to errors. Regular extraction technology is highly cost-effective, poorly robust, and has low accuracy, making it difficult to extract key information from non-standard files efficiently and accurately.
Combining regular extraction and deep learning models, the extraction rules and models are set according to the file type, and the use of deep learning models to extract key information is given priority. When failures, regular extraction is used for secondary extraction, and the accuracy is improved through error correction and updating rules and models.
It greatly saves labor costs for obtaining key information from files, improves adaptability and accuracy, and achieves efficient and accurate extraction of key information.
Smart Images
Figure CN119226337A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of software technology, and in particular to a method for extracting key information from a file. Background Art
[0002] At present, the company handles a large number of dossier documents, which contain a lot of key information, such as project names, party information, etc. This information is crucial for user information backfilling, paperless office and other tasks. However, since this information is scattered in different parts of the material documents and the material document formats are different, the traditional manual processing method is inefficient and prone to errors. The traditional regular extraction method has extremely high maintenance and development costs due to the diversity of materials, poor robustness and low accuracy. Summary of the invention
[0003] In view of the above problems, the present invention provides a method for extracting key information from dossiers, aiming to efficiently and accurately extract useful key information from a large amount of non-standard dossier materials. To achieve the above objectives, the present invention provides the following technical solutions:
[0004] A method for extracting key information from a file comprises the following steps:
[0005] S1: According to the file type from which key information is to be extracted, set the expression form of the key information to be extracted;
[0006] S2: For key information whose length may exceed the length threshold, formulate regular extraction rules; for key information whose length may not exceed the length threshold, train a deep learning model to extract key information; for key information whose length may not exceed the length threshold, if it is expressed uniformly in the dossier or has the characteristics of the preceding and following texts, formulate regular extraction rules;
[0007] S3: In the process of extracting files, the key information for which only regular extraction rules exist is extracted using regular extraction rules;
[0008] For key information that only exists in deep learning models, use deep learning models to extract key information;
[0009] For key information that has both deep learning models and regular extraction rules, deep learning models are used for extraction first. When extraction fails or when users report that key information is extracted incorrectly, regular extraction rules are used for secondary extraction.
[0010] The extracted key information is displayed to the user according to the expression form of the key information.
[0011] Preferably, the method further comprises the steps of:
[0012] S4: Correct and store errors in the key information generated during the file extraction process, and use it to regularly update regular extraction rules and deep learning models.
[0013] Preferably, the key information is expressed in the form of a schema structure;
[0014] Preferably, the deep learning model is BERT.
[0015] Preferably, the length threshold is 400 Chinese characters.
[0016] Beneficial effect: The present invention combines regular extraction with deep learning model extraction, has stronger adaptability, and greatly saves the manpower cost of obtaining key information from files. BRIEF DESCRIPTION OF THE DRAWINGS
[0017] Figure 1 This is the information extraction method in the embodiment. DETAILED DESCRIPTION
[0018] In order to make the purpose, features and advantages of the present invention more obvious and easy to understand, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described below are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0019] A method for extracting key information from a file comprises the following steps:
[0020] S1: according to the file type from which key information is to be extracted, the expression form of the key information to be extracted is set; in this embodiment, the expression form of the key information is expressed in the form of a schema structure. For example, if the key information to be extracted from the file is Party A and Party B and the corresponding addresses, the key information is expressed in the form of a schema structure as follows: [{'Party A':'Address'},{'Party B':'Address'}];
[0021] S2: For key information whose length may exceed the length threshold, formulate regular extraction rules; for key information whose length may not exceed the length threshold, train a deep learning model to extract key information; for key information whose length may not exceed the length threshold, if it is expressed uniformly in the dossier or has the characteristics of the preceding and following texts, formulate regular extraction rules;
[0022] In this embodiment, the deep learning model is BERT, and the length threshold is the length of 400 Chinese characters; the training data is obtained by manual annotation, and the training process belongs to the prior art and will not be repeated here;
[0023] S3: Figure 1 As shown, in the process of extracting files, the key information for which only regular extraction rules exist is extracted using regular extraction rules;
[0024] For key information that only exists in deep learning models, use deep learning models to extract key information;
[0025] For key information that has both deep learning models and regular extraction rules, deep learning models are used for extraction first. When extraction fails or when users report that key information is extracted incorrectly, regular extraction rules are used for secondary extraction.
[0026] Display the extracted key information to the user according to the expression form of the key information;
[0027] S4: Correct and store errors in the key information generated during the file extraction process, and use it to regularly update regular extraction rules and deep learning models.
[0028] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.
Claims
1. A method for extracting key information from a file, characterized in that: The following steps are involved: S1: According to the file type from which key information is to be extracted, set the expression form of the key information to be extracted; S2: For key information whose length may exceed the length threshold, formulate regular extraction rules; for key information whose length may not exceed the length threshold, train a deep learning model to extract key information; for key information whose length may not exceed the length threshold, if it is expressed uniformly in the dossier or has the characteristics of the preceding and following texts, formulate regular extraction rules; S3: In the process of extracting files, the key information for which only regular extraction rules exist is extracted using regular extraction rules; For key information that only exists in deep learning models, use deep learning models to extract key information; For key information that has both deep learning models and regular extraction rules, deep learning models are used for extraction first. When extraction fails or when users report that key information is extracted incorrectly, regular extraction rules are used for secondary extraction. The extracted key information is displayed to the user according to the expression form of the key information.
2. A method for extracting key information from a file according to claim 1, characterized in that: Also includes the steps: S4: Correct and store errors in the key information generated during the file extraction process, and use it to regularly update regular extraction rules and deep learning models.
3. A method for extracting key information from a file according to claim 1, characterized in that: The key information is expressed in the form of a schema structure.
4. A method for extracting key information from a file according to claim 1, characterized in that: The deep learning model is BERT.
5. A method for extracting key information from a file according to claim 1, characterized in that: The length threshold is 400 Chinese characters.