An OCR recognition result classification method based on a knowledge graph

Abstract

The application discloses an OCR recognition result classification method based on a knowledge graph, which comprises the following steps: constructing an ontology of an OCR recognition result knowledge graph; constructing a text classification model and a named entity extraction model to form a classifier; and constructing a knowledge graph based on the OCR recognition result and the classifier constructed in step S2. According to the text classification information in a specific field, the ontology is constructed, the text classifier is constructed based on the ontology, the category and key information of the OCR software recognition result are extracted by using the classifier, and the text knowledge graph is constructed, so that the purposes of automatic multi-level classification of the OCR recognition result and key information extraction are achieved. The application can realize automatic multi-level classification of the OCR recognition result and key information extraction, solve the problems that the similar technologies ignore the hierarchical relationship between categories, the classification is not high in detail, and only the category information of the text is extracted and stored, other information in the text is ignored, and the text retrieval is inconvenient.

Description

Technical Field

[0001] This invention belongs to the field of image OCR (Optical Character Recognition) technology, specifically relating to a knowledge graph-based OCR recognition result classification method. Background Technology

[0002] OCR (Optical Character Recognition) technology refers to the technology of using electronic devices (such as scanners or digital cameras) to convert paper documents into black-and-white dot matrix image files, determining the shape of characters in the image by detecting dark and light patterns, and then translating the shapes into computer text using character recognition methods for further editing and processing by word processing software. OCR software refers to software that uses OCR technology to digitize paper documents and is widely used in various fields of production and daily life.

[0003] Organizations and institutions that need to use OCR technology often have a large volume and variety of text. To facilitate the use of digitized documents, users need to classify and archive the results of OCR software recognition. Currently, in practical applications of OCR technology, users mostly use manual classification methods to categorize OCR recognition results. While manual classification is technically simple, it consumes a significant amount of manpower and time. To address this, researchers have proposed a deep learning-based text classification method. This method uses a large training text set to train a text classification model, allowing the model to automatically classify text instead of manually. How to achieve automatic classification that more accurately reflects real-world text classification has been a research hotspot in the field of OCR technology, and many related research results have already been achieved.

[0004] The published patent, "A Method for Classifying and Extracting Fields of Bills Based on Deep Learning and OCR" (application publication number CN107633239A), uses a deep learning model to classify the outline of the official seal of a bill and thus obtain the type of bill.

[0005] The published patent, "Classification Method of Digital Electronic Records of Motor Vehicles Based on OCR and Text Mining" (application publication number CN110674332 A), adopts the method of establishing an electronic record header database and determining the category of electronic records by comparing the data in the record header database with the records to be classified.

[0006] The published patent "A Multi-level Text Classification Method and System" (application publication number: CN109902178A) calculates the probability of each layer of text by inputting the text to be classified into multiple trained text classification models, selecting the nth layer text whose probability is greater than a set threshold, and normalizing the probability corresponding to the nth layer text to obtain the text classification result.

[0007] Existing OCR recognition result classification methods mainly fall into two categories: the first is non-hierarchical text classification, and the second involves constructing complex multi-level classification models using sophisticated algorithms and methods. Both approaches achieve automatic classification of OCR recognition results to some extent, but both have shortcomings: the first ignores the hierarchical relationships between categories, resulting in a low level of classification refinement; the second only extracts and stores the category information of the text, ignoring other information within the text, making text retrieval difficult. Summary of the Invention

[0008] To address the problems of existing technologies, this invention provides a knowledge graph-based method for classifying OCR recognition results. This invention constructs an ontology based on text classification information within a specific domain, builds a text classifier based on the ontology, extracts the categories and key information from the OCR software's recognition results using the classifier, and constructs a text knowledge graph using this information, thereby achieving automatic multi-level classification of OCR recognition results and extraction of key information.

[0009] A knowledge graph-based OCR recognition result classification method includes the following steps:

[0010] S1. Construct an ontology of the OCR recognition result knowledge graph;

[0011] S2. Construct a text classification model and a named entity extraction model to form a classifier, including:

[0012] S21. Construct training and testing text sets for a text classification model by manually labeling the category of each text; construct training and testing text sets for a named entity extraction model by manually labeling all named entities in the text;

[0013] S22. Extract text features from the training and testing text sets of the text classification model to train the text classification model; train the named entity extraction model.

[0014] S23. Combine the text classification model and the named entity extraction model to form a classifier;

[0015] S3. Construct a knowledge graph based on the OCR recognition results and the classifier built in step S2.

[0016] Preferably, in step S1, the ontology of the OCR recognition result knowledge graph is a formal description of the types, attributes, and relationships between types and attributes of the OCR recognition results.

[0017] Preferably, step S1 includes the following steps:

[0018] S11. Determine the target text domain for OCR recognition, and collect classification information of the text within the domain, including the names of each category and the attribute names of each type of text;

[0019] S12. Define the concepts in the knowledge graph ontology, define the hierarchy between concepts, and define the attributes of concepts;

[0020] S13. Use knowledge graph ontology modeling tools to encode and model the knowledge graph ontology.

[0021] Preferably, in step S21, the name of each text category manually annotated can be found in the ontology of the knowledge graph, and the name of each named entity manually annotated can be found in the ontology of the knowledge graph.

[0022] Preferably, in step S22, text feature extraction refers to the process of converting text features into vectors using text feature extraction algorithms, including TF-IDF and Word2Vec, after performing word segmentation, stop word removal, and part-of-speech tagging on the training and test text sets of this classification model.

[0023] Preferably, in step S22, the text classification model refers to a mathematical model that can extract text features from the OCR recognition results and classify the OCR recognition results based on the features.

[0024] Preferably, in step S22, the named entity extraction model refers to a mathematical model that can identify entities with specific meanings in the OCR recognition results, including names of people, places, organizations, time, date, currency, and percentage.

[0025] Preferably, in step S22, training the text classification model refers to the process of training machine learning models, including FastText, TextCNN, and TextRNN, using the results of text feature extraction.

[0026] Preferably, the training of the named entity extraction model in step S22 refers to the process of training machine learning models, including HMM, CRF, and LSTM+CRF models, using the training text set and test text set of the named entity extraction model constructed in step 21.

[0027] Preferably, step S3 includes the following steps:

[0028] S31. Use OCR software to recognize the text image to be classified and convert the image into computer-editable text;

[0029] S32. Input the converted computer-editable text into the classifier constructed in step S2 to obtain the category information and other key information of the text to be classified;

[0030] S33. Store the category, attributes, original image information, and computer text information of the text to be classified into a database.

[0031] Compared with the prior art, the present invention has the following beneficial effects:

[0032] 1. Construct an ontology based on text classification information within a specific domain, build a text classifier based on the ontology, use the classifier to extract the categories and key information from the OCR software recognition results, and use this information to construct a text knowledge graph, thereby achieving the goal of automatic multi-level classification and key information extraction of OCR recognition results.

[0033] 2. By clearly defining the domain of the target text for OCR recognition, collecting classification information of text within the domain, and then defining the concepts in the ontology, defining the hierarchy between concepts, and defining the attributes of concepts, the effect of hierarchical OCR recognition result categorization is achieved.

[0034] 3. By constructing a mathematical model that can identify entities with specific meanings in OCR recognition results, including names of people, places, organizations, time, date, currency, and percentage, key information extraction from OCR recognition results is further realized, solving the problem that existing technologies only focus on text category information while ignoring other information in the text.

[0035] 4. By constructing a mathematical model that can extract text features from OCR recognition results and classify the OCR recognition results based on the features, automatic classification of OCR recognition results is achieved.

[0036] 5. By manually labeling the category of each text and all named entities in the text, the application scope of this invention is wider than that of the prior art.

[0037] 6. By extracting the category information and other key information of the text to be classified, and storing the category, attributes, original image information, and computer text information converted from the image information of the text to be classified into the database, the knowledge graph of OCR recognition results is constructed, thereby making the retrieval of OCR recognition results more convenient. Attached Figure Description

[0038] Figure 1 This is a flowchart illustrating the overall process of the method of the present invention.

[0039] Figure 2 To construct a flowchart of ontology operations;

[0040] Figure 3 A flowchart for constructing classifier operations;

[0041] Figure 4 A flowchart for the operations of building a knowledge graph;

[0042] Figure 5 This is a schematic diagram of the main body structure in the method of the present invention;

[0043] Figure 6 An image of a leave request form. Detailed Implementation

[0044] The present invention will now be described in detail with reference to the accompanying drawings and specific embodiments.

[0045] like Figure 1 As shown, the present invention provides a knowledge graph-based OCR recognition result classification method, which includes the following steps:

[0046] S1. Construct an ontology of the OCR recognition result knowledge graph, such as Figure 2 As shown, the specific steps include:

[0047] S11. Define the domain of the target text and collect classification information of the text within the domain, including the names of each category and the attribute names of the text.

[0048] S12. Define the concepts in the ontology, define the hierarchy between concepts, and define the attributes of the concepts. A schematic diagram of the ontology structure after the attributes and concepts are defined is shown below. Figure 5 As shown. Figure 5 In the diagram, A represents the domain to which the text belongs, B1, B2, B3, B4, and B5 are the first-level text classification nodes within domain A, C1, C2, C3, and C4 are subclasses of category B3, and D1, D2, D3, and D4 are attributes of subclass C2.

[0049] S13. Model the ontology using the ontology modeling tool Protege, encode the ontology using the ontology description language OWL, and perform logical checks on the OWL encoding results using the embedded inference engine of the open-source semantic web application framework Jena, including hierarchical reasoning and missing class completion. Ensure that the hierarchical relationships between concepts in the ontology are correct and that the relationship chains between concepts are complete.

[0050] S2. Based on the ontology constructed in step S1, build a text classification model and a named entity extraction model to form a classifier, such as... Figure 3 As shown, the specific steps include:

[0051] S21. By manually labeling the category of each text, construct the training text set and test text set of the text classification model. Manually label the category of each text and save the text itself and its category in a uniform format. The name of the labeled category comes from the concept in the ontology constructed in step S1 above. The text feature extraction step is completed by tools later.

[0052] For every text category name, there is a corresponding category in the knowledge graph ontology; for every manually annotated named entity name, there is a corresponding attribute in the knowledge graph ontology.

[0053] For example, the annotation process is illustrated as follows: {B2 text} indicates that the category of the text "text" is B2; the training and test text sets for the named entity extraction model are constructed, all named entities in the text are manually annotated, and the text itself and its named entity annotations are saved in a unified format. Another example of the annotation process is: {word1 word2word3}, {D1 D2 D3} indicates that word1 is attribute D1, word2 is attribute D2, and word3 is attribute D3.

[0054] S22. Extract text features from the training and test text sets of the text classification model to train the model. Text feature extraction refers to the process of converting text features into vectors using text feature extraction algorithms, including TF-IDF and Word2Vec, after segmenting, removing stop words, and tagging parts of speech on the training and test text sets. The text classification model is a mathematical model capable of extracting text features from OCR recognition results and classifying them based on these features. Input the labeled training and test text sets into the CNN_LSTM model in the Kashgari tool. This model also performs the text feature extraction step, resulting in the text classification model after training. Train the named entity extraction model. The named entity extraction model is a mathematical model capable of recognizing entities with specific meanings in OCR recognition results, including names of people, places, organizations, time, date, currency, and percentages. Training the named entity extraction model refers to the process of training machine learning models, including HMM, CRF, and LSTM+CRF models, using the training and test text sets of the named entity extraction model constructed in step 21. Multiple Bi-LSTM models are built for different categories, and the labeled training and test text sets are input into the corresponding models, resulting in a set of named entity extraction models after training.

[0055] S23. Combine the text classification model and the named entity extraction model to form a classifier.

[0056] S3. Construct a knowledge graph based on the OCR recognition results and the classifier built in step S2, such as... Figure 4 As shown, the specific steps include:

[0057] S31. Use OCR software to recognize, for example Figure 6 The text image shown will be converted into computer-editable text.

[0058] S32. Input the converted computer-editable text into the classifier constructed in step S2 to obtain the category of the text to be classified as "leave request". At the same time, obtain the attribute values ​​of the attributes "date", "applicant", and "time" in the text to be classified as "March 3, 2013, March 9 of this year", "XXX", and "two months" respectively.

[0059] S33. Ontology instantiation: The category, attributes, storage address information of the original image, and storage address information of the computer text converted from the image information of the text to be classified are encoded in OWL language and stored in the Neo4j database to form knowledge nodes.

[0060] As can be seen from the above embodiments, the present invention provides a knowledge graph-based OCR recognition result classification method. By constructing an ontology based on text classification information in a specific domain, constructing a text classifier based on the ontology, using the classifier to extract the category and key information of the OCR software recognition results, and using this information to construct a text knowledge graph, the method achieves the purpose of automatic multi-level classification of OCR recognition results and key information extraction.

[0061] It should be noted that the above embodiments are preferred implementations and should not be construed as limiting the present invention's method for dynamically creating and disbanding groups based on location. Those skilled in the art can further implement the present invention using electronic hardware, computer software, or a combination of both, by combining the modules and algorithm steps of the various examples described in the embodiments disclosed herein. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of the present invention.

[0062] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A knowledge graph-based OCR recognition result classification method, characterized in that, Includes the following steps: Construct an ontology of the knowledge graph of OCR recognition results; Construct a text classification model and a named entity extraction model to form a classifier, including: By manually labeling the category of each text, we construct the training and test text sets for the text classification model; by manually labeling all named entities in the text, we construct the training and test text sets for the named entity extraction model. Text features are extracted from the training and testing text sets of the text classification model to train the text classification model; a named entity extraction model is also trained. A classifier is formed by combining a text classification model and a named entity extraction model. A knowledge graph is constructed based on the OCR recognition results and the constructed classifier; The ontology of the knowledge graph of the OCR recognition results is a formal description of the types, attributes, and relationships between types and attributes of the OCR recognition results. The name of each text category that is manually annotated can be found in the ontology of the knowledge graph, and the name of each named entity that is manually annotated can be found in the ontology of the knowledge graph. The combined text classification model and named entity extraction model form a classifier, including: OCR software is used to recognize text images to be classified and convert the images into computer-editable text. The converted computer-editable text is input into the constructed classifier to obtain the category and attributes of the text to be classified. Store the category, attributes, original image information, and computer text information of the text to be classified into a database; A knowledge graph is built based on the OCR recognition results and the constructed classifier to achieve automatic multi-level classification of OCR recognition results and extraction of key information. Clearly define the domain of the target text for OCR recognition, collect classification information of text within the domain, define concepts in the ontology, define the hierarchy and attributes between concepts, and achieve hierarchical OCR recognition result categories.

2. The knowledge graph-based OCR recognition result classification method according to claim 1, characterized in that, The ontology for constructing the knowledge graph of OCR recognition results includes: Determine the target text domain for OCR recognition, and collect classification information of the text within the domain, including the names of each category and the attribute names of each type of text; Define the concepts in the knowledge graph ontology, define the hierarchy between concepts, and define the attributes of concepts; Use knowledge graph ontology modeling tools to encode and model the knowledge graph ontology.

3. The knowledge graph-based OCR recognition result classification method according to claim 1, characterized in that, The text feature extraction refers to the process of converting text features into vectors using text feature extraction algorithms, including TF-IDF and Word2Vec, after segmenting, removing stop words, and tagging parts of speech on the training and test text sets of this classification model.

4. The knowledge graph-based OCR recognition result classification method according to claim 1, characterized in that, The text classification model refers to a mathematical model that can extract text features from OCR recognition results and classify the OCR recognition results based on the features.

5. The knowledge graph-based OCR recognition result classification method according to claim 1, characterized in that, The named entity extraction model refers to a mathematical model that can identify entities with specific meanings in OCR recognition results, including names of people, places, organizations, time, date, currency, and percentage.

6. The knowledge graph-based OCR recognition result classification method according to claim 1, characterized in that, The training text classification model refers to the process of training machine learning models, including FastText, TextCNN, and TextRNN, using the results of text feature extraction.

7. The knowledge graph-based OCR recognition result classification method according to claim 1, characterized in that, The training of the named entity extraction model refers to the process of training machine learning models, including HMM, CRF, and LSTM+CRF models, using the training text set and test text set of the constructed named entity extraction model.

Images