A text information intelligent processing system

Abstract

The present invention discloses a text information intelligent processing system, which is characterized in that the system consists of three subsystems: text preprocessing, knowledge graph construction, and knowledge query and question-answering. The text preprocessing subsystem implements preprocessing such as document reading, scanning, and entity extraction; the knowledge graph construction subsystem extracts basic elements of the knowledge graph such as relationships in documents, and uses entity matching and knowledge fusion to incrementally update these elements to the knowledge graph and realize visualization; the knowledge query and question-answering subsystem uses the constructed knowledge graph to realize dynamic query and question-answer matching of document knowledge. Compared with the existing technology, the present invention has the functions of automatically processing various types of documents, constructing domain knowledge graphs to store, manage, and display key information therein, and realizing semantic-based knowledge search and question-answering functions for user input questions based on the constructed knowledge graph, providing technical support for improving the efficiency of related business functions in various fields.

Description

Technical Field

[0001] The present invention relates to the technical fields of knowledge graphs and optical character recognition, and in particular to a method for implementing a text information intelligent processing system. Background Art

[0002] As a representative technology in the field of new-generation artificial intelligence, knowledge graph technology can help meet the knowledge retrieval needs of operation and maintenance engineering. Specifically, it is reflected in the following aspects: 1) During the graph construction process, text-based semi-structured data can be processed and stored for utilization, thereby achieving the purpose of full-text search; 2) Knowledge graph-based search is targeted at named entities, not just text matching; 3) Named entities are connected in series using relationships, and information retrieval results can be expanded along the relationship path.

[0003] At present, some scholars have introduced it into the field of information management. For example, Wang Xue applied knowledge graph technology to the field of population information query, solving the problems of visualization and intelligent retrieval of artificial information data; Lei Jie and others used Protege tools to design the ontology of scientific research archive management, and unified the storage of related scientific research personnel information, scientific research team information, financial information, etc., paving the way for the intelligent application of scientific research archives; Zhai Xing and others used knowledge graph technology to manage health and wellness information, and at the same time provided functions such as human interaction, information promotion, and question forwarding.

[0004] Optical Character Recognition (OCR) refers to the process of analyzing and identifying image files of text materials to obtain text and layout information. In other words, the text in the image is recognized and returned in the form of text. Text recognition is based on text detection and identifies the text content, converting the text information in the image into text information. The main problem that text recognition solves is the identity of each character. The recognized text usually needs to be rechecked to ensure its correctness. Text correction is also considered to belong to this link. When the recognized content is composed of words in the vocabulary, it is called dictionary recognition (Lexicon-based), otherwise it is called dictionary-free recognition (Lexicon-free).

[0005] The existing document information retrieval technology has low automated processing and computing capabilities, high costs, cumbersome and complex document information management, poor intelligent question-answering effects, and document processing systems usually cannot take into account both intelligent question-answering and visualization functions, which is not convenient for users to use. Summary of the Invention

[0006] The purpose of the present invention is to provide a text information intelligent processing system to address the deficiencies of the existing technology. The intelligent system constructed by using a text preprocessing subsystem, a knowledge graph construction subsystem and a knowledge query and question-answering subsystem is used as a text information processing tool to realize dynamic query of document knowledge and intelligent processing of question and answer. By utilizing technologies in the fields of knowledge graphs and OCR, the automatic processing and calculation of large amounts of document data are realized, providing users with efficient document information retrieval and intelligent question-answering functions, and achieving the effect of reducing document information management and retrieval costs. The tool automatically processes various types of documents, constructs a domain knowledge graph to store, manage and display the key information therein, and implements semantic-based knowledge search and question-answering functions for user-input questions based on the constructed knowledge graph, providing technical support for improving the efficiency of related business functions in various fields.

[0007] The specific technical solution for achieving the purpose of the present invention is: a text information intelligent processing system, which is characterized by using an intelligent system constructed by a text preprocessing subsystem, a knowledge graph construction subsystem and a knowledge query and question-answering subsystem as a text information processing tool to realize dynamic query of document knowledge and intelligent processing of question and answer matching. Through the nine modules in the text preprocessing subsystem, the knowledge graph construction subsystem and the knowledge query and question-answering subsystem: a source document information extraction module, a directory-based coarse-grained graph construction module, an entity extraction module, a relationship mining and completion module, a graph data preprocessing module, a knowledge graph insertion module, a knowledge graph visualization module, a question-answer generation module and a candidate answer sorting and output module, they work together to realize document automatic processing, key information extraction, graph construction and intelligent question-answering functions.

[0008] The document automation processing refers to the use of OCR to process documents, identify and store text, pictures, and tables in the documents; the key information extraction refers to the use of regular matching and named entity recognition technology to obtain important information in the document, such as: project leader, project budget, start / end date, etc.; the graph construction refers to the construction of a coarse-grained graph of the document title hierarchy and a fine-grained graph of key information; the intelligent question and answer function refers to the user inputting questions in text form, and the system retrieves and predicts answers based on the constructed knowledge graph, and returns the answer with the highest execution rate.

[0009] The source document information extraction module extracts and stores information from the document to be parsed through the following four steps:

[0010] 1) Addressing and reading documents: Obtain the address of the document that the user wants to process and read the document content; 2) Document text information recognition (OCR technology): Use OCR technology to extract text information from documents in PDF, Word, TXT, and HTML formats; 3) Image and table information extraction and storage: Identify images and tables in documents and store them on the hard disk; 4) Text formatting: Process formatting issues such as headers and footers, correct line breaks, and the accuracy of image placement in text.

[0011] The directory-based coarse-grained graph construction module realizes the construction of a coarse-grained knowledge graph of the document title level according to the information extracted in the source document information extraction module through the following four steps: 1) Identify titles and levels: extract titles from documents in PDF, Word, TXT, and HTML formats, and calculate the level of each title; 2) Filter correct titles: filter out correct titles based on title regular expressions; 3) Build a directory tree: store titles in a tree form according to the hierarchical relationship of titles, and build links between titles and the content corresponding to the titles; 4) Build a coarse-grained graph and store it: build a coarse-grained knowledge graph based on the hierarchical relationship between titles, and store it on the server side.

[0012] The entity extraction module identifies and extracts key entities from documents through the following four steps: 1) Entity classification: presetting entity types to facilitate subsequent entity processing; 2) Entity recognition: constructing entity extraction rules, identifying several types of entities with strong regularity, building and training deep learning models, and using the models to extract entities from documents; 3) Entity screening: screening the extracted entities and deleting erroneous entities; 4) Entity disambiguation: merging and unifying different nouns (full name, abbreviation, nickname, etc.) that refer to the same entity.

[0013] The relationship mining and completion module extracts the relationship between entities obtained by the entity extraction module from the document information through the following four steps: 1) Relationship positioning: based on the position of the entity in the document, determine the range of possible appearance of the relationship between the pair of entities in the document; 2) Relationship mining: use rules to determine and extract the relationship between entities, build and train a model for extracting the relationship between entities; 3) Relationship screening: screen the extracted relationships and delete erroneous relationships; 4) Relationship disambiguation: based on existing entities and relationships, infer relationships that are not mentioned in some documents or that the system fails to extract.

[0014] The graph data preprocessing module implements the preparation work before building a fine-grained graph through the following five steps: 1) Extracting information reading: reading entities, relationships, pictures, tables and other information into the system; 2) Acquiring entity information and filtering out entity information; 3) Acquiring relationships between entities: filtering out relationship information; 4) Format processing: unifying entities and relationships into a format that is convenient for system processing; 5) Information storage: storing key information such as entities and relationships in a JSON file.

[0015] The knowledge graph insertion module inserts the entities and relationships extracted from the previous module into the knowledge graph through the following four steps: 1) Initialize the knowledge graph: configure the necessary content of the knowledge graph and create an empty knowledge graph; 2) Insert entity nodes: insert entities as nodes into the knowledge graph; 3) Insert relationships between entities: construct relationships between entities as edges between entity nodes; 4) Graph self-correction: use rules to correct the constructed knowledge graph.

[0016] The knowledge graph visualization module realizes the visualization display of the knowledge graph through the following four steps: 1) Graph node type setting: setting the node type for each node; 2) Node drawing: using relevant tools to draw nodes; 3) Node relationship drawing: drawing the relationship between nodes; 4) Graph visualization display: presenting the complete knowledge graph effect.

[0017] The question-answer generation module implements automated processing of user input questions and prediction of question-answering deep learning models through the following three steps: 1) Model construction and training: building a model and using training data for training; 2) Testing input questions: using the model to calculate the answers to input questions; 3) Answer return and formatting: formatting the answers output by the model and returning them.

[0018] The candidate answer sorting and output module obtains and outputs the most likely answer to the user input question through the following three steps: 1) Node information reading: find the node selected by the model as the answer, read the answer node and the nodes on the path; 2) Node weight calculation: use the attention mechanism to give different importance to each node based on the question features, and calculate the weight value of each node; 3) Candidate answer screening and output: screen according to the weights of the answer node and the answer path node, and output the final selected answer.

[0019] Compared with the existing technology, the present invention utilizes technologies in the fields of knowledge graphs, OCR, etc. to realize automatic processing and calculation of large amounts of document data, provide users with efficient document information retrieval and intelligent question-and-answer functions, reduce the cost of document information management and retrieval, and provide technical support for improving the efficiency of related business functions in various fields. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 This is a system architecture diagram of the present invention;

[0021] Figure 2 This is a functional framework diagram of the present invention;

[0022] Figure 3 An example of information extraction from source documents;

[0023] Figure 4 This is an example of the entity extraction effect;

[0024] Figure 5 This is an example of the effect of relation extraction;

[0025] Figure 6 This is an example of the entity alignment effect;

[0026] Figure 7 Generate renderings for the graph;

[0027] Figure 8 Generate renderings for graph relationships;

[0028] Figure 9 This is a graph visualization effect diagram;

[0029] Figure 10 This is a question-and-answer effect diagram based on the knowledge graph of the present invention. DETAILED DESCRIPTION

[0030] See Figure 1 The present invention consists of nine functional modules in three subsystems: text preprocessing, knowledge graph construction, and knowledge query and question answering. It realizes the intelligent processing of document automation processing, key information extraction, graph construction, and intelligent question and answer matching. The text preprocessing subsystem is responsible for realizing preprocessing such as document reading, scanning, and entity extraction, and supports downstream graph construction; the knowledge graph construction subsystem is responsible for extracting basic elements of the knowledge graph such as relationships in documents, and incrementally updating these elements to the knowledge graph using technologies such as entity matching and knowledge fusion, and realizing the visualization of the knowledge graph; the knowledge query and question answering subsystem is responsible for using the constructed knowledge graph to realize dynamic query and question and answer matching of document knowledge. The nine functional modules are: source document information extraction module, directory-based coarse-grained graph construction module, entity extraction module, relationship mining and completion module, graph data preprocessing module, knowledge graph insertion module, knowledge graph visualization module, question answer generation module, and candidate answer sorting and output module.

[0031] See Figure 2, the user inputs the file to be processed into the source document information extraction module for mining, extracts key information from it, and inputs it into the entity extraction module and the directory-based coarse-grained knowledge graph construction module. Entity extraction extracts entity information from key information, and the directory-based coarse-grained knowledge graph construction module mines and completes the relationship between entities based on the directory structure information in the key information. The relationship mining and completion module mines and completes the relationship between entities based on the entities and key information. The extracted relationships and entities will be used to construct the knowledge graph. The knowledge graph insertion module inserts entities and relationships into the initialized knowledge graph with pre-set node types, relationship types, etc. in the knowledge graph preprocessing module. The graph content can be displayed through the knowledge graph visualization module. When the user enters the question text, the question answer generation module will search for candidate answers from the knowledge graph. The candidate answers are sorted by the candidate answer sorting and output module and then output to the client.

[0032] The present invention is further described in detail with reference to the following specific examples and accompanying drawings. The processes, conditions, experimental methods, etc. for implementing the present invention, except for those specifically mentioned below, are common knowledge and common common sense in the art and are not particularly limited by the present invention.

[0033] Example 1

[0034] Step 1: Extracting file information

[0035] See Figure 3 The file extension types are .pdf, .docx, and .doc. First, the Jacob tool is used in the Java environment for file type conversion to convert .docx and .doc into .pdf files. Then, pdfplumber and pypdf2 tools are used to identify and extract PDF fine-grained text information, and the specific content of the PDF file is divided into three parts: specific tables, pictures, and unstructured text.

[0036] (1) The specific steps for table extraction are as follows:

[0037] 1) Identify a page of the original PDF and convert it to an image.

[0038] 2) Binarize the image using the adaptiveThreshold function in the OpenCV tool. In optical character recognition algorithm research, quickly and effectively binarizing document images is a key step in the image preprocessing stage. The Niblack algorithm calculates the binarization threshold by mathematically calculating the mean and standard deviation of the grayscale values of pixels within the neighborhood of the current target point and its template operator. When determining the image binarization, the mean m and standard deviation s of the grayscale values of pixels within an n×n neighborhood centered at (x, y) are first calculated.

[0039] The grayscale average m of the pixels in the n×n neighborhood centered at (x, y) is calculated by the following formula (a):

[0040]

[0041] The standard deviation s of the pixels in the n×n neighborhood centered at (x, y) is calculated by the following formula (b):

[0042]

[0043] The binarization threshold T(x, y) of the image is calculated by the following formula (c):

[0044] T(x, y)=k·s(x, y)+m(x, y) (c);

[0045] Where: k is a correction coefficient predetermined based on experience, generally k is 0.1 to 0.5.

[0046] The final threshold of the current observation point is determined by the following formula (d):

[0047]

[0048] 3) Use the convolution kernels of (1, 20) and (20, 1) to perform a closing operation on the image to determine the horizontal / vertical lines in the image and obtain the image used to find the table.

[0049] 4) Use the findContours and boundingRect functions of the OpenCV tool to find rectangles in the image, sort them from largest to smallest by area, and traverse them. If table_list is empty or there is a rectangle not included in table_list, it is considered a table. Otherwise, traversal stops.

[0050] 5) Traverse each table and use the PaddleOCR offline model to find text boxes within the table area. Perform rectangle detection on the area again. If there are fewer than 4 text boxes or fewer than 4 rectangles in the area, skip it.

[0051] 6) Take the rectangle list from the previous step and traverse it to determine the position of each row and column.

[0052] 7) Treat the rectangle as a cell and intercept the cell content for OCR text recognition as the cell text.

[0053] (2) The specific steps of image extraction are as follows:

[0054] 1) Use Fitz to detect the image of the current page in PDF and store it.

[0055] 2) Identify the captions existing in the page and replace the references of all the detected pictures one by one in sequence.

[0056] 3) Add the references of the redundant pictures to the end of the page.

[0057] Step 2: The specific steps of entity and relationship extraction are as follows:

[0058] (1) Entity and relationship extraction in the table

[0059] 1) Detection of multi-page tables

[0060] Before converting the table into a graph spectrum, it is necessary to detect multi-page tables first. For two adjacent tables in the recognition results, first judge whether these two tables are on adjacent page numbers. At the same time, according to the number of columns of the two tables, their positions in the text page, and the table names, judge whether these two tables belong to multi-page tables. If it is judged as a multi-page table, merge the contents of the two tables.

[0061] 2) Table name recognition

[0062] The name of the table entity adopts the table name in the materials. Extract the context (adjacent text) of each recognized table in the document, and use templates to match these context texts to find the text content that conforms to the characteristics of the table name. The template form is ".*[Table](\s*)[0 - 9a-zA-Z]*.*", which matches the text containing the displayed table content.

[0063] 3) Recognition of the first basis and the second basis

[0064] In the process of converting the table into a graph spectrum, first estimate which category the table belongs to according to a series of predefined Chinese and English keywords (such as "interface", "weight", "parameter", "factor", "frequency", etc.). At the same time, calculate the rows and columns where the first basis and the second basis start. During this period, it is necessary to consider whether the reference basis spans rows or columns, and also consider how many layers of the reference basis are required to uniquely determine an attribute.

[0065] (2) Entity and relationship extraction in unstructured text

[0066] Refer to Figures 4 and 5 , manually label some data (including part-of-speech tagging and word segmentation), cooperate with the model to learn and generalize to other data to train a deep learning model, and use the Baidu word segmentation tool LAC to obtain keywords from nouns (n), other proper names (nz), ordinary verbs (v), adjectives (a), and punctuation marks (w).

[0067] 1) Entity Alignment: If there is a high degree of character overlap between two entities, it is considered that the two names are likely to refer to the same entity. Based on this, the grammatical structure and meaning of the sentences containing the two entities are further analyzed to determine the grammatical components of the entities in the sentences. The Levenshtein ratio LRx,y between short texts X and Y is calculated using the following formula (e):

[0068]

[0069] Where: len X with len Y is the length of short text X, Y; Idist X，Y It is the Levenshtein distance between texts. Compared with the original Levenshtein distance, the addition and deletion operations are still +1, but the replacement operation is +2. This is done to avoid similar calculations of LR. "a"，"b" ≠ 0. However, Levenshtein ratio does not consider the impact of common substrings between short texts on text similarity, so it is necessary to calculate the common substring ratio D by the following formula (f): X，Y :

[0070]

[0071] Where: CSlen X，Y is the length of the longest common substring between short texts X and Y. In this paper, there will be some pure numbers ε represented by the following formula (g) X，Y :

[0072]

[0073] Therefore, the similarity P of short text X and Y X，Y Calculated by the following formula (h):

[0074]

[0075] Where: W LR With W D are the weights of the corresponding parameters, taking 1 and 0.8.

[0076] See Figure 6 If both entity similarity and entity semantics are similar, the two names are considered to be different names of the same entity.

[0077] Step 3: Build a knowledge graph

[0078] 1) When two entities appear in similar contexts, they can be considered to be interconnected and there is an edge between the entity nodes.

[0079] 2) Two entities are at the same level or in a superior-subordinate relationship in the project document directory, and there may be a subordinate relationship. It can be considered that there is an edge between the two entities.

[0080] 3) After disambiguation, two entities point to the same entity. It is considered that the two entities are the same at the semantic level, and there is an edge between the entity nodes.

[0081] See Figure 7 ,If two entities satisfy one of the above rules, it can be considered that there is a relationship edge between the two entity nodes in the knowledge graph.

[0082] Step 4: Graph visualization

[0083] See Figures 8 and 9 , Neovis visualization tool projects node information to the front-end web page, the effect is as follows Figure 9 shown.

[0084] Step 5: Question Semantic Analysis

[0085] (1) According to the TF-ID algorithm, the semantic features of the text are obtained by constructing the frequency of occurrence of words in the document. The calculation is expressed by the following formula (i):

[0086]

[0087] Where: tf ij is the characteristic term t j In the document d i The number of times it appears in the j Used to represent the inverse ratio to t j The number of occurrences in all texts; N represents the total number of documents; n j For the occurrence of feature t j The number of documents, to prevent n j is 0, corrected to n j +1.

[0088] (2) The specific construction process of the model is as follows:

[0089] 1) Query all nodes from neo4j database.

[0090] 2) Merge the paragraph title and paragraph text in the node as training corpus.

[0091] 3) Use the jieba module in entity_solver to segment all corpora and remove stop words.

[0092] 4) Delete low-frequency words with a frequency of 1 after word segmentation.

[0093] 5) Build a bag-of-words model and construct a TF-IDF model.

[0094] 6) Create a text similarity matrix.

[0095] Step 6: Answer retrieval

[0096] 1) Regularize input. This includes normalizing spaces between Chinese and English, spaces between Chinese characters, extra spaces between English characters, spaces around punctuation marks, and standardizing the case of English characters (e.g., making them all lowercase).

[0097] 2) Segment the question and obtain word vectors in the same way as in the construction method.

[0098] 3) Query the top k similar sentences based on the text similarity matrix and return them.

[0099] 4) The final answer is returned in Figure 10 The question and answer effect diagram shown.

[0100] The above is only a further explanation of the present invention and is not intended to limit this patent. Any equivalent implementation of the present invention should be included in the scope of the claims of this patent.

Claims

1. A text information intelligent processing system, characterized in that: The intelligent system constructed by using text preprocessing subsystem, knowledge graph construction subsystem and knowledge query and question answering subsystem is a text information processing tool to realize semantic-based knowledge search and question answering. The text preprocessing subsystem includes: source document information extraction module, directory-based coarse-grained graph construction module and entity extraction module; the source document information extraction module is used to extract information from source documents; the directory-based coarse-grained graph construction module is used to analyze the document directory structure and construct a tree-like knowledge graph based on it; the entity extraction module is used to extract key entity information from document information; the knowledge graph construction module is used to extract key entity information from document information. The construction subsystem includes: a relationship mining and completion module, a graph data preprocessing module, a knowledge graph insertion module, and a knowledge graph visualization module. The relationship mining and completion module is used to extract key relationship information from document information and complete missing relationships; the graph data preprocessing module is used to implement graph data preprocessing, including connection information and edge information of nodes in the graph and mapping information of node identifiers and node names; the knowledge graph insertion module is used to construct a knowledge graph and insert entities and relationships into the knowledge graph; the knowledge graph visualization module is used to visualize the knowledge graph; the knowledge query and question-answering subsystem includes: a question answer generation module and a candidate answer sorting and output module. The question answer generation module is used to find and calculate suitable candidate answers based on the input question; the candidate answer sorting and output module is used to find the answer with the highest confidence from the candidate answers and output it, so as to realize dynamic query of document knowledge and intelligent processing of question-answer pairing; the knowledge graph insertion module inserts the entities and relationships extracted in the previous module into the knowledge graph, which specifically includes the following steps: 1) Initialize the knowledge graph Use Cypher statements to initialize the graph, which clears all entities and relationships in the knowledge graph and clears the maintained entity and relationship tables; 2) Entity node insertion: Use Cypher statements to insert entities in a relationship triple. First, determine the entity type based on the relationship, then number and name it. For entities in table data, record their various attributes in the entity node. If an entity with the same name is encountered during the insertion process, check whether its attribute information is exactly the same to eliminate the possibility of duplicate insertion of the same node. 3) Inserting relationships between entities Use Cypher statements to insert relations in the relation triples. First, determine the relation type based on the relation template, then establish the relation structure. After obtaining the relation structure, query the knowledge graph based on the names of its head and tail entities, and connect the queried head and tail entities and relations. 4) Graph self-correction During the process of inserting entities and relationships, the graph will correct the inserted data based on the maintained entity table and relationship table. For unique entities, if repeated insertion operations occur, the system will report an error and feedback the duplicate node information based on the entity table.

2. The text information intelligent processing system according to claim 1, characterized in that The source document information extraction module extracts and stores information from the document to be parsed, specifically including the following steps: 1) Addressing and reading documents The document address entered by the user on the web page is transmitted to the server using POST technology. The server finds the corresponding path file. If the file is in PDF format, it is directly stored in the memory; if it is in Word or HTML format, the file content is read in read-only mode and saved in PDF format. 2) Document text information recognition Use the text feature extraction algorithm provided by the CNOCR model to analyze and identify the content read by the addressing and reading document operation, and extract text and layout information; 3) Image and table information extraction and storage Use the fast-rcnn deep learning image detection model to extract table and picture content, and determine whether the recognized content has header and cell table features, so as to distinguish whether the recognized content is a table or a picture. The extracted tables and images are stored in the hard disk, and the access addresses are recorded in the document text content; 4) Text formatting Use positive and negative matching to perform the following processing: locate and delete headers and footers, identify whether there are line breaks after the image and table position information, if not, it means that the text after the image and table is not correctly wrapped, then add line breaks, determine whether there are blank lines or lines with only meaningless symbols, and delete them if they exist.

3. The text information intelligent processing system according to claim 1, characterized in that The directory-based coarse-grained graph construction module constructs a coarse-grained knowledge graph at the document title level based on the extracted information, specifically including the following steps: 1) Identify titles and levels Based on the characteristics of titles in the text, rules are generated, such as the presence of a serial number before the title and a comma or semicolon after the serial number. Based on these rules, titles are found in the document and classified into the correct hierarchy based on the title number and label type. If the document is in HTML format, the hierarchical relationship of the HTML language can be used as auxiliary information to query the title in the document. 2) Filter the correct title For the filtered titles, the correct titles are filtered out according to regular expressions. The regular expressions need to be specified according to the specific content and writing style of the text; 3) Build a directory tree: After filtering the title with a regular expression, obtain the title and its hierarchy used to construct the directory tree. Utilize the title hierarchy relationship to construct a directory tree in a tree-like structure, save it locally in the form of a dictionary, and link the title with the content corresponding to the title. 4) Constructing and storing coarse-grained graphs A coarse-grained knowledge graph is constructed based on the hierarchical relationships between titles and stored on the server. Subsequently, the relationship mining and completion module performs entity extraction, relationship mining and completion to further improve the knowledge graph.

4. The text information intelligent processing system according to claim 1, characterized in that The entity extraction module extracts key entity information from document information, specifically including the following steps: 1) Entity Classification Based on the document content, the entity types that frequently appear in the document are preset and classified; 2) Entity Recognition Construct entity extraction rules to identify several types of entities with strong regularity. Use predefined part-of-speech (POS) tag rules to identify high-quality noun phrases in documents. Build and train a deep learning model, using the model's bidirectional LSTM combined with conditional random fields to extract entities from documents. 3) Entity screening Based on the statistical learning method, the entity vocabulary is screened by scoring and ranking the vocabulary according to the statistical index characteristics of the candidate phrases. The statistical indicators include: TF-IDF, PMI and C-Value; 4) Entity Disambiguation Use a bootstrapping method based on pattern matching to automatically discover new patterns, prepare seed samples or define initial patterns, match the corpus with the patterns, discover new synonym pairs, and mine new patterns based on the newly discovered synonym pairs. Repeat the above steps until the system determines that no more synonym pairs can be found.

5. The text information intelligent processing system according to claim 1, characterized in that The relationship mining and completion module extracts the relationships between entities obtained by the entity extraction module from the document information, specifically including the following steps: 1) Relationship positioning Based on the location of the entities in the document, the possible range of the relationship between the two entities is determined in the document, and the accuracy of entity classification is enhanced by narrowing the range; 2) Relationship Mining Use patterns to express the way relationships are mentioned in text, match the patterns with corpus, obtain and implement relationship instances, build and train a relationship extraction deep learning model to extract relationships between entities, use annotated corpus to learn the extraction model, accept text as input, input it into the embedding layer and generate the corresponding tag sequence through CRF. Each tag indicates whether the corresponding character is an entity and a relationship. The pattern can be divided into character pattern, grammatical pattern and semantic pattern according to granularity; the character pattern regards natural language as a character sequence, and the pattern is represented as a set of regular expressions; the grammatical pattern is a pattern for extracting lexical and syntactic information; the semantic pattern introduces concepts into the description of the pattern and defines a pattern based on concept constraints; 3) Relationship screening Combined with the lexical, syntactic, and semantic information or background knowledge in the context, the extracted relationships are screened and incorrect relationships are deleted. The screening uses a sentence-level attention mechanism to assign a weight to each sentence in the entity pair. A larger weight indicates that the sentence expresses the target relationship more strongly, while a smaller weight indicates that the sentence is more likely to be noise. 4) Relationship disambiguation Based on the matrix and translation schemes - TransH and TransD, prediction is achieved by using the vector relationship between head and tail entities and relations in a certain space.

6. The text information intelligent processing system according to claim 1, characterized in that The preparatory work before the atlas data preprocessing module constructs a fine-grained atlas specifically includes the following steps: 1) Extract information and read Read the image address and table address, unify their formats using the preset template, and store them in the memory in the form of a dictionary; 2) Entity Information Acquisition The extracted entities in the entity table are screened for quality, duplicate entities are deleted, and similarity matching is used for entities. For entity pairs with a similarity greater than 0.8, a semantic analysis algorithm is used to determine whether they refer to the same entity. If so, a disambiguation operation is performed. The completeness of each entity is then determined, and the entity is segmented using the Jieba word segmentation tool. If a single character appears in the segmentation result, the entity is judged to have quality problems and is deleted and the entity list is reorganized. 3) Obtaining relationships between entities The relationship table is quality-screened using similarity matching. For relationship pairs with a similarity greater than 0.6, a semantic analysis algorithm is used to determine whether they refer to the same relationship. If so, a disambiguation operation is performed. For each relationship, the corresponding head and tail entities are found in the entity table through similarity matching. If the head or tail entity is missing, the relationship is deleted. 4) Format processing Integrate the relational table and the entity table, combining the entities in the entity table into triples of "head entity-relationship-tail entity" through the relationships in the relational table. De-duplicate the triples and store the resulting triples in memory as a dictionary. 5) Information storage Store the triple dictionary in JSON format and return the storage path.

7. The text information intelligent processing system according to claim 1, characterized in that The knowledge graph visualization module implements the visualization of the knowledge graph according to the following steps: 1) Graph node type settings When building a visual knowledge graph, the display architecture of the knowledge graph is designed according to the number and types of nodes and relationships. Specifically, different types of nodes are assigned corresponding colors, and each node is assigned a corresponding logo based on its meaning. 2) Node drawing Based on the background data of the knowledge graph, the node information is projected onto the front-end web page using the Neovis visualization tool. The previous circular nodes are replaced with corresponding logos according to their types, and the node sizes are adjusted according to their importance in the knowledge graph. 3) Drawing relationships between nodes Based on the backend data of the knowledge graph, we first determine the relationship between two entities in the frontend. Then, we use Neovis to create a relationship arrow with a specific color and name based on the relationship type. Then, we connect the arrow to the head and tail entities respectively. 4) Graph visualization After establishing all entities and nodes, use Neovis to activate the visual knowledge graph so that the graph can be displayed in a dynamic form, and insert common operation functions such as deletion, modification, insertion, and query into the visual interface. When the corresponding button is clicked, the command information will be transmitted to the server through POST technology. The server will perform corresponding operations according to the instructions. When the front end receives the update information, it will update the graph.

8. The text information intelligent processing system according to claim 1, characterized in that The question answer generation module searches for and calculates suitable candidate answers based on the input question, specifically including the following steps: 1) Model construction and training A graph neural network is used as the main framework of the answer prediction model. The graph is read into the model through the knowledge graph reading interface to obtain the contextual features of each node in the knowledge graph. By propagating and learning the structural information around the node, a total of 1,371 public, non-confidential documents were collected from the internet using a Python crawler in advance as training data. This training data is input into the model to train the model and allow it to find potential patterns in the data. 2) Test input problem The TF-IDF algorithm is used to construct the frequency of word occurrences in documents to obtain the semantic features of the text and obtain relatively superficial semantic information. A pre-trained language model is used to obtain the contextual semantic information of the text. Through deep neural network training, semantic features including the context of the question and reasoning information are mined. 3) Answer return and format processing The answer node location information and path information found by the answer prediction model are saved in the form of tuples and passed to the candidate answer sorting and output module to facilitate the subsequent calculation of the candidate node weights.

9. The text information intelligent processing system according to claim 1, characterized in that The candidate answer ranking and output module finds the answer with the highest confidence from the candidate answers and outputs it, specifically including the following steps: 1) Node information reading Use the depth-first algorithm to traverse the GNN model of the question answer generation module to find all nodes on the answer path, access the answer nodes and path nodes, and read their information; 2) Node weight calculation After using a graph neural network to learn the structural information of the knowledge graph, the optimal search node is obtained by calculating the similarity between question features and node features. For question text features and node features, a single-layer neural network is constructed to map the two features to the same vector space. After that, a question-based attention mechanism is constructed. The question features are used to assign different importance to each node, and the weight value of each node is calculated. Based on the weight value, the total weight representation of all nodes is obtained. 3) Candidate answer screening and output The weights of the candidate answer nodes and path nodes are weighted and summed to calculate the total confidence of the candidate answers. The answer with the highest confidence is transmitted to the web page using POST technology, output to the text box, and displayed to the user.

Images