Government affair file information extraction and question and answer method and device and medium

Abstract

The invention relates to a government affair file information extraction and question answering method and device and a medium, and the method comprises the steps: carrying out the entity extraction of a government affair file through employing a BERT-CRF joint model, and obtaining a structured entity set; performing relation extraction on the structured entity set to generate a semantic relation set between the entities; constructing a knowledge graph according to the structured entity set and the semantic relationship set, storing entity nodes into a graph database, and storing an embedded vector of an entity text into a vector database; when a query request of a user is received, relation query of the graph database and semantic retrieval of the vector database are carried out, sub-graph structures and semantic matching vectors related to query are extracted, and a mixed retrieval result is obtained; and inputting the mixed retrieval result into a large language model, and generating a question and answer response text conforming to a preset format by applying a dynamic prompt template. According to the method, the document processing efficiency and accuracy are effectively improved, and a solid technical support is provided for intelligent management of government affair documents.

Description

Technical Field

[0001] The present invention relates to the field of government information technology, and in particular to a government document information extraction and question-answering method, device and medium. Background Art

[0002] In terms of technological evolution, methods for extracting elements from official documents have undergone an iterative evolution, from rule-based templates and statistical machine learning to deep learning and large language models (LLMs). Early rule-based template methods (such as regular expressions and XPath location techniques) were effective in scenarios with fixed formats, but they required frequent rule adjustments and were costly to maintain. Statistical methods (such as CRFs and SVMs) broke through rule limitations but relied heavily on manual feature engineering, making it difficult to capture latent semantic features of discontinuous distributions. Deep learning methods (such as BiLSTM-CRFs and the multimodal model LayoutLMv3) have made some progress by automatically learning sequence features and integrating text layouts, but they still face significant bottlenecks in handling heterogeneous official documents (such as those with variable field positions) and recognizing specialized terminology (coverage of less than 35%). In recent years, large language models (such as BERT and GPT) have demonstrated strong semantic understanding capabilities, but they lack the ability to perceive the unique layout structure of official documents and have limited accuracy in recognizing domain-specific terms.

[0003] Existing technologies face three core challenges: First, format heterogeneity results in a recall rate of less than 60% for conventional layout analysis algorithms. For example, the position of key fields such as "main addressee" varies significantly across different institutions' documents. Second, insufficient contextual support leads to a 41% drop in the F1 score of the BiLSTM model for isolated short sentences. Third, significant domain knowledge barriers exist, resulting in low accuracy in recognizing specialized terminology in fields like military and diplomacy. Although retrieval-augmented generation (RAG) and knowledge graph fusion technologies (such as the LightRAG framework) have improved some entity recognition performance through knowledge augmentation mechanisms, their limited structural parsing capabilities and lack of domain knowledge bases continue to hinder their application in complex document scenarios.

[0004] There is an urgent need for a technical solution that integrates deep semantic understanding and structured feature perception to solve the problems of format sensitivity, semantic complexity and domain adaptability in official document element extraction, thereby providing reliable support for efficient management and intelligent decision-making of government documents. Summary of the Invention

[0005] The present invention provides a government document information extraction and question-answering method, device and medium, which aims to solve the problem of insufficient accuracy in government document information extraction due to format sensitivity, semantic complexity and domain knowledge barriers.

[0006] To achieve the above objectives, the present invention provides a first aspect of a government document information extraction and question-answering method, comprising the following steps: The BERT-CRF joint model is used to extract entities from the input government documents, obtaining a structured entity set including the document number, issuing unit, and text content; Performing relationship extraction on the structured entity set to generate a set of semantic relationships between entities; Constructing a knowledge graph based on the structured entity set and the semantic relationship set, storing the entity nodes in a graph database, and storing the embedding vectors of the entity texts in a vector database; When receiving a user query request, a hybrid search operation is performed: a relational query in the graph database and a semantic search in the vector database are performed to extract the subgraph structure and semantic matching vectors related to the query, and obtain the hybrid search results; The mixed retrieval results are input into a large language model, and a dynamic prompt template is applied to generate a question-answer response text that conforms to a preset format.

[0007] Furthermore, the method for extracting entities from input government documents includes: Preprocess government documents to generate text sequences that meet the BERT encoder input specifications; The pre-processed government documents are fed into the BERT-CRF model, and character-level contextual features are extracted through BERT's multi-head self-attention mechanism. Applying state transition constraint rules in the CRF layer, decoding generates the optimal label sequence that conforms to the document entity boundary; According to the continuous entity fragments in the optimal label sequence, corresponding fields are intercepted from the original text of the government document to generate a structured entity set including the document number, the issuing unit and the text content.

[0008] Furthermore, the method for performing relationship extraction on the structured entity set includes: For entity pairs in the structured entity set, extracting their context texts to form sentence pair inputs, wherein the sentence pair inputs include the entity and its preceding and following semantic contexts; Calling the relationship classification module of the Deepseek API framework, performing relationship reasoning on the sentence pair input based on the Transformer architecture, and outputting classification results of the official document release relationship and the copy-to relationship; The BERT-CRF model is used in parallel to perform sequence annotation on the text between entities. The BIO annotation system is used to identify continuous text segments containing temporal relationships to obtain sequence annotation results. The classification results are merged with the sequence labeling results, and standardized mapping is performed according to the preset official document relationship type table to generate a semantic relationship set of triple structure, among which the time association relationship is generated by binding the time segment in the sequence labeling result with the official document entity.

[0009] Furthermore, the knowledge graph construction method includes: The entities in the structured entity set are stored in the graph database as nodes, and the node attributes include the entity type and the original text fragment; The relationships in the semantic relationship set are stored in the graph database as labeled edges. The edge labels correspond to predefined document relationship types, and the embedding vectors of the entity text are simultaneously stored in the vector database. Execute the incremental update algorithm to perform entity deduplication verification on the newly added document data: if the entity already exists in the graph database, directly establish a relationship edge with the new document node; if the entity does not exist, create a new node and establish a complete relationship topology; A knowledge graph is constructed based on the node-edge-node triple structure to ensure that the same entity nodes remain unique across document scenarios, and the semantic similarity index between entities is maintained through a vector database.

[0010] Furthermore, the incremental update algorithm includes: Generate an entity unique identifier through the joint hash calculation of the entity name and attribute value; Traversing the graph database nodes based on the unique identifier and performing entity existence verification; When it is detected that an entity in a newly added document has the same unique identifier as an existing node, a relationship edge is established between the node and the new document node; When a new entity is detected, a new node is created and a complete relationship topology is established between it and the associated document nodes; The embedding vectors of the newly added entities are synchronously updated in the vector database, and the semantic similarity index is rebuilt.

[0011] Furthermore, the method for performing the hybrid search operation includes: Locate the starting node based on the entity keywords in the user query, traverse along the predefined relationship edge types in the knowledge graph, and extract the subgraph structure containing the document nodes and their associated relationships; Convert the user query text into an embedding vector, perform semantic matching with the stored entity vectors, and filter out the entity set related to the query intent; Graph embedding technology is used to jointly analyze subgraph structures and semantic matching entities to generate cross-document retrieval results. The search results are filtered for timeliness based on the timestamp of the official document issuance, and the latest version of the official document information is retained first to form the final mixed search results.

[0012] Furthermore, the method for constructing the dynamic prompt template includes: Establish a preset template library including entity and relationship extraction templates, retrieval enhancement generation templates, keyword extraction templates and hybrid information retrieval templates; Set separators in entity and relationship extraction templates to standardize the output format of entity and relationship triples; A timestamp mechanism is embedded in the retrieval enhancement generation template. When multiple versions of knowledge conflicts are detected, the answer priority is sorted based on semantic matching and timeliness. Configure a JSON output framework for the keyword extraction template to automatically extract high-level concepts and specific details from the query; A cross-modal comparison mechanism is set up in the hybrid information retrieval template to compare the credibility of the knowledge graph query results with the vector database search results, and the reference source is marked in the response text; The format of the final generated question and answer response text is standardized through the Markdown structured output constraint mechanism.

[0013] Furthermore, the method further comprises: In the entity extraction stage, the layout characteristics of government documents are analyzed to determine the layout type, which includes paragraph structure distribution, title hierarchy and signature position characteristics; Generate a corresponding layout identifier according to the layout type, and insert the layout identifier into the starting position of the input sequence; In the input sequence of the BERT-CRF model, the text characters are concatenated with the layout position codes to form an enhanced context representation vector, where the layout position codes are generated by a two-dimensional coordinate mapping algorithm and reflect the physical position of the text in the original document; In the relationship extraction stage, relationship reasoning of entity pairs is performed based on the enhanced context representation vector, wherein the layout identifier participates in the attention weight calculation of the Transformer architecture, guiding the model to focus on layout-related context features.

[0014] To achieve the above-mentioned purpose, the second aspect of the present invention provides an electronic device, including a memory and a processor, wherein the memory is used to store a program that supports the processor to execute the government document information extraction and question-and-answer method, and the processor is configured to execute the program stored in the memory.

[0015] To achieve the above-mentioned purpose, the third aspect of the present invention provides a computer-readable storage medium, on which a computer program is stored. When the computer program is run by a processor, the steps of the government document information extraction and question-and-answer method are executed.

[0016] Beneficial effects of the present invention: Compared with the existing technology, the present invention provides a method, device and medium for government document information extraction and question-answering, which effectively solves the problem of insufficient accuracy in government document information extraction due to format sensitivity, semantic complexity and domain knowledge barriers by introducing the BERT-CRF joint model and hybrid retrieval technology. First, the BERT-CRF model combines deep language understanding and sequence labeling capabilities to automatically extract key entities in government documents, breaking through the limitations of traditional methods in format heterogeneity; secondly, through relationship extraction, semantic relationships between entities are established, and combined with graph databases and vector databases, efficient knowledge graph storage and semantic retrieval are achieved, overcoming the shortcomings of traditional retrieval methods in cross-document queries and professional terminology recognition; finally, the design of dynamic prompt templates combined with large language models optimizes the question-answering generation process and ensures the high accuracy and consistency of question-answering results. Compared with traditional methods, the present invention significantly improves the accuracy of information extraction when processing complex official document scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments.

[0018] Figure 1 This is a flowchart of a government document information extraction and question-answering method disclosed in an embodiment of the present invention.

[0019] Figure 2 This is a Transformer structure diagram disclosed in an embodiment of the present invention.

[0020] Figure 3 This is a structural diagram of a BERT-CRF model disclosed in an embodiment of the present invention.

[0021] Figure 4 This is a relationship extraction flow chart disclosed in an embodiment of the present invention.

[0022] Figure 5 This is a diagram of an example scenario structure of a knowledge graph disclosed in an embodiment of the present invention.

[0023] Figure 6 This is a retrieval mode flow chart disclosed in an embodiment of the present invention.

[0024] Figure 7 This is a loss function comparison diagram disclosed in an embodiment of the present invention.

[0025] Figure 8 This is a performance indicator comparison chart disclosed in an embodiment of the present invention.

[0026] Figure 9 This is a training time comparison chart disclosed in an embodiment of the present invention. DETAILED DESCRIPTION

[0027] In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the embodiments described are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of the present invention.

[0028] According to an embodiment of the present invention, it should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the following production method, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0029] like Figure 1 As shown, the present invention provides a government document information extraction and question-answering method, comprising the following steps: Step S100: Use the BERT-CRF joint model to extract entities from the input government documents to obtain a structured entity set including the document number, issuing unit, and text content; Step S200: performing relationship extraction on the structured entity set to generate a semantic relationship set between entities; Step S300: construct a knowledge graph based on the structured entity set and the semantic relationship set, store the entity nodes in a graph database, and store the embedded vectors of the entity texts in a vector database; Step S400: When a user query request is received, a hybrid search operation is performed: a relational query of the graph database and a semantic search of the vector database are performed to extract subgraph structures and semantic matching vectors related to the query, and obtain hybrid search results; Step S500: input the mixed search results into a large language model, and apply a dynamic prompt template to generate a question-answer response text that conforms to a preset format. In this embodiment, as described in step S100 above, government documents contain a large number of structured elements, such as document number, issuing unit, text content, issuing time and other information. In order to achieve efficient and accurate information extraction, this method combines BERT and CRF for automatic extraction.

[0030] First, BERT (Bidirectional Encoder Representations from Transformers) is a pre-trained language representation model that uses a bidirectional encoder based on the Transformer architecture. Figure 2The bidirectional encoder module of the Transformer architecture is demonstrated. It consists of multiple layers of stacked encoders, each of which includes two processing stages: first, the input sequence is converted into a vector representation containing semantic and positional information through word embedding and positional encoding; the input is then passed through a multi-head self-attention mechanism to calculate global semantic associations, and residual connections and layer normalization are used to stabilize gradient propagation; then, a fully connected feedforward layer further deepens the feature representation, and residual connections and layer normalization are performed again to optimize training stability. The stacking design of multiple encoder layers (Transformer modules * N) gives the model powerful contextual modeling capabilities, can capture long-range dependencies, and ultimately outputs deep semantic features through the top fully connected layer, providing efficient structured semantic representations for downstream tasks.

[0031] BERT's pre-training process includes the Masked Language Model (MLM) and Next Sentence Prediction (NSP). The MLM task randomly masks some words in the input sentence and requires the model to predict these words, thereby learning the contextual relationships between words. The NSP task requires the model to determine whether two sentences appear consecutively in the original text, helping the model learn the logical and semantic connections between texts. This allows BERT to capture richer semantic information within the context.

[0032] However, while BERT provides powerful semantic understanding at the word level, it cannot directly handle dependencies between labels in named entity recognition (NER) tasks. For example, when recognizing that the labels "B_LOC" and "I_LOC" belong to the same entity, BERT cannot automatically establish this relationship. Therefore, this step adds a conditional random field (CRF) layer to BERT to optimize the accuracy of sequence labeling.

[0033] Conditional Random Fields (CRFs) are discriminative models used for sequence labeling tasks. In named entity recognition, CRFs can ensure the consistency of entity labels by modeling the dependencies between labels. For example, they can ensure that a person's name is not mistakenly split into multiple entities. Compared to traditional classification methods, CRFs can effectively exploit the dependencies between previous and next labels, thereby improving the accuracy of entity recognition.

[0034] Therefore, this step adds a CRF layer to the BERT output layer to enhance the modeling of dependencies between labels. The CRF layer optimizes the prediction of label sequences through conditional probability calculations, ensuring that the prediction of each label remains consistent with the preceding and following labels. This process ensures that the entities extracted from the text are complete and accurate. When processing complex official documents, the CRF layer can effectively improve the overall performance of the model, ensuring that all named entities (such as document number, issuing unit, and document content) are correctly extracted from the text.

[0035] To ensure that the entire model is adaptable to the specific named entity recognition task, the BERT-CRF model was fine-tuned on the annotated NER dataset. Through fine-tuning, BERT can adjust its internal parameters according to the characteristics of the target domain to adapt to the specific language structure and context. In addition, during the fine-tuning process, the CRF layer will further optimize its parameters based on the dependencies between sequence labels, thereby improving the accuracy of entity recognition. During the fine-tuning process, the negative log-likelihood (NLL) loss function was used. This loss function can optimize the prediction accuracy of the label sequence for sequence labeling tasks. Specifically, the NLL loss function is as follows:

[0036] in, is the negative log-likelihood (NLL) loss function, which represents the negative log probability of the model predicting the correct label sequence. Represents a given input sequence Time label sequence The conditional probability of Indicates that at time step ,Label and input The corresponding launch score, is the total length of the input sequence (total number of time steps), Represents the true label sequence The total score is obtained by accumulating the emission scores at each time step. Represents a candidate label sequence The loss function encourages the model to generate label sequences with higher probabilities, thereby optimizing the prediction of labels.

[0037] Through the design of this joint model, it is possible to accurately extract various key information from the text when processing government documents and organize this information in the form of structured entities.

[0038] Figure 3The structural diagram of the BERT-CRF model is shown. The input layer of the BERT-CRF model is composed of a concatenation of word vectors, position vectors, and segment vectors. The encoder establishes global semantic associations through a multi-head self-attention mechanism, which is then passed to the feedforward network via a residual connection. Finally, the CRF layer optimizes the label sequence based on state transition rules to ensure the accuracy of entity boundaries. This process significantly improves entity recall for heterogeneous official documents and addresses the performance degradation of the traditional BiLSTM-CRF model in short sentence scenarios. The specific implementation process is as follows: Input government documents (such as original official documents) need to be preprocessed to convert them into text sequences that meet the BERT encoder input specifications. This includes: Text segmentation: Divide text into blocks according to the paragraphs of the document to ensure the effectiveness of long text processing; Character-level segmentation: The Chinese BERT model is based on character-level processing, without the need for additional segmentation, and directly inputs the original character sequence; Input encoding: Generates "word vectors", "position vectors" and "segment vectors" of the input sequence, which represent the semantics of the characters, position information and paragraph affiliation respectively.

[0039] The preprocessed text sequence is fed into the BERT model, where character-level contextual features are extracted via a multi-layer Transformer encoder. The specific mechanism includes: a multi-head self-attention mechanism to compute global semantic associations between characters, for example, capturing the dependency between the "issuing document number" and its context; after each encoding layer, layer normalization and residual connections are used to stabilize gradient propagation, enhancing the model's ability to capture deep semantic features; a feedforward network then further deepens the feature representation to extract nonlinear semantic information. The hidden state output by BERT is mapped to the label space via a linear layer and then fed into a CRF layer for sequence decoding. The CRF uses a state transition matrix to constrain the logical relationships between labels, avoid label conflicts, and generate a globally optimal entity annotation sequence. Finally, based on the label sequence, continuous entity fragments are extracted from the original text to form a structured entity set. The extracted entities are categorized and stored by type (issuing document number, issuing unit, document content, etc.) and adapted to the target database format.

[0040] In this embodiment, as described in step S200 above, a relationship extraction task is further performed based on the structured entity set extracted in step S100. The purpose of relationship extraction is to identify and extract semantic relationships between entities, such as the relationship between the "issuing number" and "issuing unit" of an official document, or the temporal association between the "issuing unit" and "issuing time." Relationship extraction can reveal the logical relationships between different entities in a document, thereby generating a complete set of semantic relationships.

[0041] This process employed a deep learning-based approach, combining the BERT-CRF model with the relationship classification module of the Deepseek API framework. First, using the entity information extracted by the BERT-CRF joint model, contextual analysis was performed for each entity pair. By extracting the sentences or paragraphs containing the entities and converting them into sentence pairs, it was possible to further determine whether a semantic relationship existed between the two entities. For example, by analyzing the context of the two entities in an official document, we could determine whether they were related by "publication," meaning that a specific unit had issued the document.

[0042] The relationship classification module in the Deepseek API framework is based on the Transformer architecture. By performing relational reasoning on sentence pairs, it can classify and identify different types of relationships. For example, the "Publish" and "Cc" relationships in official documents can be efficiently classified using this module. Deepseek's advantage lies in its ability to fully consider contextual details, thereby reducing the misclassification problems that occur in traditional methods due to inadequate rule-based and artificial feature design.

[0043] After completing relationship classification, the BIO annotation method is used to sequentially annotate text segments between entities. By annotating the entity's beginning (B-), internal (I-), and non-entity portion (O), the BIO annotation system helps the model better understand entity boundaries and the semantic relationships between them within the text. This allows for accurate identification of text segments containing temporal relationships, such as the temporal relationship between "issuing time" and "issuing unit." The annotated results are organized into a structured triple format (entity-relationship-entity), further forming a set of semantic relationships.

[0044] On this basis, the triple-form relationships are standardized and mapped (see Tables 1-5). The extracted relationships are then normalized using a predefined table of official document relationship types. For example, relationships such as "issuing unit - publication - official document" and "official document - inclusion - body" can be identified and classified, and converted into standardized triples (e.g., <issuing unit, publication, official document>).

[0045] Figure 4 This is a flowchart for relation extraction. Entity pairs and their context are processed by the Deepseek API's relation classification module, which outputs classification results. Simultaneously, the temporally related text is processed using the BERT-CRF sequence labeling model to extract continuous segments. These results are combined and mapped into standard triples according to the relation type tables in Tables 1-5. These are then stored in a graph database (entity nodes and relationship edges) and a vector database (entity and relationship vectors), respectively.

[0046] Table 1 Document structure relationship table

[0047] Table 2 Relationship between organization and circulation

[0048] Table 3 Time correlation

[0049] Table 4 Contact information class relationships

[0050] Table 5 Urgency class relationships

[0051] In this embodiment, as described in step S300 above, a knowledge graph is constructed based on the aforementioned structured entity set and semantic relationship set, and the entity nodes and semantic relationships are stored in a graph database. At the same time, the embedding vectors of the entity texts are stored in the vector database to support subsequent semantic retrieval and question-answer generation. The goal of this step is to store the extracted key information and the relationships between entities in a structured manner in the form of a knowledge graph. Figure 5 Issued a notice on environmental protection for a municipal government department and designated relevant organizing units and contacts.

[0052] First, each entity in the structured entity set extracted in step S200 is stored as a node in a graph database. Graph databases use a node-edge structure to store entities and the relationships between them. The graph structure makes information access more intuitive and efficient. For example, in government documents, entities such as "issuing document number," "issuing unit," and "text content" are stored as independent nodes in the graph database, while the relationships between them (such as "published" and "included") are represented by directed edges. Each node contains basic information about the entity, such as the entity type (such as "issuing document number"), the original text content of the entity, etc.

[0053] Next, a text embedding vector for each entity is generated through a deep learning model and stored in a vector database. The text embedding vector is generated by mapping text information into a high-dimensional vector space, thereby preserving its semantic information and contextual connections. In this way, the vector database can efficiently perform semantic retrieval, so that in subsequent query processes, relevant entities and official document information can be found according to the query intent. For example, when a user queries for "urgent documents issued by a certain unit", the official document entities related to the query are quickly matched by comparing the similarity between the embedding vector of the query text and the entity embedding vector stored in the vector database.

[0054] Furthermore, building a knowledge graph also involves storing semantic relationships. The inter-entity relationships generated in step S200 are stored as directed edges in the graph database, with each edge label representing a specific relationship type. For example, <issuing unit, published, official document> indicates that a unit has issued an official document, <official document, contains, body> indicates that a document contains body content, and <official document, issued on, issue time> indicates the issue time of a document. By connecting these directed edges, the graph database can clearly display the various semantic relationships between entities, further enhancing data connectivity and visualization.

[0055] It is worth noting that as government documents are constantly updated and added, the method of the present invention incorporates an incremental update algorithm to ensure the continuous updating of the knowledge graph and data consistency. When a new government document is processed, it is checked whether the new entity already exists in the graph database. If the entity already exists, the relevant relationships of the new document are directly connected to the existing entity without repeatedly creating the entity node. This incremental update method can effectively avoid the storage of redundant data and ensure the scalability and efficiency of the knowledge graph.

[0056] Preferably, the incremental update algorithm includes: Step S310: Generate an entity unique identifier by performing a joint hash calculation on the entity name and the attribute value; Step S320: traverse the graph database nodes based on the unique identifier and perform entity existence verification; Step S330: When it is detected that the entity in the newly added document has the same unique identifier as an existing node, a relationship edge is established between the node and the new document node; Step S340: When a new entity is detected, a new node is created and a complete relationship topology between the new node and the associated document node is established; Step S350: Synchronously update the embedding vector of the newly added entity in the vector database and rebuild the semantic similarity index.

[0057] In this embodiment, as described in step S400 above, after receiving the user query request, a hybrid retrieval operation is first performed, that is, the advantages of the graph database and the vector database are combined to achieve efficient and accurate retrieval of relevant information in the knowledge graph. Traditional retrieval methods have certain limitations. Especially in cross-document queries, the retrieval method based on vector embedding may cause the query results to deviate from the semantic core, while the keyword-based full-text indexing method may be affected by the expression method and has weak support. In order to optimize these limitations, this method proposes a graph index enhanced retrieval (G-RAG) method combined with LightRAG, which aims to improve retrieval accuracy and enhance the ability of cross-document queries.

[0058] First, a relational query is performed using a graph database based on the entity keywords in the user's query. Entities in a graph database are represented by nodes and edges, with nodes representing entities and edges representing relationships between them. After receiving a user query, the starting node in the graph database is located based on the entity information in the query, such as "issuing unit" and "issuance time."

[0059] For example, if a user queries "urgent documents issued by a certain unit," the query will first identify the entities "unit" and "urgent documents," and then search the graph database for the relationship between them. By traversing the edges in the graph, all document nodes with the "issued" relationship are extracted, and those with an "urgent" urgency are selected. This process, leveraging the graph database's node-edge structure, enables fast and efficient relationship queries, locating the most relevant documents.

[0060] In parallel with relational queries, semantic retrieval using the vector database is also performed. Unlike traditional keyword-based retrieval, the vector database stores text embedding vectors of entities, enabling semantic-based similarity matching in high-dimensional vector space.

[0061] For example, when searching for "urgent official documents issued by a certain unit," the user's query text is first converted into an embedding vector and compared with the document entity vectors stored in the vector database. If the query "urgent official document" has semantic similarities with a document entity stored in the database, that document entity is returned as a search result. This semantic matching method can identify all documents relevant to the user's query intent, regardless of whether they contain the exact same vocabulary.

[0062] Based on relational queries, not only can we extract subgraph structures relevant to the query from the graph database, but we can also enhance the relevance of search results through semantic matching. Specifically, we conduct a comprehensive analysis of the subgraph structure obtained from the graph database (including entities and their relationships) and the semantic vectors matched in the vector database. This comprehensive analysis helps the system better understand the query context, accurately extract relevant entity nodes and their relationships, and ensure that search results closely match the query intent.

[0063] For example, when a user queries "urgent documents issued by the XX Municipal Environmental Protection Bureau," the graph database first extracts all document nodes issued by the "XX Municipal Environmental Protection Bureau," filters out expedited documents through edge relationships, and performs version filtering on the documents in the subgraph based on the "issue timestamp" attribute of the document nodes. For example, if there are multiple documents with the same name, the latest version is retained first to avoid interference from older data. Simultaneously, the vector database finds semantically similar entities to "urgent documents" by matching the embedding vector of the query text. Ultimately, these two parts of the results are combined to form a more comprehensive and accurate search result set.

[0064] In this embodiment, as described in step S500 above, the results of the hybrid search are input into the large language model (LLM), and combined with the dynamic prompt template to generate a question-answer response text that conforms to the preset format. Figure 6 This step leverages LMM's powerful text generation capabilities to generate precise, structured answers based on the user's query and its associated data. Dynamic prompt templates guide and optimize this process, ensuring the generated answers accurately and clearly present the required information.

[0065] The hybrid search results obtained in step S400 are passed as input to the large language model. At this stage, the hybrid search results include entities extracted from the graph database and the relationships between them (such as issuing units, issuance time, etc.), as well as relevant documents and information obtained through semantic matching of the vector database. By inputting multimodal data, the large language model can obtain more comprehensive background information.

[0066] For example, if a user queries "urgent documents issued by the XX Municipal Environmental Protection Bureau," a hybrid search retrieves information such as the document's title, issuing unit, and degree of urgency. The large language model uses this information as contextual input and generates a complete and clear answer based on it.

[0067] To further improve the accuracy and pertinence of question-answer generation, this method applies dynamic prompt templates. Dynamic prompt templates are a series of optimization rules designed for LMMs. They use carefully constructed prompts to guide the model to generate answers according to a preset format. These templates are based on different task requirements, such as knowledge extraction, relational reasoning, and cross-document querying. They can dynamically adjust the generation process to ensure that the generated content meets user expectations.

[0068] Specifically, the system automatically selects an appropriate prompt template based on the user's query. For example, when processing information about official document releases, selecting a template related to "Official Document Information Generation" prompts the LMM to return relevant information in a predefined format. During this process, the template includes guidance on key information the LMM should focus on (such as the issuing unit, issuance time, and urgency), and standardizes the generated content structure, such as whether to list relevant attachments or copy recipients.

[0069] Furthermore, dynamic prompt templates can handle conflicting versions of knowledge. For example, if multiple copies of official documents are retrieved, the template prioritizes the latest version based on the timestamp mechanism, and determines the most relevant document based on semantic matching. This effectively avoids confusion caused by multiple versions of documents and ensures accurate and timely answers.

[0070] Based on the guidance of dynamic prompt templates, the large language model will generate question-and-answer response text in a preset format based on the input mixed retrieval results, and present it to the user in a concise and easy-to-understand manner.

[0071] To verify the effectiveness of the hybrid retrieval question-answering system based on the BERT-CRF model and knowledge graph enhancement proposed in this invention, this embodiment designed and implemented a complete comparative experiment. The specific implementation process is as follows: This paper uses a document processing dataset released by Yancheng Big Data Group Co., Ltd. in December 2024. This dataset, based on cutting-edge deep learning technology and combined with the document writing standards and semantic knowledge in the "Measures for the Processing of Official Documents by National Administrative Organs," is trained through a deep learning model. It can automatically identify and extract textual information from images or scans of official documents, such as the document number, confidentiality level and confidentiality period, urgency, issuing agency logo, document number, signatory, title, primary addressee, main text, attachment description, and the issuing agency's signature.

[0072] In this study, we compared the performance of the BiLSTM-CRF and BERT-CRF models in detail. During training, we used the cross-entropy loss function to measure the model error. The loss function is defined as follows:

[0073] in, is the true label of the i-th sample, is the model's predicted value for that sample, and N is the total number of samples. The loss function reflects the difference between the model's predicted value and the true label. A smaller value indicates a more accurate model prediction. By calculating and summing the loss for all samples, we can comprehensively evaluate the model's performance.

[0074] In terms of performance evaluation, CRF accuracy and F1 score are used as the main indicators. The F1 score formula is as follows:

[0075] Among them, the definitions of precision and recall are:

[0076]

[0077] True Positives represents the number of samples correctly predicted as positive; False Positives represents the number of samples incorrectly predicted as positive; and False Negatives represents the number of positive samples incorrectly predicted as negative. The F1 score is the harmonic mean of precision and recall, providing a comprehensive assessment of a model's performance across different dimensions.

[0078] Loss function comparison: Figure 7 The figure shows the loss functions and performance comparisons of the BiLSTM-CRF and BERT-CRF models during training. The loss comparison in the left figure shows that the training loss of the BERT-CRF model decreases rapidly as training progresses and stabilizes after the fifth epoch, while the training and validation losses of the BiLSTM-CRF model remain consistently low. The performance comparison in the right figure shows that the validation accuracy of the BiLSTM-CRF model shows an upward trend, demonstrating good training results. In contrast, the validation F1 value of BERT-CRF gradually increases with training progress, ultimately reaching around 0.95, indicating that the BERT-CRF model exhibits superior performance in the later stages of training. Overall, BERT-CRF exhibits faster convergence and better performance during training.

[0079] Performance index comparison: Figure 8 The performance comparison of the two models on the validation set is shown. BiLSTM-CRF achieved validation accuracy, recall, and F1 scores of 83.96%, 82.25%, and 83.10%, respectively, which is middling, but exhibits performance fluctuations due to class imbalance. In contrast, BERT-CRF achieved an F1 score of 96.09%, precision of 96.17%, and recall of 96.03%, significantly outperforming BiLSTM-CRF and demonstrating BERT-CRF's superior performance on complex tasks.

[0080] Training time comparison: Figure 9A comparison of the training times of the two models is shown. BiLSTM-CRF takes only 360.99 seconds to train, while BERT-CRF requires 9363.59 seconds, approximately 26 times longer. This difference stems from the BERT model's large number of parameters and complex computational structure, which results in longer training time and hardware resource requirements. Despite BERT-CRF's longer training time, its significant improvements in precision, recall, and F1 score (F1 score increased by approximately 13%) demonstrate that BERT-CRF is more suitable for government document processing scenarios requiring higher accuracy.

[0081] While the BERT-CRF model requires significantly more training time and computational resources than BiLSTM-CRF, it significantly outperforms key performance indicators such as precision, recall, and F1 score when handling complex tasks. BERT-CRF, with its powerful contextual modeling capabilities, demonstrates exceptional robustness, particularly in scenarios with long-range semantic dependencies and few samples, demonstrating its potential for application in high-precision government tasks.

[0082] According to another aspect of an embodiment of the present application, an electronic device is provided, including a processor and a memory, wherein the processor is configured to implement the steps of the method when executing a computer program stored in the memory.

[0083] In the above embodiments of the present invention, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0084] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only exemplary. For example, the division of the units can be a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0085] In addition, the functional units in the various embodiments of the present invention may be integrated into a single processing unit, each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0086] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk, etc. Various media that can store program codes.

[0087] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as within the scope of protection of the present invention.

Claims

1. A government document information extraction and question-answering method, characterized in that: The steps include: The BERT-CRF joint model is used to extract entities from the input government documents, obtaining a structured entity set including the document number, issuing unit, and text content; Performing relationship extraction on the structured entity set to generate a set of semantic relationships between entities; Constructing a knowledge graph based on the structured entity set and the semantic relationship set, storing the entity nodes in a graph database, and storing the embedding vectors of the entity texts in a vector database; When receiving a user query request, a hybrid search operation is performed: a relational query in the graph database and a semantic search in the vector database are performed to extract the subgraph structure and semantic matching vectors related to the query, and obtain the hybrid search results; The mixed retrieval results are input into a large language model, and a dynamic prompt template is applied to generate a question-answer response text that conforms to a preset format.

2. The government document information extraction and question-answering method according to claim 1, characterized in that: Methods for extracting entities from input government documents include: Preprocess government documents to generate text sequences that meet the BERT encoder input specifications; The pre-processed government documents are fed into the BERT-CRF model, and character-level contextual features are extracted through BERT's multi-head self-attention mechanism. Applying state transition constraint rules in the CRF layer, decoding generates the optimal label sequence that conforms to the document entity boundary; According to the continuous entity fragments in the optimal label sequence, corresponding fields are intercepted from the original text of the government document to generate a structured entity set including the document number, the issuing unit and the text content.

3. The government document information extraction and question-answering method according to claim 1, characterized in that: The method for performing relationship extraction on the structured entity set includes: For entity pairs in the structured entity set, extracting their context texts to form sentence pair inputs, wherein the sentence pair inputs include entities and their preceding and following semantic contexts; Calling the relationship classification module of the Deepseek API framework, performing relationship reasoning on the sentence pair input based on the Transformer architecture, and outputting classification results of the official document release relationship and the copy-to relationship; The BERT-CRF model is used in parallel to perform sequence annotation on the text between entities. The BIO annotation system is used to identify continuous text segments containing temporal relationships to obtain sequence annotation results. The classification results are merged with the sequence labeling results, and standardized mapping is performed according to the preset official document relationship type table to generate a semantic relationship set of triple structure, among which the time association relationship is generated by binding the time segment in the sequence labeling result with the official document entity.

4. The government document information extraction and question-answering method according to claim 1, wherein: The construction methods of knowledge graph include: The entities in the structured entity set are stored in the graph database as nodes, and the node attributes include the entity type and the original text fragment; The relationships in the semantic relationship set are stored in the graph database as labeled edges. The edge labels correspond to predefined document relationship types, and the embedding vectors of the entity text are simultaneously stored in the vector database. Execute the incremental update algorithm to perform entity deduplication verification on the newly added document data: if the entity already exists in the graph database, directly establish a relationship edge with the new document node; if the entity does not exist, create a new node and establish a complete relationship topology; A knowledge graph is constructed based on the node-edge-node triple structure to ensure that the same entity nodes remain unique across document scenarios, and the semantic similarity index between entities is maintained through a vector database.

5. The government document information extraction and question-answering method according to claim 4, characterized in that: The incremental update algorithm includes: Generate an entity unique identifier through the joint hash calculation of the entity name and attribute value; Traversing the graph database nodes based on the unique identifier and performing entity existence verification; When it is detected that an entity in a newly added document has the same unique identifier as an existing node, a relationship edge is established between the node and the new document node; When a new entity is detected, a new node is created and a complete relationship topology is established between it and the associated document nodes; The embedding vectors of the newly added entities are synchronously updated in the vector database, and the semantic similarity index is rebuilt.

6. The government document information extraction and question-answering method according to claim 1, characterized in that: Methods for performing hybrid retrieval operations include: Locate the starting node based on the entity keywords in the user query, traverse along the predefined relationship edge types in the knowledge graph, and extract the subgraph structure containing the document nodes and their associated relationships; Convert the user query text into an embedding vector, perform semantic matching with the stored entity vectors, and filter out the entity set related to the query intent; Graph embedding technology is used to jointly analyze subgraph structures and semantic matching entities to generate cross-document retrieval results. The search results are filtered for timeliness based on the timestamp of the official document issuance, and the latest version of the official document information is retained first to form the final mixed search results.

7. The government document information extraction and question-answering method according to claim 1, characterized in that: The construction method of dynamic prompt template includes: Establish a preset template library including entity and relationship extraction templates, retrieval enhancement generation templates, keyword extraction templates and hybrid information retrieval templates; Set separators in entity and relationship extraction templates to standardize the output format of entity and relationship triples; A timestamp mechanism is embedded in the retrieval enhancement generation template. When multiple versions of knowledge conflicts are detected, the answer priority is sorted based on semantic matching and timeliness. Configure a JSON output framework for the keyword extraction template to automatically extract high-level concepts and specific details from the query; A cross-modal comparison mechanism is set up in the hybrid information retrieval template to compare the credibility of the knowledge graph query results with the vector database search results, and the reference source is marked in the response text; The format of the final generated question and answer response text is standardized through the Markdown structured output constraint mechanism.

8. The government document information extraction and question-answering method according to claim 1, wherein: The method further comprises: In the entity extraction stage, the layout characteristics of government documents are analyzed to determine the layout type, which includes paragraph structure distribution, title hierarchy and signature position characteristics; Generate a corresponding layout identifier according to the layout type, and insert the layout identifier into the starting position of the input sequence; In the input sequence of the BERT-CRF model, the text characters are concatenated with the layout position codes to form an enhanced context representation vector, where the layout position codes are generated by a two-dimensional coordinate mapping algorithm and reflect the physical position of the text in the original document; In the relationship extraction stage, relationship reasoning of entity pairs is performed based on the enhanced context representation vector, wherein the layout identifier participates in the attention weight calculation of the Transformer architecture, guiding the model to focus on layout-related context features.

9. An electronic device comprising a memory and a processor, characterized in that: The memory is used to store a program that supports the processor to execute the government document information extraction and question-and-answer method described in any one of claims 1-8, and the processor is configured to execute the program stored in the memory.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the government document information extraction and question-answering method described in any one of claims 1 to 8 are executed.

Images