Literature element extraction method and system, electronic equipment and storage medium
By constructing an evidence graph and combining a hybrid expert model with a document element ontology, the problems of knowledge lag and model illusion in academic literature are solved, and document element extraction with high accuracy and reliability is achieved.
Patent Information
- Application Number
- CN202511712872.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-11-20
- Publication Date
- 2026-02-17
AI Technical Summary
Existing technologies suffer from problems such as lagging knowledge updates, lack of suppression of model illusions, and superficial cross-modal semantic understanding when processing complex academic literature, resulting in incomplete, unreliable, and insufficient depth of understanding in the extraction results.
A pre-trained multimodal model is used for analysis to construct an evidence graph and determine target prompts by combining it with the document element ontology. A hybrid expert model is used to extract document elements, and the accuracy of the results is verified by the evidence graph. The document element ontology is dynamically updated to adapt to the evolution of academic concepts.
It achieves precise structural capture of academic literature, significantly improves the credibility and accuracy of extraction results, suppresses model illusion, and ensures the traceability and interpretability of verification.
Smart Images
Figure CN121542406A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of document processing technology, and specifically relates to a method, system, electronic device and storage medium for extracting document elements. Background Technology
[0002] With the rapid development of scientific research, the number of academic documents is growing exponentially. Traditional manual reading and information summarization methods can no longer meet the needs of researchers to quickly acquire and organize knowledge. Therefore, how to automatically and accurately extract structured core research elements (such as research background, research methods, experimental conclusions, etc.) from complex academic documents containing multimodal data such as text, charts, and formulas has become a core challenge in the field of knowledge management.
[0003] In related technologies, there are schemes that utilize deep learning models, especially vision-language pre-trained models (such as the LayoutLM series models), for document information extraction. These schemes typically input document images and text sequences into the model together, and by jointly learning page layout information and text content, they can identify and extract entities such as titles, authors, paragraphs, and their basic relationships in the document.
[0004] However, the aforementioned existing technical solutions still have the following significant drawbacks when applied to complex academic literature scenarios:
[0005] First, knowledge updates lag behind. Existing solutions mostly rely on static, predefined, general-domain knowledge ontology, which cannot specifically model the internal logical structure of academic research, nor can they incorporate new terms and relationships in a timely manner as academic concepts evolve rapidly. This results in incomplete or outdated extraction results when dealing with emerging or interdisciplinary literature.
[0006] Secondly, there is a lack of effective suppression of model illusions. When directly applying large language models for feature extraction, the model may generate content that seems reasonable but lacks supporting evidence from the original text, i.e., the "illusion" problem, which seriously affects the reliability and practical value of the extraction results.
[0007] Third, cross-modal semantic understanding is superficial. While existing solutions can handle page layout, they lack effective modeling and reasoning capabilities for the deep semantic relationships (such as confirmation, rebuttal, supplementary explanation, etc.) between chart content and text description, thus limiting the depth of understanding of the documents. Summary of the Invention
[0008] To address the aforementioned problems in existing technologies—namely, the issues of knowledge lag, difficulty in suppressing model illusions, and superficial cross-modal semantic understanding when applied to extracting document elements in complex academic literature scenarios—this invention provides a document element extraction method, comprising:
[0009] Obtain the target document, which consists of several document elements;
[0010] The target document is parsed using a pre-trained multimodal model to identify document elements within the target document;
[0011] An evidence graph is constructed based on the identified document elements, and target prompt words are determined by combining the evidence graph with the preset document element ontology. The evidence graph is used to indicate the index relationship between document elements in the target document, and the document element ontology is used to indicate the element type, element attributes and logical relationship between elements in the document element extraction task.
[0012] Based on the target prompt words, a pre-trained feature extraction model is used to perform a document feature extraction task to extract target features from the target document. The feature extraction model adopts a hybrid expert model architecture that includes multiple expert sub-models, and the multiple expert sub-models are used to perform different types of document feature extraction tasks.
[0013] The target element is verified using the evidence map, and the target element that meets the verification is output.
[0014] In some preferred embodiments, the evidence graph includes multiple nodes and edges, and the construction of the evidence graph based on identified document elements includes:
[0015] Based on the parsing results of the target document, determine the spatial coordinates and content of each document element in the target document;
[0016] The identified document elements are used as nodes in the evidence graph;
[0017] Based on the spatial coordinates and content of each element, and combined with preset spatial and grammatical rules, the edges between each node in the evidence graph are defined, and multiple nodes are used to represent the association between corresponding elements.
[0018] In some preferred embodiments, the nodes and edges in the evidence graph are accompanied by confidence scores output by the multimodal model.
[0019] In some preferred embodiments, determining the target prompt words by combining the evidence map and a preset document element ontology includes:
[0020] Based on the evidence map, determine the document type and topic of the target document;
[0021] Traverse the preset historical case library and determine the prompt template that best matches the document type and topic of the target document from the historical case library;
[0022] The prompt template is integrated with the document element ontology to generate the target prompt word.
[0023] In some preferred embodiments, verifying the target element using the evidence map includes:
[0024] Based on the target element, a subgraph matching query is performed in the evidence graph to determine whether there is a node or edge in the evidence graph that matches the target element.
[0025] When a node or edge matching the target element is found, the target element is determined to satisfy the verification.
[0026] If no node or edge matching the target element is found, the target element is determined not to meet the verification.
[0027] Based on the verification results, feedback optimization is performed on the document element ontology or the dynamic prompts.
[0028] In some preferred embodiments, the feature extraction model includes a gating network, through which the feature extraction model adaptively allocates document feature extraction tasks to corresponding expert sub-models for execution.
[0029] In some preferred embodiments, the target prompts are adaptively updated as the document element ontology evolves, and the evolution mechanism of the document element ontology includes:
[0030] Obtain the document element extraction results of newly added documents according to the preset cycle;
[0031] Based on the document element extraction results, determine the candidate new elements or candidate new relationships that are not yet included in the current document element ontology;
[0032] Determine the overall confidence level of the candidate new elements or the candidate new relationships;
[0033] When the overall confidence level exceeds a preset threshold, the candidate new element or the candidate new relationship is updated in the document element ontology.
[0034] In a second aspect, the present invention provides a document element extraction system for performing the document element extraction method as described in the first aspect, comprising:
[0035] The data acquisition module is used to acquire target documents, which consist of several document elements;
[0036] The document parsing module is used to parse the target document using a pre-trained multimodal model in order to identify the document elements in the target document;
[0037] The prompt word construction module is used to construct an evidence map based on the identified document elements, and to determine the target prompt words by combining the evidence map with the preset document element ontology. The evidence map is used to indicate the index relationship between each document element in the target document, and the document element ontology is used to indicate the element type, element attributes and logical relationship between elements in the document element extraction task.
[0038] The element extraction module is used to perform a document element extraction task based on the target prompt words and a pre-trained element extraction model, and to extract target elements from the target document.
[0039] The element verification module is used to verify the target element using the evidence map and output the target element that meets the verification requirements.
[0040] In a third aspect, the present invention provides an electronic device comprising:
[0041] At least one processor;
[0042] and a memory communicatively connected to at least one of the processors;
[0043] The memory stores instructions that can be executed by the processor to implement the document element extraction method as described in the first aspect.
[0044] In a fourth aspect, the present invention provides a computer-readable storage medium, characterized in that the computer-readable storage medium stores computer instructions for execution by the computer to implement the document element extraction method as described in the first aspect.
[0045] The beneficial effects of this invention are:
[0046] Compared to existing technologies, this invention accurately captures the internal structure of academic documents by pre-setting a dedicated document element ontology. At the same time, through the dual mechanism of target prompt word constraints and evidence graph verification, it can effectively identify and suppress the illusory output of large models, making the extraction results verifiable and significantly improving the credibility of the results.
[0047] On the one hand, the dynamic prompts in this invention provide the model with clear task boundaries and terminology specifications, and combined with the specialized capabilities of the hybrid expert model, significantly improve the accuracy of extracting complex research elements.
[0048] On the other hand, the construction of the evidence map in this invention focuses on quickly establishing index relationships, and the verification stage further utilizes the map for sub-graph matching, which not only ensures the verification speed, but also makes the verification process traceable and interpretable. Attached Figure Description
[0049] Other features, objects, and advantages of this application will become more apparent from the following detailed description of non-limiting embodiments with reference to the accompanying drawings:
[0050] Figure 1 This is a flowchart illustrating a document element extraction method provided in an embodiment of the present invention;
[0051] Figure 2 This is a schematic diagram of the framework of a document element extraction system provided in an embodiment of the present invention;
[0052] Figure 3 This is a schematic diagram of the structure of a computer system that implements the methods, systems, and electronic devices of this application. Detailed Implementation
[0053] The present application will now be described in further detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the invention. Furthermore, it should be noted that, for ease of description, only the parts relevant to the invention are shown in the accompanying drawings.
[0054] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.
[0055] To more clearly explain the document element extraction method of the present invention, the following is in conjunction with... Figure 1 The steps in the embodiments of the present invention will be described in detail below.
[0056] The first embodiment of the present invention provides a method for extracting document elements, including steps S10-S40, each step of which is described in detail below:
[0057] Step S10: Obtain the target document, which consists of several document elements;
[0058] Step S20: Use a pre-trained multimodal model to parse the target document in order to identify document elements in the target document;
[0059] Step S30: Construct an evidence map based on the identified document elements, and determine target prompt words by combining the evidence map and the preset document element ontology. The evidence map is used to indicate the index relationship between document elements in the target document, and the document element ontology is used to indicate the element type, element attributes and logical relationship between elements in the document element extraction task.
[0060] Step S40: Based on the target prompt words, a pre-trained feature extraction model is used to perform a document feature extraction task to extract target features from the target document. The feature extraction model adopts a hybrid expert model architecture that includes multiple expert sub-models, and the multiple expert sub-models are used to perform different types of document feature extraction tasks.
[0061] Step S50: Verify the target element using the evidence map and output the target element that meets the verification requirements.
[0062] In this embodiment, the system first receives a document uploaded by the user as the target document. This document can be adapted to various document formats. For example, the user uploads a PDF academic paper on object detection as the target document. This academic paper consists of multiple document elements, including but not limited to: title, author information, abstract, main text paragraphs, chapter titles (such as Introduction, Methodology, etc.), figures and tables, figure and table titles, formulas, and references.
[0063] Upon receiving the target document, this embodiment first parses the unstructured document into structured data. Specifically, this embodiment uses a pre-trained multimodal large model (e.g., a model based on LayoutLMv3 or Donut) for parsing, which can simultaneously understand text content and page layout information.
[0064] Specifically, this multimodal model parses the input PDF paper page by page, identifying each individual literal element on the page. For example, for each literal element, the model output includes:
[0065] Content: This refers to the text information (OCR recognition result) or image data of the document element;
[0066] Location coordinates: These are the coordinates of the document element within the bounding box on the page.
[0067] Type: This refers to the logical category of the element, such as paragraph, title, figure, table, caption, etc.
[0068] Confidence score: The confidence level of the model in the above recognition results.
[0069] Through this process, the original PDF document is parsed into a collection containing multiple structured document elements.
[0070] Based on this, this embodiment further constructs an evidence map and determines target prompts by combining the ontology of document elements.
[0071] This embodiment constructs an evidence map representing the internal structure and relationships of the target document based on the analysis results, specifically including:
[0072] Based on the parsing results of the target document, determine the spatial coordinates and content of each document element in the target document;
[0073] The identified document elements are used as nodes in the evidence graph;
[0074] Based on the spatial coordinates and content of each element, and combined with preset spatial and grammatical rules, the edges between each node in the evidence graph are defined, and multiple nodes are used to represent the association between corresponding elements.
[0075] Specifically, for node creation: each identified document element (such as a paragraph or a figure) is added to the graph as a node. Each node stores all the information of the element, including its content, location coordinates, type, and confidence score.
[0076] For edge creation: Based on preset spatial and grammatical rules, connections are established between nodes, i.e., edges are created.
[0077] For example, regarding spatial rules: if the bounding box of a node of type caption is immediately below the bounding box of a node of type figure, then an edge of type has_caption is created between the two nodes.
[0078] For the grammar rule, if the text "as shown in Fig. 3" is detected in the content of a paragraph node, and there is a caption node in the graph with the content "Fig. 3: ...", then an edge of type "refers_to" is established between the two nodes.
[0079] The evidence map generated in this embodiment effectively represents the physical proximity and logical reference relationships of various parts (text, charts, etc.) in the target document.
[0080] More specifically, the nodes and edges in the evidence graph are each accompanied by a confidence score output by the multimodal model.
[0081] Furthermore, as a feasible implementation method, in order to further improve the flexibility and accuracy of evidence map construction, especially when dealing with documents with complex layouts or unclear relationships, the above-mentioned sub-steps for constructing evidence maps can be replaced by a neural network learning-based approach.
[0082] Specifically, this implementation can construct a weighted evidence graph based on a graph attention network. Before feeding the document elements into the neural network, the attributes of each document element (i.e., a node in the graph) are converted into a machine-understandable numerical vector.
[0083] For example, for each identified document element, the following features are extracted and encoded:
[0084] Content features: Input the text content of an element into a pre-trained text encoder (such as Sentence-BERT) to generate a fixed-dimensional vector that can represent its semantic information.
[0085] Positional features: The normalized bounding box coordinates (x_min, y_min, x_max, y_max, width, height) of the element are treated as a numerical vector. The coordinates are usually normalized relative to the page size so that their values fall within the range [0, 1].
[0086] Type characteristics: The logical type of the element (such as paragraph, figure, caption) is one-hot encoded and converted into a binary vector.
[0087] Finally, the three features mentioned above are combined to form a comprehensive feature vector for each node.
[0088] Meanwhile, in this embodiment, a lightweight graph attention network is trained to calculate the strength of the relationship between any two nodes.
[0089] For example, in the model training process, a training set containing a large number of labeled documents is prepared. The labeled data can be obtained in two ways: one is manual annotation, which precisely defines the correct relationships between different elements (such as "citation relationship" and "subordination relationship"); the other is to automatically generate a batch of high-quality, high-confidence "pseudo-labels" as the initial training data through the aforementioned "spatial rules and grammatical rules".
[0090] The model's task is to perform link prediction. For any pair of input nodes, the model needs to predict the probability that a relationship exists between them. During training, the model learns to minimize the cross-entropy loss between its predictions and the true labels.
[0091] This implementation method does not rely on fixed spatial and grammatical rules, but learns general patterns of document layout from historical data, and has better adaptability to previously unseen typesetting formats.
[0092] After constructing the evidence map, this embodiment combines a pre-defined, evolvable ontology of document elements and generates target prompts to guide the next extraction step through search enhancement generation techniques. Specifically, these include:
[0093] Based on the evidence map, determine the document type and topic of the target document;
[0094] Traverse the preset historical case library and determine the prompt template that best matches the document type and topic of the target document from the historical case library;
[0095] The prompt template is integrated with the document element ontology to generate the target prompt word.
[0096] In this embodiment, the document element ontology predefines the key elements and their hierarchical relationships in academic research. For example, its top-level elements may include research background, technical methods, and experimental results; technical methods may further include sub-elements such as model architecture, loss function, and optimizer.
[0097] As an example, the process of generating target prompts includes:
[0098] Analyze the node content of the evidence map (such as title and abstract), such as determining that the subject of the target document is target detection and the type is academic paper;
[0099] Search a historical case library (which stores a large number of high-quality prompt word templates) for the prompt template that best matches "object detection academic paper". The template may be a framework instruction, such as: Please extract [technical method] and [experimental results] from the literature.
[0100] The retrieved templates are then integrated with the document element ontology. For example, the placeholder for "[technical method]" in the template is replaced with a specific sub-element defined in the ontology, generating more refined instructions: Please extract the technical method, specifically including its model architecture, loss function, and training dataset;
[0101] Ultimately, the target prompts generated in this embodiment not only include task instructions but also specify the types and structures of elements to be extracted, laying the foundation for the subsequent element extraction process.
[0102] In addition, as a feasible implementation method, a dedicated small language model can be trained to directly generate dynamic prompt words based on document features and ontology state.
[0103] Furthermore, in this embodiment, the feature extraction model adopts a Mixture-of-Experts (MoE) architecture.
[0104] In one preferred embodiment, the feature extraction model includes a gating network, through which the feature extraction model adaptively allocates document feature extraction tasks to corresponding expert sub-models for execution.
[0105] Specifically, the architecture comprises a gating network and multiple expert sub-models. Each expert sub-model is specifically fine-tuned for a particular type of feature extraction task.
[0106] For example, Expert 1 is good at extracting details related to technical methods (such as model name and number of network layers); Expert 2 is good at analyzing and understanding experimental results (such as performance indicators such as mAP and FPS) from tables and text; and Expert 3 is good at summarizing research background and related work.
[0107] When the system receives a target prompt (such as "Please extract the model architecture"), the gating network analyzes the intent of the prompt and determines that the task is best handled by Expert 1. Therefore, the gating network routes the task and relevant content from the target document (which could be the text of the Methodology section located by the evidence graph) to Expert 1.
[0108] After receiving the task, Expert 1 processes the text within the specified scope and extracts the target elements, such as: {Model Architecture: YOLOv8-L}.
[0109] This embodiment utilizes a hybrid expert model and gating network structure to significantly improve the extraction accuracy of different types of elements.
[0110] Furthermore, this embodiment utilizes an evidence graph to verify the extracted target elements and outputs the target elements that satisfy the verification, specifically including:
[0111] Based on the target element, a subgraph matching query is performed in the evidence graph to determine whether there is a node or edge in the evidence graph that matches the target element.
[0112] When a node or edge matching the target element is found, the target element is determined to satisfy the verification.
[0113] If no node or edge matching the target element is found, the target element is determined not to meet the verification.
[0114] Based on the verification results, feedback optimization is performed on the document element ontology or the dynamic prompts.
[0115] For example, for each extracted target element, such as {model architecture: YOLOv8-L}, a subgraph matching query is performed once in the evidence graph constructed in the above steps. The logic of this query is: "In the evidence graph, does there exist a text node whose content contains 'YOLOv8-L', and does this node have a path association with a node whose chapter title is 'Methodology' or 'Model'?"
[0116] If the query is successful, that is, if direct or indirect evidence supporting the extraction result is found in the graph (for example, YOLOv8-L is indeed mentioned in a paragraph of the Methods section), then the target element is considered to meet the verification, has high credibility, and is accepted as the final output.
[0117] If the query fails, meaning no relevant evidence can be found in the graph, the target element may be an illusion product of the element extraction model. It should be marked as not meeting the validation and discarded or submitted for manual review instead of being output as the final result.
[0118] Through the above process, this embodiment realizes an adaptive, high-precision document element extraction method with self-verification capability.
[0119] In a preferred embodiment, the target prompts in this example are adaptively updated as the document element ontology evolves. The evolution mechanism of the document element ontology includes:
[0120] The document element extraction results of newly added documents are obtained according to a preset period; based on the document element extraction results, candidate new elements or candidate new relationships that are not yet included in the current document element ontology are determined; the comprehensive confidence level of the candidate new elements or the candidate new relationships is determined; when the comprehensive confidence level exceeds a preset threshold, the candidate new elements or the candidate new relationships are updated to the document element ontology.
[0121] In this embodiment, the evolution process of the document element ontology is initiated according to a preset cycle (preferably, once a week for every 1,000 new documents processed).
[0122] It first constructs a fact base for discovering new knowledge by collecting the document element extraction results of all newly added documents within a preset period, which have been processed and verified by the first embodiment.
[0123] Through in-depth analysis of the aforementioned fact base, new concepts or connections that are not yet defined in the current ontology of document elements but may have significant value are discovered.
[0124] For example, suppose that in the current literature element ontology, there are only two sub-elements under the technical methods: model architecture and loss function. However, when analyzing a recent batch of papers on large language models, it was found that in the technical methods section of a large number of papers, in addition to extracting the defined model architecture, an undefined but highly co-occurring term, such as quantization technique, frequently appears.
[0125] Based on this, we further identified that "quantization techniques" repeatedly appeared in the context of describing model optimization, and therefore marked it as a candidate new element belonging to the technical approach.
[0126] For example, suppose we are processing a batch of papers on new drug development and extracting a large number of drug molecule elements and protein target elements. By analyzing the sentences connecting these two types of elements, we further discover a frequently occurring sentence pattern: "[Drug molecule A] demonstrates potent inhibition of [protein target B]". However, the current literature element ontology may only define the generalized relationship of "associated with". In this case, by normalizing the verb phrase "demonstrates potent inhibition of", we extract a new relationship type—"inhibits"—and mark it as a candidate new relationship connecting "drug molecule" and "protein target".
[0127] Meanwhile, to prevent accidental terms or incorrect associations from being introduced into the ontology, this implementation calculates a comprehensive confidence level for each candidate (new element or new relationship).
[0128] For example, this confidence level takes into account the following dimensions:
[0129] Frequency of occurrence: the number of times the candidate appears in new literature within the period; breadth of distribution: whether the candidate appears in literature from multiple different institutions and with multiple different authors to exclude internal terms of a specific research group; contextual consistency: whether the context in which the candidate appears is highly consistent.
[0130] Based on this, the scores of the above dimensions are integrated into a comprehensive confidence score within the range of [0, 1] using a weighted formula.
[0131] Finally, in this embodiment, the calculated overall confidence level is compared with a preset threshold (preferably, such as 0.85). When the overall confidence level exceeds the threshold, the candidate is determined to be a mature, stable, and important new knowledge.
[0132] For candidate new elements (such as "quantitative techniques"), add them as new child nodes under the corresponding parent node (such as "technical methods") of the document element ontology.
[0133] For candidate new relations (such as "suppression"), define the edge type in the schema layer of the document element ontology and specify the type of the nodes it connects.
[0134] In addition, when the overall confidence level is below the threshold, the candidate is temporarily stored in an observation list and its statistical data is tracked in subsequent evolutionary cycles.
[0135] Once the document element ontology is updated, these new elements and relationships will be taken into account when generating target prompts in the next step. For example, when processing a paper on large language models again, the generated target prompts will automatically evolve to: "...Please extract the 'technical methods,' specifically including its 'model architecture,' 'loss function,' and 'quantization techniques'...".
[0136] This implementation method achieves self-iteration and growth of knowledge through this evolutionary mechanism, ensuring the adaptive updating of the ontology of document elements.
[0137] Further, please refer to Figure 2 The second embodiment of the present invention proposes a document subject classification system for performing the above-described document element extraction method, including:
[0138] Data acquisition module 210 is used to acquire target documents, which are composed of several document elements;
[0139] The document parsing module 220 is used to parse the target document using a pre-trained multimodal model in order to identify document elements in the target document;
[0140] The prompt word construction module 230 is used to construct an evidence map based on the identified document elements, and to determine the target prompt word by combining the evidence map and the preset document element ontology. The evidence map is used to indicate the index relationship between each document element in the target document, and the document element ontology is used to indicate the element type, element attributes and logical relationship between elements in the document element extraction task.
[0141] The element extraction module 240 is used to perform a document element extraction task based on the target prompt words and a pre-trained element extraction model to extract target elements from the target document.
[0142] The element verification module 250 is used to verify the target element using the evidence map and output the target element that meets the verification.
[0143] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working process and related descriptions of the system described above can be found in the corresponding processes in the foregoing method embodiments, and will not be repeated here.
[0144] It should be noted that the document element extraction method and system provided in the above embodiments are only illustrative examples of the division of the above functional modules. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the modules or steps in the embodiments of the present invention can be further decomposed or combined. For example, the modules in the above embodiments can be merged into one module, or further divided into multiple sub-modules to complete all or part of the functions described above. The names of the modules and steps involved in the embodiments of the present invention are only for distinguishing the various modules or steps and are not considered as an improper limitation of the present invention.
[0145] An electronic device according to a third embodiment of the present invention includes:
[0146] At least one processor;
[0147] and a memory communicatively connected to at least one of the processors;
[0148] The memory stores instructions that can be executed by the processor to implement the document element extraction method described above.
[0149] A fourth embodiment of the present invention provides a computer-readable storage medium storing computer instructions, which are executed by the computer to implement the above-described document element extraction method.
[0150] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working process and related descriptions of the storage device and processing device described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.
[0151] The following is for reference. Figure 3 It shows a schematic diagram of the structure of a computer system for implementing embodiments of the systems, methods, and electronic devices of this application. Figure 3 The server shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.
[0152] like Figure 3As shown, the computer system includes a Central Processing Unit (CPU) 301, which can perform various appropriate actions and processes based on programs stored in Read Only Memory (ROM) 302 or programs loaded from storage section 308 into Random Access Memory (RAM) 303. The RAM 303 also stores various programs and data required for system operation. The CPU 301, ROM 302, and RAM 303 are interconnected via a bus 304. An Input / Output (I / O) interface 305 is also connected to the bus 304.
[0153] The following components are connected to I / O interface 305: an input section 306 including a keyboard, mouse, etc.; an output section 307 including a cathode ray tube (CRT), liquid crystal display (LCD), and speakers, etc.; a storage section 308 including a hard disk, etc.; and a communication section 309 including a network interface card such as a LAN (Local Area Network) card and a modem, etc. The communication section 309 performs communication processing via a network such as the Internet. A drive 310 is also connected to I / O interface 305 as needed. Removable media 311, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., are installed on drive 310 as needed so that computer programs read from them can be installed into storage section 308 as needed.
[0154] Specifically, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication section 309, and / or installed from removable medium 311. When the computer program is executed by central processing unit (CPU) 301, it performs the functions defined in the methods of this application. It should be noted that the computer-readable medium described above in this application can be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this application, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in connection with an instruction execution system, apparatus, or device. In this application, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on a computer-readable medium can be transmitted using any suitable medium, including but not limited to: wireless, wire, optical fiber, RF, etc., or any suitable combination thereof.
[0155] Computer program code for performing the operations of this application can be written in one or more programming languages or a combination thereof, including object-oriented programming languages such as Java, Smalltalk, and C++, as well as conventional procedural programming languages such as "C" or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).
[0156] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.
[0157] The terms “first”, “second”, etc., are used to distinguish similar objects, not to describe or indicate a specific order or sequence.
[0158] The term "comprising" or any other similar term is intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus / device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent in such process, method, article, or apparatus / device.
[0159] The technical solution of the present invention has been described above with reference to the preferred embodiments shown in the accompanying drawings. However, it will be readily understood by those skilled in the art that the scope of protection of the present invention is obviously not limited to these specific embodiments. Without departing from the principles of the present invention, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will all fall within the scope of protection of the present invention.
Claims
1. A method for extracting document elements, characterized in that, include: Obtain the target document, which consists of several document elements; The target document is parsed using a pre-trained multimodal model to identify document elements within the target document; An evidence graph is constructed based on the identified document elements, and target prompt words are determined by combining the evidence graph with the preset document element ontology. The evidence graph is used to indicate the index relationship between document elements in the target document, and the document element ontology is used to indicate the element type, element attributes and logical relationship between elements in the document element extraction task. Based on the target prompt words, a pre-trained feature extraction model is used to perform a document feature extraction task to extract target features from the target document. The feature extraction model adopts a hybrid expert model architecture that includes multiple expert sub-models, and the multiple expert sub-models are used to perform different types of document feature extraction tasks. The target element is verified using the evidence map, and the target element that meets the verification is output.
2. The document element extraction method according to claim 1, characterized in that, The evidence graph includes multiple nodes and edges, and the construction of the evidence graph based on the identified document elements includes: Based on the parsing results of the target document, determine the spatial coordinates and content of each document element in the target document; The identified document elements are used as nodes in the evidence graph; Based on the spatial coordinates and content of each element, and combined with preset spatial and grammatical rules, the edges between each node in the evidence graph are defined, and multiple nodes are used to represent the association between corresponding elements.
3. The document element extraction method according to claim 2, characterized in that, The nodes and edges in the evidence graph are each accompanied by a confidence score output by the multimodal model.
4. The document element extraction method according to claim 1, characterized in that, The step of determining target prompts by combining the evidence map and the preset document element ontology includes: Based on the evidence map, determine the document type and topic of the target document; Traverse the preset historical case library and determine the prompt template that best matches the document type and topic of the target document from the historical case library; The prompt template is integrated with the document element ontology to generate the target prompt word.
5. The document element extraction method according to claim 1, characterized in that, Verifying the target element using the evidence map includes: Based on the target element, a subgraph matching query is performed in the evidence graph to determine whether there is a node or edge in the evidence graph that matches the target element. When a node or edge matching the target element is found, the target element is determined to satisfy the verification. If no node or edge matching the target element is found, the target element is determined not to meet the verification. Based on the verification results, feedback optimization is performed on the document element ontology or the dynamic prompts.
6. The document element extraction method according to claim 1, characterized in that, The feature extraction model includes a gating network, through which the feature extraction model adaptively allocates document feature extraction tasks to corresponding expert sub-models for execution.
7. The document element extraction method according to claim 1, characterized in that, The target prompts are adaptively updated as the document element ontology evolves, and the evolution mechanism of the document element ontology includes: Obtain the document element extraction results of newly added documents according to the preset cycle; Based on the document element extraction results, determine the candidate new elements or candidate new relationships that are not yet included in the current document element ontology; Determine the overall confidence level of the candidate new elements or the candidate new relationships; When the overall confidence level exceeds a preset threshold, the candidate new element or the candidate new relationship is updated in the document element ontology.
8. A document element extraction system, characterized in that, include: The data acquisition module is used to acquire target documents, which consist of several document elements; The document parsing module is used to parse the target document using a pre-trained multimodal model in order to identify the document elements in the target document; The prompt word construction module is used to construct an evidence map based on the identified document elements, and to determine the target prompt words by combining the evidence map with the preset document element ontology. The evidence map is used to indicate the index relationship between each document element in the target document, and the document element ontology is used to indicate the element type, element attributes and logical relationship between elements in the document element extraction task. The element extraction module is used to perform a document element extraction task based on the target prompt words and a pre-trained element extraction model, and to extract target elements from the target document. The element verification module is used to verify the target element using the evidence map and output the target element that meets the verification requirements.
9. An electronic device, characterized in that, include: At least one processor; and a memory communicatively connected to at least one of the processors; The memory stores instructions that can be executed by the processor to implement the document element extraction method according to any one of claims 1-7.
10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that are executed by the computer to implement the document element extraction method according to any one of claims 1-7.