Information extraction methods, devices, equipment and media

Abstract

This invention provides an information extraction method, apparatus, device, and medium, relating to the field of information extraction technology. The method includes acquiring candidate key-value pairs extracted from the current target text block; verifying the candidate key-value pairs based on the current global state dictionary and the target parameter field set to be extracted; updating the global state dictionary based on the verification results; and determining the next target text block based on the literature knowledge graph of the document to be processed and the chapter metadata of each text block in the text block sequence when the verification result indicates missing data; using the next target text block as the current target text block, continuing this step until a preset termination condition is met; and outputting the global state dictionary for the current extraction round. Thus, this invention maintains the context state through the global state dictionary and dynamically addresses the next target text block by combining the literature knowledge graph and chapter metadata when data is missing, overcoming the information silo problem of a single text block and thereby improving the completeness of the information extraction results.

Description

Technical Field

[0001] This invention relates to the field of information extraction technology, and in particular to an information extraction method, apparatus, device and medium. Background Technology

[0002] Scientific and technological literature contains a wealth of knowledge and key parameters. In order to guide new research and development activities and build high-quality professional databases, it is necessary to extract structured data from these long documents.

[0003] Currently, the common approach is to cut long documents into multiple text slices and then input them one by one into a large language model for information extraction, following the original text order. However, key parameter information in long documents is often scattered across chapters, so when complete parameter information cannot be provided within a certain slice, the information extraction results will have missing information. Summary of the Invention

[0004] This invention provides an information extraction method, apparatus, device, and medium to solve the technical problem that information is missing in the extraction results due to the use of slice-based sequential information extraction in the prior art.

[0005] This invention provides an information extraction method, the method comprising: The document to be processed is segmented into a sequence of text blocks, and a global state dictionary is initialized; the global state dictionary is used to store the extracted key-value pairs. Obtain candidate key-value pairs extracted from the current target text block, and verify the candidate key-value pairs based on the current global state dictionary and the target parameter field set to be extracted; Based on the verification results, the global state dictionary is updated and the next target text block is determined; The next target text block is taken as the current target text block, and the process of obtaining candidate key-value pairs extracted from the current target text block continues until the preset termination condition is met, and the global state dictionary under the current extraction round is output. When the verification result indicates missing data, the next target text block is determined based on the literature knowledge graph of the document to be processed and the chapter metadata of each text block in the text block sequence.

[0006] In some embodiments, the verification of the candidate key-value pairs based on the current global state dictionary and the target parameter field set to be extracted includes: Determine the target parameter fields in the target parameter field set to be extracted that match the candidate key-value pairs; If the candidate key-value pair is missing the first parameter value corresponding to the target parameter field, then the verification result is determined to be missing data. If the first parameter value is not missing in the candidate key-value pair, but the first parameter value is inconsistent with the second parameter value corresponding to the target parameter field in the current global state dictionary, then the context feature words of the current target text block are extracted. If the context feature words do not contain the control group feature words, the verification result is determined to be a data conflict; the control group feature words include at least the feature words used to characterize different experimental conditions or different control groups; If the first parameter value is not missing in the candidate key-value pair and the second parameter value corresponding to the target parameter field does not exist in the current global state dictionary, or if the first parameter value and the second parameter value in the candidate key-value pair are inconsistent but the control group feature word exists in the context feature word of the current target text block, then the verification result is determined to be data complete and conflict-free.

[0007] In some embodiments, if the verification result indicates missing data, the next target text block is determined in the following manner: Based on the literature knowledge graph, the target chapter type corresponding to the target parameter field is determined; Retrieve the target chapter metadata that matches the target chapter type from the chapter metadata of each text block in the text block sequence; The first text block in the text block sequence that is associated with the target chapter metadata is determined as the next target text block.

[0008] In some embodiments, if the verification result indicates a data conflict, the next target text block is determined in the following manner: Determine the text block associated with the second parameter value in the current global state dictionary; The next target text block is constructed by concatenating the text block associated with the second parameter value and the current target text block.

[0009] In some embodiments, when the verification result indicates a data conflict, after taking the next target text block as the current target text block, obtaining candidate key-value pairs extracted from the current target text block includes: Obtain the first parameter value and the second parameter value of the data conflict; Based on the context content of the current target text block, the first parameter value and the second parameter value are compared and inferred to determine the target parameter value; The target parameter field and the target parameter value are used as candidate key-value pairs extracted from the current target text block.

[0010] In some embodiments, if the verification result indicates that the data is complete and conflict-free, the next target text block is determined in the following manner: The next adjacent text block in the text block sequence is determined as the next target text block.

[0011] In some embodiments, updating the global state dictionary based on the verification result includes: If the verification result is that the data is missing, then the target parameter field and the related text block identifier are recorded in the global state dictionary, and the value of the target parameter field is marked as null. If the verification result is a data conflict, then the first parameter value, the second parameter value, and the related text block identifier corresponding to the target parameter field are associated in the global state dictionary and marked as a data conflict. If the verification result is that the data is complete and there are no conflicts, then the first parameter value corresponding to the target parameter field and the related text block identifier are associated and recorded in the global state dictionary.

[0012] In some embodiments, the preset termination condition includes: All target parameter fields in the target parameter field set of the current global state dictionary are not empty; The current global state dictionary does not contain any fields marked as data conflicts; and All text blocks in the text block sequence have been read.

[0013] In some embodiments, the method further includes: Obtain the global state dictionary under multiple extraction rounds; Data alignment is performed on the target parameter fields in multiple global state dictionaries to obtain the set of parameter values ​​corresponding to the target parameter fields; Based on the number of times the same parameter value appears in the set of parameter values ​​corresponding to the target parameter field, the merged parameter value and the associated confidence index corresponding to the target parameter field are determined; Based on the merge parameter value corresponding to the target parameter field, the confidence index, and the text block index, the corresponding target structured data is output.

[0014] The present invention also provides an information extraction device, the device comprising: An initialization module is used to segment the document to be processed into a sequence of text blocks and initialize a global state dictionary; the global state dictionary is used to store the extracted key-value pairs. The extraction module is used to obtain candidate key-value pairs extracted from the current target text block, and to verify the candidate key-value pairs based on the current global state dictionary and the target parameter field set to be extracted. The update module is used to update the global state dictionary and determine the next target text block based on the verification result; The output module is used to take the next target text block as the current target text block, continue to execute the process of obtaining candidate key-value pairs extracted from the current target text block until a preset termination condition is met, and output the global state dictionary under the current extraction round. When the verification result indicates missing data, the next target text block is determined based on the literature knowledge graph of the document to be processed and the chapter metadata of each text block in the text block sequence.

[0015] The present invention also provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the information extraction method as described above.

[0016] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the information extraction method as described above.

[0017] The present invention also provides a computer program product, including a computer program that, when executed by a processor, implements the information extraction method as described above.

[0018] The information extraction method, apparatus, device, and medium provided by this invention first divide the document to be processed into a sequence of text blocks and initialize a global state dictionary. Then, candidate key-value pairs are extracted from the current target text block, and the candidate key-value pairs are verified based on the current global state dictionary and the target parameter field set to be extracted. In this way, the global state dictionary realizes a global record of the extracted key-value pairs, forming a verification benchmark with historical context memory. Then, when the verification result is that the data is missing, the domain prior knowledge carried by the document knowledge graph and the chapter metadata of each text block in the text block sequence are combined to perform semantic inference on the distribution position of the missing data in the document, thereby accurately locating and jumping to the text block most likely to contain the target information. This overcomes the information fragmentation problem of the traditional sequential scanning mode and improves the accuracy of recalling related parameters scattered in different structural parts of long documents. Finally, the determined next target text block is used as the current target text block, and the above process is continuously repeated until the preset termination condition is met. The global state dictionary under the current extraction round is output, thereby improving the completeness of structured information extraction from long and complex documents. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0020] Figure 1 This is one of the flowcharts illustrating the information extraction method provided by the present invention.

[0021] Figure 2 This is the second flowchart of the information extraction method provided by the present invention.

[0022] Figure 3 This is a schematic diagram of the information extraction device provided by the present invention.

[0023] Figure 4 This is a schematic diagram of the structure of the electronic device provided by the present invention. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0025] Materials science is a crucial cornerstone of modern industry and high-tech development, and scientific literature serves as the core carrier of knowledge and process parameters for materials research and development. With the explosive growth of global scientific output, the number of related documents is vast. Researchers need to extract key information such as material composition, preparation processes, and performance parameters to guide new research and development activities. However, traditional methods relying on manual reading and information extraction suffer from extremely low efficiency, strong subjectivity, susceptibility to errors, and difficulty in scaling, severely hindering the discovery and optimization iteration speed of new materials.

[0026] To address this issue, researchers have explored various automated information extraction techniques. Traditional methods based on keyword matching or regular expressions, while achieving partial automation, struggle to handle the complex and varied sentence structures and technical terminology found in scientific literature, resulting in unsatisfactory extraction accuracy and recall. Subsequent machine learning and deep learning models, such as named entity recognition, while performing better on specific tasks, typically require extensive supervised training with manually labeled data within the domain. This leads to high model building costs and limited generalization capabilities, making it difficult to handle long-distance dependencies and implicit associations present throughout the entire document.

[0027] In recent years, researchers have begun to explore the use of large language models for information extraction. By designing specific prompts, they can directly input document fragments into the large language model and require it to output the necessary parameter information in a specified format, thus simplifying the model training process to some extent.

[0028] However, due to the limitations of the model's context window length, processing an entire document requires breaking it down into smaller segments and processing them sequentially according to the original text. This segmented approach disrupts the overall structure and logical connections of the article, making it difficult for the model to understand key information that is interdependent across multiple segments. For example, the specific process parameters of a material system mentioned in the introduction may only be detailed in a figure in the experimental section. Therefore, this fragmentation of information often results in incomplete and inaccurate extraction results.

[0029] To address the above problems, this invention provides an information extraction method. In this embodiment, the execution entity is an information extraction system, which includes a processor and a memory. The processor executes the steps of this method by running a computer program stored in the memory. In specific implementation, an intelligent agent runs within the processor. This intelligent agent, as the core execution entity, is responsible for driving the invocation of reading tools, thinking tools, and writing tools, and maintaining a global state dictionary. The execution of the method steps in subsequent embodiments is also implemented by the aforementioned information extraction system through the intelligent agent, and will not be described in detail here.

[0030] Figure 1 This is one of the flowcharts illustrating the information extraction method provided in this embodiment of the invention, such as... Figure 1 As shown, the method includes the following steps 110, 120, 130 and 140.

[0031] Step 110: The document to be processed is segmented into a sequence of text blocks, and a global state dictionary is initialized; the global state dictionary is used to store the extracted key-value pairs.

[0032] Here, the document to be processed refers to the original document input into the system. Its form can be a scientific paper, research report, or other document carrying material process parameter information, and the format includes, but is not limited to, PDF, Word, or plain text. Considering the limitation of the context window length on large language models, it is not possible to process the entire document at once. Therefore, in this embodiment, the document to be processed is first divided into a sequence of text blocks.

[0033] Specifically, the original document to be processed is first converted to a different format. For example, when the original document is in PDF format, a document parsing tool can be used to convert the PDF file into Markdown format that can be read directly by a machine. During the conversion process, the main text, table content, and headings at all levels in the original document are retained to preserve the structured information, while headers, footers, page numbers and other interfering elements that are irrelevant to the understanding of the content are removed.

[0034] After format conversion of the document to be processed, since the document is usually quite long, this embodiment will also segment the document according to a preset segmentation rule to form multiple ordered text blocks, i.e., a text block sequence. Each text block in the text block sequence carries a unique text block identifier, so that the agent can accurately locate and call specific text blocks as needed based on the text block identifier.

[0035] In practical applications, the preset segmentation rules can be flexibly set according to extraction requirements. For example, the segmentation rules can be equal-length segmentation based on a preset number of characters, or semantically-aware segmentation based on the natural paragraph boundaries or chapter boundaries of the document, without any restrictions. Furthermore, when segmenting according to the preset character length, the configuration can be flexibly adjusted based on the context window capacity of the large language model used.

[0036] In addition, during the segmentation process, the corresponding chapter metadata, such as abstract, introduction, experimental section, results and discussion, will be labeled for each text block based on the title information of the chapter in which the segmented text block is located.

[0037] After constructing the text block sequence, the global state dictionary is initialized synchronously. The global state dictionary is a key-value pair data structure maintained in memory by the information extraction system. It is used to store in real-time the parameter fields extracted by the agent during the reading of each text block and their corresponding parameters. Here, the keys of the global state dictionary are parameter field names, such as material name, deposition equipment, annealing temperature, etc.; the values ​​are the specific parameter values ​​extracted from the text.

[0038] It should be noted that each parameter value in the global state dictionary in this embodiment is bound to an associated text block identifier, i.e., a source tag, which is used to record which text block in the text block sequence the parameter value comes from. In addition, during the initialization phase, the values ​​corresponding to all target parameter fields in the global state dictionary are set to empty values.

[0039] Furthermore, during the initialization phase, a corresponding target parameter field set is predefined based on the extraction task. It should be understood that the target parameter field set refers to the predefined collection of all target parameter fields that need to be extracted from the literature to be processed. For example, in the scenario of extracting material process parameters, the target parameter field set may include target parameter fields such as material name, deposition equipment, annealing temperature, annealing atmosphere, film thickness, and remanent polarization intensity.

[0040] Step 120: Obtain candidate key-value pairs extracted from the current target text block, and verify the candidate key-value pairs based on the current global state dictionary and the target parameter field set to be extracted.

[0041] After constructing the text block sequence and initializing the global state dictionary, the agent is invoked to begin executing the information extraction and verification loop. At the beginning of each loop, the agent reads the text content of the current target text block and, based on the semantic understanding capabilities of the large language model, identifies and extracts candidate key-value pairs from the current target text block.

[0042] Here, candidate key-value pairs refer to the pairing results of unverified parameter fields and parameter values ​​identified by the agent in the context of the current target text block. For example, identifying the annealing temperature as 400℃ constitutes a candidate key-value pair.

[0043] After obtaining candidate key-value pairs, the thinking tool is invoked to compare the candidate key-value pairs obtained from the current target text block with the target parameter field set one by one. At the same time, it compares with the information stored in the global state dictionary to determine whether there are any missing or conflicting data in the candidate key-value pairs, and then determines the verification result.

[0044] In this embodiment, the verification result includes at least three scenarios: missing data, data conflict, and complete data without conflict. Different verification results will trigger different subsequent processing logic.

[0045] Specifically, the candidate key-value pairs obtained from the current target text block are iterated and compared with the target parameter field set to check for cases where a target parameter field has been identified but its corresponding parameter value is still empty. If such cases exist, it indicates that the current text block fails to provide the corresponding parameter information, resulting in missing data. Furthermore, for parameter values ​​contained in candidate key-value pairs, before writing them into the global state dictionary, the system searches the current global state dictionary to see if the same target parameter field already has a historical value. If a historical value exists and the values ​​are inconsistent, the agent further utilizes the semantic understanding capabilities of the large language model to verify whether there is a data conflict.

[0046] Step 130: Based on the verification result, update the global state dictionary and determine the next target text block.

[0047] After completing the above verification, the agent calls the writing tool to write the verified parameter information into the global state dictionary. For example, it writes the newly confirmed parameter values ​​and their corresponding text block identifiers, or marks the fields with missing or conflicting information with the corresponding status. At the same time, it dynamically determines the next target text block from the text block sequence based on the current verification results.

[0048] In one example, when the validation result indicates missing data, it means that in the currently instantiated material entity or parameter system, the value of a required parameter field of the target parameter field set is still empty. Simply reading the current text block sequentially cannot fill the missing field. Therefore, the next target text block is determined based on the literature knowledge graph of the document to be processed and the chapter metadata of each text block in the text block sequence. In this way, by combining the semantic features of the missing parameter with the chapter attribution information of the text block, targeted retrieval of target information across the entire text is achieved.

[0049] Here, the literature knowledge graph refers to the domain knowledge about the structural patterns of scientific and technological literature built into the large language model. This knowledge graph encodes the regularity of which chapters of a document a specific type of target parameter field typically appears in. For example, the deposition equipment field typically appears in the experimental methods chapter, and the material properties field typically appears in the results and discussion chapter. Chapter metadata is the chapter title information labeled for each text block during the text block sequence segmentation stage.

[0050] Based on the knowledge graph of the literature, the agent can determine the target chapter type corresponding to the missing target parameter field, and then locate the text block that matches the target chapter type by retrieving the chapter metadata of each text block. This enables precise skipping reading across paragraphs, avoids invalid sequential scanning, and improves the efficiency of obtaining scattered information in long documents.

[0051] In one example, when the verification result is a data conflict, it indicates that there is an inconsistency between the parameter value in the candidate key-value pair extracted from the current target text block and the existing historical parameter value in the global state dictionary. In this case, the historical text block associated with the historical key-value pair that conflicts with the candidate key-value pair in the current global state dictionary is concatenated with the content of the current target text block to form a combined text block that simultaneously contains two conflicting parameter values ​​and their respective contexts. This combined text block is then used as the next target text block, enabling the agent to make an accurate judgment after obtaining the complete original text context.

[0052] In one example, when the verification result is that the data is complete and conflict-free, the next adjacent text block of the current target text block in the text block sequence is determined as the next target text block, and the reading and extraction of the document content continues in sequence.

[0053] Step 140: Take the next target text block as the current target text block, continue to execute the process of obtaining candidate key-value pairs extracted from the current target text block until the preset termination condition is met, and output the global state dictionary under the current extraction round.

[0054] In this embodiment, after dynamically determining the next target text block, the next target text block is used as new input to continue the above-described process of extracting candidate key-value pairs, verifying, updating the global state dictionary and determining the next target text block, until the preset termination condition is met.

[0055] In practical applications, the preset termination conditions can be set flexibly, such as reaching the maximum number of text blocks read, or all target parameter fields have been successfully extracted and there are no pending conflicts.

[0056] Once the preset termination condition is met, the current global state dictionary is used as the output of this extraction round. This global state dictionary contains the parameter values ​​of all target parameter fields extracted from the documents to be processed and their corresponding text block identifiers.

[0057] Furthermore, before outputting the current global state dictionary, the agent can also initiate a global verification process, that is, relocate to the corresponding text block in the original document to be processed based on the text block identifier, compare the text content in the text block with the parameter value of the corresponding target parameter field, and output the global state dictionary after the comparison is correct.

[0058] For example, when extracting information from a materials science paper on the fabrication process of HfO2 ferroelectric thin films, the target parameter field set includes mandatory fields such as material name, deposition equipment, annealing temperature, annealing time, and film thickness. Initially, all fields in the global state dictionary are empty. The agent first reads text block chunk0, identifies the material name as HfO2, verifies that the data is complete and conflict-free, writes the material name HfO2 and the text block identifier chunk0 into the global state dictionary, and proceeds sequentially to text block chunk1. After reading text block chunk1, the agent does not find any new mandatory field information and continues sequentially to text block chunk2. When reading text block chunk3, the agent identifies the annealing temperature as 400℃, verifies that the data is complete and conflict-free, and writes it into the global state dictionary. If the deposition device field in the global state dictionary is still empty, the agent triggers a data missing detection. Based on the literature knowledge graph and the chapter metadata of each text block, it retrieves text blocks labeled with the experimental section's chapter metadata as text block chunk2 and text block chunk3. If text block chunk2 has not yet been read, it jumps to text block chunk2 for reading; if it has been read but no deposition device information is found, it continues searching within that chapter until the missing field is filled. The entire iteration process continues until the preset termination condition is met, and finally outputs a global state dictionary containing all target parameter fields.

[0059] The information extraction method provided in this invention first divides the document to be processed into a sequence of text blocks and initializes a global state dictionary. Then, it obtains candidate key-value pairs extracted from the current target text block and verifies the candidate key-value pairs based on the current global state dictionary and the target parameter field set to be extracted. In this way, the global state dictionary realizes a global record of the extracted key-value pairs, forming a verification benchmark with historical context memory. Then, when the verification result is that the data is missing, it combines the domain prior knowledge carried by the document knowledge graph and the chapter metadata of each text block in the text block sequence to perform semantic inference on the distribution position of the missing data in the document, thereby accurately locating and jumping to the text block most likely to contain the target information. This overcomes the information fragmentation problem of the traditional sequential scanning mode and improves the accuracy of recalling related parameters scattered in different structural parts of long documents. Finally, the determined next target text block is used as the current target text block and the above process is continuously repeated until the preset termination condition is met. The global state dictionary under the current extraction round is output, thereby improving the completeness of structured information extraction from long and complex documents.

[0060] In some embodiments, the verification of the candidate key-value pairs based on the current global state dictionary and the target parameter field set to be extracted includes: Determine the target parameter fields in the target parameter field set to be extracted that match the candidate key-value pairs; If the candidate key-value pair is missing the first parameter value corresponding to the target parameter field, then the verification result is determined to be missing data. If the first parameter value is not missing in the candidate key-value pair, but the first parameter value is inconsistent with the second parameter value corresponding to the target parameter field in the current global state dictionary, then the context feature words of the current target text block are extracted. If the context feature words do not contain the control group feature words, the verification result is determined to be a data conflict; the control group feature words include at least the feature words used to characterize different experimental conditions or different control groups; If the first parameter value is not missing in the candidate key-value pair and the second parameter value corresponding to the target parameter field does not exist in the current global state dictionary, or if the first parameter value and the second parameter value in the candidate key-value pair are inconsistent but the control group feature word exists in the context feature word of the current target text block, then the verification result is determined to be data complete and conflict-free.

[0061] It should be understood that candidate key-value pairs are combinations of parameter names and parameter values ​​identified and extracted by the agent from the natural language text of the current target text block, with the key names typically presented in the original natural language form. The target parameter field set, on the other hand, is a collection of parameter fields to be extracted, represented by standardized internal field names predefined by the system. For example, in the scenario of extracting material process parameters, the target parameter field set may include standardized field names such as material name, deposition equipment, annealing temperature, annealing atmosphere, film thickness, and remanent polarization intensity.

[0062] Since the same parameter concept may have multiple different natural language expressions in scientific and technological literature, the key name in the candidate key-value pair may not be completely consistent with the standardized field name in the target parameter field set. Therefore, in this embodiment, the agent uses the semantic understanding capability of the large language model to perform semantic judgment on the key name of the candidate key-value pair and the field names in the target parameter field set.

[0063] Specifically, the keys of each key-value pair in the candidate key-value pairs are sequentially compared with the target parameter fields in the target parameter field set for semantic similarity. Keys that are semantically the same or have high similarity are mapped to the corresponding target parameter fields, thereby determining the target parameter fields for this verification.

[0064] For example, the target parameter field set contains the standardized field name "annealing temperature." The agent extracts a candidate key-value pair from a text block with the key name "annealing temperature." While this key name doesn't perfectly match the annealing temperature at the character level, semantic matching determines that it has the same semantic meaning as the annealing temperature field in the target parameter field set. Therefore, it is identified as the target parameter field for this validation. If a key name in a candidate key-value pair cannot establish a semantic correspondence with any field in the target parameter field set, that key-value pair will not participate in subsequent validation processes.

[0065] After determining the target parameter field for this validation, the system continues to check whether the candidate key-value pairs contain the first parameter value corresponding to that target parameter field. Here, the first parameter value refers to the specific parameter value extracted by the agent from the current target text block for that target parameter field.

[0066] If the first parameter value corresponding to the target parameter field is missing in the candidate key-value pair, then the result of this verification is determined to be data missing. For example, if the target parameter field set includes the field "deposition device", and the agent extracts candidate key-value pairs from the current target text block while reading it, the candidate key-value pairs extracted from the current target text block only contain the deposition device field, but do not contain any parameter values ​​that can match the deposition device field, then it is determined that the deposition device field has missing data.

[0067] If no first parameter value corresponding to the target parameter field is missing in the candidate key-value pair, the process proceeds to the consistency verification phase. Specifically, the current global state dictionary is searched to confirm whether the target parameter field already has a historical value in the global state dictionary, i.e., the second parameter value. It should be understood that the second parameter value is the parameter value corresponding to the target parameter field that was written into the global state dictionary by the agent in a previous round of reading before this verification.

[0068] If the target parameter field in the global state dictionary has no historical value, then there is no second parameter value to compare with the first parameter value, and the data is determined to be complete and conflict-free.

[0069] If the target parameter field already has a second parameter value in the global state dictionary, the first parameter value is compared with the second parameter value. If the two parameter values ​​are inconsistent, further contextual semantic analysis is triggered. Specifically, contextual feature words are extracted from the current target text block, and it is determined whether there are control group feature words among the contextual feature words.

[0070] Here, contextual feature words refer to words or phrases located near the first parameter value in the current target text block that explain the experimental context or conditions attribution of that parameter value. Control group feature words refer to words that can characterize that the experimental conditions or samples described in the current context belong to a certain control group or specific preconditions, such as sample A / B, group 1 / 2, reference sample, etc., which characterize different experimental groups; and words that characterize different experimental conditions, such as different annealing temperatures, changes with temperature, and different conditions.

[0071] If no control group feature words are found in the context feature words, it is determined that there is a contradiction between the first parameter value and the second parameter value. The verification result is a data conflict, and the subsequent repeated reading strategy is triggered. The historical text block that generated the second parameter value is concatenated with the current target text block and re-input into the agent for cross-comparison and inference.

[0072] If the candidate key-value pair does not lack the first parameter value corresponding to the target parameter field, and the first parameter value is consistent with the existing second parameter value of the target parameter field in the global state dictionary, or the target parameter field has no historical value in the global state dictionary, then the verification result is determined to be that the data is complete and conflict-free.

[0073] If the first parameter value in a candidate key-value pair is inconsistent with the second parameter value in the global state dictionary, but the contextual feature words of the current target text block contain control group feature words, it indicates that the two different parameter values ​​belong to different experimental conditions or comparison groups. In this case, the verification result is determined to be data complete and conflict-free. In the subsequent global state dictionary update step, the two parameter values, along with their corresponding experimental condition annotations and text block identifiers, are linked and recorded together to fully preserve the process parameter information for different experimental conditions in the literature.

[0074] The information extraction method provided in this invention achieves refined hierarchical processing of candidate key-value pairs by combining an ordered combination of integrity checks and consistency checks with semantic judgment of control group feature words in context feature words, thus ensuring the accuracy of the verification results.

[0075] In some embodiments, if the verification result indicates missing data, the next target text block is determined in the following manner: Based on the literature knowledge graph, the target chapter type corresponding to the target parameter field is determined; Retrieve the target chapter metadata that matches the target chapter type from the chapter metadata of each text block in the text block sequence; The first text block in the text block sequence that is associated with the target chapter metadata is determined as the next target text block.

[0076] In this embodiment, the agent invokes its built-in document knowledge graph, which encodes the correspondence between different types of target parameter fields and document chapters in scientific and technological literature. Here, the target chapter type refers to the chapter category most likely to contain the missing target parameter field information, inferred from the document knowledge graph, and is represented as a chapter type name or its semantic equivalent.

[0077] In the literature knowledge graph, the typical correspondence between different target parameter fields and target chapter types includes, but is not limited to: fields describing experimental equipment and preparation process parameters correspond to target chapter types of experimental section or materials and methods; fields describing sample structural parameters correspond to target chapter types of experimental section; fields describing material performance characterization results correspond to target chapter types of results or results and discussion; and fields describing the overall performance comparison of material systems correspond to target chapter types of conclusion or abstract.

[0078] In this embodiment, the agent inputs the target parameter field as a query entity into the document knowledge graph. Through semantic association and attribute reasoning between entities in the document knowledge graph, it matches and determines the chapter type to which the target parameter field belongs in the document, and finally obtains the target chapter type corresponding to the target parameter field.

[0079] For example, the literature knowledge graph has pre-established association mapping relationships between target parameter fields such as annealing temperature, deposition equipment, and remanent polarization intensity and chapter types such as experimental methods, preparation processes, and performance characterization. When annealing temperature is used as the target parameter field, it is input into the literature knowledge graph as a query entity. Through semantic retrieval and association reasoning, the experimental methods and sample preparation chapter types connected to the target parameter field in the literature knowledge graph are matched, thereby determining that the target chapter type corresponding to the annealing temperature target parameter field is the experimental process chapter.

[0080] After determining the target chapter type, the chapter metadata of each text block in the text block sequence is searched and matched to filter out the chapter titles and their corresponding metadata with the same annotation type, thereby locating the text area that may contain the target parameters.

[0081] For example, if the target chapter type is experimental process, and the chapter metadata of the literature includes chapter titles such as introduction, experimental methods, sample preparation, results and discussion, as well as their corresponding type labels, then the metadata related to chapter titles such as experimental methods and sample preparation with the type label of experimental process will be retrieved and filtered from the chapter metadata sequence, and used as the target chapter metadata on which the target parameters are subsequently extracted.

[0082] It should be noted that chapter metadata is a chapter affixation tag added to each text block during the document preprocessing stage when the text is segmented. It records the chapter metadata to which each text block belongs, such as which chapter it belongs to, the chapter title, and the chapter type. By retrieving chapter metadata, the range of chapters matching the target chapter type can be quickly located within the entire text block sequence without scanning each block individually.

[0083] Since the starting position of the target chapter is most likely to contain the overall description and key parameter information of the chapter, reading from that position allows for finding the required parameter via the shortest path. Furthermore, if the required parameter is not found in the first text block, the agent will read subsequent text blocks of the chapter sequentially until the corresponding parameter is found or it is confirmed that the chapter does not contain the relevant information. Therefore, in this embodiment, after determining the source data of the target chapter, the first text block in the text block sequence corresponding to the target chapter title metadata is determined as the next target text block.

[0084] For example, in a materials science paper's text block sequence, text block 0 corresponds to the abstract, text blocks 1 to 3 correspond to the introduction, text blocks 4 to 6 correspond to experimental methods, and text blocks 7 to 10 correspond to results and discussion. When the agent sequentially reads text block 2, it detects missing data in the deposition equipment field. Based on the literature knowledge graph, it determines the target chapter type as experimental methods. If the chapter metadata of text blocks 4 to 6 matches the experimental methods, then text block 4 is identified as the next target text block, and the agent directly jumps to text block 4 for reading.

[0085] It should be further noted that in this embodiment, in the case of missing data, after adjusting the chapter metadata of each text block in the document knowledge graph and text block sequence to the next target text block, only the parameter values ​​corresponding to the target parameter fields of the missing data are read in the next target text block. After reading, the current target text block is returned, and the next adjacent text block is read in the order of the text block sequence.

[0086] The information extraction method provided in this invention locates the corresponding target chapter metadata based on the semantic analysis results of the target parameter field when data is missing, and selects the first text block from multiple text blocks matched by the target chapter metadata as the next target text block, thereby achieving accurate and efficient cross-paragraph jumps and improving the information extraction efficiency of long documents.

[0087] In some embodiments, if the verification result indicates a data conflict, the next target text block is determined in the following manner: Determine the text block associated with the second parameter value in the current global state dictionary; The next target text block is constructed by concatenating the text block associated with the second parameter value and the current target text block.

[0088] When the agent determines through its thinking tools that the verification result indicates a data conflict, it means that the same target parameter field has generated contradictory parameter values ​​in different text blocks that cannot be semantically interpreted by the control group. Based on this, the agent needs to simultaneously obtain the original content of the historical text block that generated the second parameter value and the original content of the current target text block that generated the first parameter value, and perform cross-referencing.

[0089] Specifically, since each stored parameter value in the global state dictionary is bound to the text block identifier of its source text block, the agent can locate the historical text block that generated the second parameter value by querying the text block identifier bound to the second parameter value corresponding to the target parameter field in the global state dictionary. The agent outputs a repeat reading instruction. After receiving this instruction, the information extraction system concatenates the old value's source text block with the current target text block in a certain order to form a new composite text block. This composite text block is then provided to the agent as the next target text block for reading. This composite text block retains the original context information of both text blocks, allowing the agent to simultaneously obtain the original context that generated the two conflicting parameter values ​​during comparative reading.

[0090] In some embodiments, when the verification result indicates a data conflict, after taking the next target text block as the current target text block, obtaining candidate key-value pairs extracted from the current target text block includes: Obtain the first parameter value and the second parameter value of the data conflict; Based on the context content of the current target text block, the first parameter value and the second parameter value are compared and inferred to determine the target parameter value; The target parameter field and the target parameter value are used as candidate key-value pairs extracted from the current target text block.

[0091] In this embodiment, the agent, having simultaneously acquired the original text context that generated the first and second parameter values, invokes the semantic understanding and reasoning capabilities of the large language model to conduct an in-depth comparative analysis of the original contexts of the two conflicting parameter values, infers the root cause of the contradiction between them, and determines the final target parameter value based on this.

[0092] Specifically, the methods for determining the target parameter value include, but are not limited to, the following: Scenario 1: Conflict caused by typos or formatting errors. If, after comparing and reading two original texts, the agent determines that there is an obvious typo in the source text of one of the parameter values, such as mistakenly writing 400℃ as 500℃, while it is consistently recorded as 400℃ in many other places in the text, then the parameter value that appears consistently multiple times is determined as the target parameter value; Scenario 2: Conflicts arising from inconsistencies in the descriptions within the literature. If the agent determines that the parameter values ​​in the two original texts come from different writing stages of the literature, such as inconsistencies between the abstract and the experimental methods section, then considering the contextual logic of the two original texts, the parameter value from the experimental methods section will be prioritized as the target parameter value. Scenario 3: Conflicts caused by differences in unit conversion or expression methods. If the agent determines that two parameter values ​​actually describe the same value, such as 400℃ recorded in one place and 673K ​​recorded in another, then the two expressions will be unified into the standard expression to determine the target parameter value; Scenario 4: Conflicts that cannot be resolved through reasoning. If the agent still cannot determine which parameter value is correct after comparison and reading, both the first and second parameter values ​​are retained as parallel candidate values, and the field is marked as unresolved conflict in the global state dictionary for subsequent global verification or manual review.

[0093] Finally, the agent uses the target parameter fields and target parameter value combinations determined through comparison and inference as candidate key-value pairs and submits them to the subsequent writing tool to update the global state dictionary.

[0094] The information extraction method provided in this invention effectively avoids the risk of illusion in large language models when there is no original text basis by concatenating the original source text of conflicting parameter values ​​into a unified context and performing comparative reasoning, thereby improving the accuracy and credibility of the extraction results.

[0095] In some embodiments, if the verification result indicates that the data is complete and conflict-free, the next target text block is determined in the following manner: The next adjacent text block in the text block sequence is determined as the next target text block.

[0096] When the agent determines through its thinking tool that the verification result is complete and conflict-free, it indicates that the candidate key-value pairs extracted from the current target text block meet the requirements of the target parameter field set and do not contradict the existing data in the global state dictionary. In this case, the agent outputs a sequential reading instruction, and the information extraction system loads the next adjacent text block according to its sequence number, allowing the agent to continue extracting and verifying candidate key-value pairs.

[0097] The information extraction method provided in this invention ensures that, under the condition that information acquisition is smooth and there is no need to jump or backtrack, the text block sequence can be traversed in natural order with the lowest computational cost, maximizing the use of information in each part of the document and avoiding the omission of parameter information scattered throughout the text.

[0098] In some embodiments, updating the global state dictionary based on the verification result includes: If the verification result is that the data is missing, then the target parameter field and the related text block identifier are recorded in the global state dictionary, and the value of the target parameter field is marked as null. If the verification result is a data conflict, then the first parameter value, the second parameter value, and the related text block identifier corresponding to the target parameter field are associated in the global state dictionary and marked as a data conflict. If the verification result is that the data is complete and there are no conflicts, then the first parameter value corresponding to the target parameter field and the related text block identifier are associated and recorded in the global state dictionary.

[0099] Specifically, if the verification result indicates missing data, it means that the current target text block failed to provide a valid parameter value for the target parameter field. In this case, the agent invokes the writing tool to mark the value of the target parameter field as null in the global state dictionary, and simultaneously records the text block identifier of the current target text block, indicating that the field has been searched but a valid value could not be obtained in that text block. By recording the text block identifier, the system can track which text blocks have been searched during subsequent reading, avoiding repeated invalid reads on the same text block.

[0100] If the verification result indicates a data conflict, the agent invokes the writing tool to associate and record the first parameter value, second parameter value, and their respective text block identifiers corresponding to the target parameter field in the global state dictionary, and marks the field as a data conflict. This ensures that the global state dictionary can completely record conflict information and that the logic for determining the preset termination condition can identify unresolved conflict fields in the current global state dictionary, thus preventing premature termination of the extraction process.

[0101] If the verification result shows that the data is complete and conflict-free, the agent calls the writing tool to associate and record the first parameter value corresponding to the target parameter field and the text block identifier of its source text block in the global state dictionary. If the field already has an old value in the global state dictionary, the old value is overwritten with the first parameter value, and the bound text block identifier is updated synchronously; if the field is being written for the first time, the first parameter value and the corresponding text block identifier are recorded directly.

[0102] The information extraction method provided in this embodiment of the invention ensures that the global state dictionary can accurately and completely reflect the current extraction state throughout the entire extraction process, recording both the confirmed valid parameter values ​​and their traceability information, and marking the missing and conflicting fields that are yet to be processed.

[0103] In some embodiments, the preset termination condition includes: All target parameter fields in the target parameter field set of the current global state dictionary are not empty; The current global state dictionary does not contain any fields marked as data conflicts; and All text blocks in the text block sequence have been read.

[0104] In this embodiment, the system can terminate the iteration loop of the current extraction round and output the result when the following three sub-conditions are met simultaneously: Sub-condition 1: All required target parameter fields defined in the target parameter field set have been assigned non-empty parameter values ​​in the global state dictionary, meaning that no field still has the empty value it had at initialization. This sub-condition ensures the integrity of the extraction results and prevents the system from terminating prematurely if there are still required fields that have not been filled. Sub-condition 2: No target parameter field in the global state dictionary is still in a data conflict marking state. This means that all fields marked as conflicting during the extraction process have been resolved through conflict resolution, ultimately determining a unique target parameter value. This sub-condition ensures the internal consistency of the extraction results, preventing the system from outputting unreliable results when unresolved conflicts still exist. Sub-condition 3: The text block with the largest sequence number in the text block sequence has been read and processed by the agent module.

[0105] It should be noted that in actual processing, if sub-conditions one and two are met, but the entire text block sequence has not yet been read, the system will continue to read the remaining text blocks sequentially to check whether there is any correction or supplementary information for the extracted parameter values ​​in subsequent text blocks, ensuring the accuracy of the final result. Conversely, if the entire text block sequence has been read, but there are still null value fields or unresolved conflict fields, the system will add missing or unresolved conflict annotations to the relevant fields in the output result to accurately reflect the extraction status and provide clear reference information for subsequent manual review or multi-round merging verification.

[0106] The information extraction method provided in this embodiment of the invention ensures the accuracy of the extraction results in a single round by setting the termination conditions described above.

[0107] In some embodiments, the method further includes: Obtain the global state dictionary under multiple extraction rounds; Data alignment is performed on the target parameter fields in multiple global state dictionaries to obtain the set of parameter values ​​corresponding to the target parameter fields; Based on the number of times the same parameter value appears in the set of parameter values ​​corresponding to the target parameter field, the merged parameter value and the associated confidence index corresponding to the target parameter field are determined; Based on the merge parameter value corresponding to the target parameter field, the confidence index, and the text block index, the corresponding target structured data is output.

[0108] The information extraction system performs multiple extraction processes independently for the same document to be processed. Each execution is considered an independent extraction round, outputting an independent global state dictionary. Due to the inherent randomness of output in large language models during each call, multiple independent extractions will generate multiple global state dictionaries that are highly similar yet may have subtle differences, denoted as Global State Dictionary 1, Global State Dictionary 2, ..., Global State Dictionary N, where N is the preset total number of extraction rounds.

[0109] refer to Figure 2 As shown, in a specific embodiment, N is 3, that is, the system performs three complete independent extraction processes for the same document to be processed, and obtains global state dictionary 1, global state dictionary 2 and global state dictionary 3, a total of three extraction results for subsequent merging and verification.

[0110] The information extraction system submits multiple global state dictionaries to a large language model specifically designed for merging. First, a data alignment operation is performed. Specifically, for each global state dictionary, using the target parameter field name as the alignment key, data entries referring to the same target parameter field in the multiple dictionaries are merged accordingly, resulting in a set of all parameter values ​​generated by each target parameter field in multiple rounds of extraction—that is, the parameter value set.

[0111] For material process parameter information extraction scenarios, the combination of the material name field and the key test condition field can be used as the joint primary key for data alignment. Data entries from multiple global state dictionaries that refer to the same material entity and the same test condition can be grouped into one group, and the parameter value set of each target parameter field can be statistically analyzed within this group.

[0112] The information extraction system performs statistical analysis on the set of parameter values ​​for each target parameter field, using the frequency of occurrence of identical parameter values ​​in the set as the primary basis, and determines the merged parameter values ​​and corresponding confidence level indicators according to the following rules: Consistency priority rule: If a parameter value in the parameter value set appears more than the majority of the total number of sampling rounds, for example, appearing two or three times in three sampling rounds, then the parameter value is determined as the merged parameter value, and a corresponding confidence level is assigned according to the number of occurrences: merged parameter values ​​that appear three times are assigned a high confidence level; merged parameter values ​​that appear twice are assigned a medium confidence level. Conflict handling record rules: If the parameter values ​​extracted from the three rounds of parameter value sets are all different, the three parameter values ​​will be recorded as a merged parameter value, for example, by connecting them with the separator "‖", or by retaining the parameter value with the most comprehensive and complete information. The merged parameter value will be given a low confidence level label and a multi-value parallel label will be added to indicate that the reliability of this field is low in downstream applications and manual verification is recommended.

[0113] Finally, the information extraction system uses the merged target parameter field, merged parameter value, confidence level identifier, and source text block identifier as core fields to generate and output target structured data according to a preset output format. Each data record in the target structured data includes: the target parameter field name, merged parameter value, confidence level identifier, and the source text block identifier that supports the merged parameter value.

[0114] Among them, the confidence level indicator provides a quantifiable reference for the reliability of data in the construction and application of downstream material databases. For example, high-confidence data can be directly entered into the database for use, medium-confidence data is recommended to be entered into the database after quick verification, and low-confidence data is recommended to be submitted for manual review before deciding whether to enter into the database.

[0115] The information extraction method provided in this invention improves the robustness of the final output structured data by performing multiple rounds of independent extraction and intelligent merging verification on the same document to be processed, and by using a majority voting mechanism to avoid occasional errors caused by randomness in a single model call.

[0116] Based on any of the above embodiments, the present invention also provides an information extraction device, with reference to Figure 3 The device includes: The initialization module 310 is used to segment the document to be processed into a sequence of text blocks and initialize a global state dictionary; the global state dictionary is used to store the extracted key-value pairs. The extraction module 320 is used to obtain candidate key-value pairs extracted from the current target text block, and to verify the candidate key-value pairs based on the current global state dictionary and the target parameter field set to be extracted. Update module 330 is used to update the global state dictionary and determine the next target text block based on the verification result; Output module 340 is used to take the next target text block as the current target text block, continue to execute the process of obtaining candidate key-value pairs extracted from the current target text block until a preset termination condition is met, and output the global state dictionary under the current extraction round. When the verification result indicates missing data, the next target text block is determined based on the literature knowledge graph of the document to be processed and the chapter metadata of each text block in the text block sequence.

[0117] The information extraction device provided by this invention first divides the document to be processed into a sequence of text blocks and initializes a global state dictionary. Then, it obtains candidate key-value pairs extracted from the current target text block and verifies the candidate key-value pairs based on the current global state dictionary and the target parameter field set to be extracted. In this way, the global state dictionary realizes a global record of the extracted key-value pairs, forming a verification benchmark with historical context memory. Then, when the verification result is that the data is missing, it combines the domain prior knowledge carried by the document knowledge graph and the chapter metadata of each text block in the text block sequence to perform semantic inference on the distribution position of the missing data in the document, thereby accurately locating and jumping to the text block most likely to contain the target information. This overcomes the information fragmentation problem of the traditional sequential scanning mode and improves the accuracy of recalling related parameters scattered in different structural parts of long documents. Finally, the determined next target text block is used as the current target text block and the above process is continuously repeated until the preset termination condition is met. The global state dictionary under the current extraction round is output, thereby improving the completeness of structured information extraction from long and complex documents.

[0118] Figure 4 An example is a schematic diagram of the physical structure of an electronic device, such as... Figure 4 As shown, the electronic device may include: a processor 410, a communications interface 420, a memory 430, and a communication bus 440, wherein the processor 410, the communications interface 420, and the memory 430 communicate with each other via the communication bus 440. The processor 410 can call logical instructions in the memory 430 to execute an information extraction method, the method including: The document to be processed is segmented into a sequence of text blocks, and a global state dictionary is initialized; the global state dictionary is used to store the extracted key-value pairs. Obtain candidate key-value pairs extracted from the current target text block, and verify the candidate key-value pairs based on the current global state dictionary and the target parameter field set to be extracted; Based on the verification results, the global state dictionary is updated and the next target text block is determined; The next target text block is taken as the current target text block, and the process of obtaining candidate key-value pairs extracted from the current target text block continues until the preset termination condition is met, and the global state dictionary under the current extraction round is output. When the verification result indicates missing data, the next target text block is determined based on the literature knowledge graph of the document to be processed and the chapter metadata of each text block in the text block sequence.

[0119] Furthermore, the logical instructions in the aforementioned memory 430 can be implemented as software functional units and, when sold or used as independent products, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0120] On the other hand, the present invention also provides a computer program product, the computer program product comprising a computer program that can be stored on a non-transitory computer-readable storage medium, wherein when the computer program is executed by a processor, the computer is able to execute the information extraction methods provided by the above methods, the methods comprising: The document to be processed is segmented into a sequence of text blocks, and a global state dictionary is initialized; the global state dictionary is used to store the extracted key-value pairs. Obtain candidate key-value pairs extracted from the current target text block, and verify the candidate key-value pairs based on the current global state dictionary and the target parameter field set to be extracted; Based on the verification results, the global state dictionary is updated and the next target text block is determined; The next target text block is taken as the current target text block, and the process of obtaining candidate key-value pairs extracted from the current target text block continues until the preset termination condition is met, and the global state dictionary under the current extraction round is output. When the verification result indicates missing data, the next target text block is determined based on the literature knowledge graph of the document to be processed and the chapter metadata of each text block in the text block sequence.

[0121] In another aspect, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the information extraction methods provided by the methods described above, the methods comprising: The document to be processed is segmented into a sequence of text blocks, and a global state dictionary is initialized; the global state dictionary is used to store the extracted key-value pairs. Obtain candidate key-value pairs extracted from the current target text block, and verify the candidate key-value pairs based on the current global state dictionary and the target parameter field set to be extracted; Based on the verification results, the global state dictionary is updated and the next target text block is determined; The next target text block is taken as the current target text block, and the process of obtaining candidate key-value pairs extracted from the current target text block continues until the preset termination condition is met, and the global state dictionary under the current extraction round is output. When the verification result indicates missing data, the next target text block is determined based on the literature knowledge graph of the document to be processed and the chapter metadata of each text block in the text block sequence.

[0122] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0123] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0124] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An information extraction method, characterized in that, The method includes: The document to be processed is segmented into a sequence of text blocks, and a global state dictionary is initialized; the global state dictionary is used to store the extracted key-value pairs. Obtain candidate key-value pairs extracted from the current target text block, and verify the candidate key-value pairs based on the current global state dictionary and the target parameter field set to be extracted; Based on the verification results, the global state dictionary is updated and the next target text block is determined; The next target text block is taken as the current target text block, and the process of obtaining candidate key-value pairs extracted from the current target text block continues until the preset termination condition is met, and the global state dictionary under the current extraction round is output. When the verification result indicates missing data, the next target text block is determined based on the literature knowledge graph of the document to be processed and the chapter metadata of each text block in the text block sequence.

2. The information extraction method according to claim 1, characterized in that, The validation of the candidate key-value pairs based on the current global state dictionary and the target parameter field set to be extracted includes: Determine the target parameter fields in the target parameter field set to be extracted that match the candidate key-value pairs; If the candidate key-value pair is missing the first parameter value corresponding to the target parameter field, then the verification result is determined to be missing data. If the first parameter value is not missing in the candidate key-value pair, but the first parameter value is inconsistent with the second parameter value corresponding to the target parameter field in the current global state dictionary, then the context feature words of the current target text block are extracted. If the context feature words do not contain the control group feature words, the verification result is determined to be a data conflict; the control group feature words include at least the feature words used to characterize different experimental conditions or different control groups; If the first parameter value is not missing in the candidate key-value pair and the second parameter value corresponding to the target parameter field does not exist in the current global state dictionary, or if the first parameter value and the second parameter value in the candidate key-value pair are inconsistent but the control group feature word exists in the context feature word of the current target text block, then the verification result is determined to be data complete and conflict-free.

3. The information extraction method according to claim 2, characterized in that, If the verification result indicates missing data, the next target text block is determined in the following way: Based on the literature knowledge graph, the target chapter type corresponding to the target parameter field is determined; Retrieve the target chapter metadata that matches the target chapter type from the chapter metadata of each text block in the text block sequence; The first text block in the text block sequence that is associated with the target chapter metadata is determined as the next target text block.

4. The information extraction method according to claim 2, characterized in that, If the verification result indicates a data conflict, the next target text block is determined in the following manner: Determine the text block associated with the second parameter value in the current global state dictionary; The next target text block is constructed by concatenating the text block associated with the second parameter value and the current target text block.

5. The information extraction method according to claim 4, characterized in that, In the case where the verification result indicates a data conflict, after taking the next target text block as the current target text block, the step of obtaining candidate key-value pairs extracted from the current target text block includes: Obtain the first parameter value and the second parameter value of the data conflict; Based on the context content of the current target text block, the first parameter value and the second parameter value are compared and inferred to determine the target parameter value; The target parameter field and the target parameter value are used as candidate key-value pairs extracted from the current target text block.

6. The information extraction method according to claim 2, characterized in that, If the verification result indicates that the data is complete and conflict-free, the next target text block is determined in the following way: The next adjacent text block in the text block sequence is determined as the next target text block.

7. The information extraction method according to claim 2, characterized in that, The step of updating the global state dictionary based on the verification result includes: If the verification result is that the data is missing, then the target parameter field and the related text block identifier are recorded in the global state dictionary, and the value of the target parameter field is marked as null. If the verification result is a data conflict, then the first parameter value, the second parameter value, and the related text block identifier corresponding to the target parameter field are associated in the global state dictionary and marked as a data conflict. If the verification result is that the data is complete and there are no conflicts, then the first parameter value corresponding to the target parameter field and the related text block identifier are associated and recorded in the global state dictionary.

8. The information extraction method according to any one of claims 1 to 7, characterized in that, The preset termination conditions include: All target parameter fields in the target parameter field set of the current global state dictionary are not empty; The current global state dictionary does not contain any fields marked as data conflicts; and All text blocks in the text block sequence have been read.

9. The information extraction method according to any one of claims 1 to 7, characterized in that, The method further includes: Obtain the global state dictionary under multiple extraction rounds; Data alignment is performed on the target parameter fields in multiple global state dictionaries to obtain the set of parameter values ​​corresponding to the target parameter fields; Based on the number of times the same parameter value appears in the set of parameter values ​​corresponding to the target parameter field, the merged parameter value and the associated confidence index corresponding to the target parameter field are determined; Based on the merge parameter value corresponding to the target parameter field, the confidence index, and the text block index, the corresponding target structured data is output.

10. An information extraction device, characterized in that, include: The initialization module is used to segment the document to be processed into a sequence of text blocks and initialize the global state dictionary; The global state dictionary is used to store the extracted key-value pairs; The extraction module is used to obtain candidate key-value pairs extracted from the current target text block, and to verify the candidate key-value pairs based on the current global state dictionary and the target parameter field set to be extracted. The update module is used to update the global state dictionary and determine the next target text block based on the verification result; The output module is used to take the next target text block as the current target text block, continue to execute the process of obtaining candidate key-value pairs extracted from the current target text block until a preset termination condition is met, and output the global state dictionary under the current extraction round. When the verification result indicates missing data, the next target text block is determined based on the literature knowledge graph of the document to be processed and the chapter metadata of each text block in the text block sequence.

11. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the information extraction method as described in any one of claims 1 to 9.

12. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the information extraction method as described in any one of claims 1 to 9.

Images