Document entity relation hierarchical extraction method based on atlas evidence coverage gap feedback

Abstract

This invention discloses a hierarchical document entity relationship extraction method based on graph evidence coverage gap feedback, comprising the following steps: After slicing the document set, an initial extraction model is used for the first round of extraction to obtain candidate knowledge units and their evidence anchors, and a candidate graph is constructed; gap nodes are generated based on the evidence strength difference, and a reverse mapping between gap nodes and document fragments is established to form an evidence coverage gap graph; based on the association between gap nodes and corresponding fragments, the coverage probability and computational cost of candidate fragment-model combinations to fill the gaps are predicted, and the coverage gain per unit cost is determined; according to the total computational budget and the coverage gain per unit cost, the current round of re-extraction combination is selected, and the enhanced extraction model is called to perform re-extraction; the supplementary knowledge units are written into the candidate graph, the set of affected gap nodes is located and incrementally updated, and if the stopping condition is not met, the next iteration begins. This invention solves the problem of difficulty in efficiently improving the quality of knowledge graph extraction under limited budget.

Description

Technical Field

[0001] This invention relates to the technical field of knowledge graph construction, and specifically to a hierarchical extraction method for document entity relationships based on feedback from knowledge graph evidence coverage gaps. Background Technology

[0002] With the deepening of the digital transformation of industry documents, knowledge graphs have become the core foundation for scenarios such as intelligent retrieval, compliance risk control, and business decision-making. Accurately extracting entities, relationships, and their evidence anchors from unstructured documents is the core task of knowledge graph construction.

[0003] Existing document entity relation extraction techniques mainly fall into two categories. One is a full-scale extraction scheme based on a single model, which extracts information from all documents in a single call to the extraction model. This method is simple to implement, but in large-scale document scenarios, the computational cost is prohibitively high if a high-precision model is used, while the extraction quality is difficult to meet standards if a lightweight model is used. The other is a multi-stage sampling and verification scheme, which first completes the initial extraction using a basic model, and then extracts low-confidence segments at a fixed ratio for manual or high-precision model verification. While this method can improve accuracy to some extent, the sampling strategy is disconnected from the actual gaps in the document graph. A large number of qualified segments consume verification resources, while truly high-value gaps cannot be prioritized for filling.

[0004] Meanwhile, existing extraction schemes generally lack closed-loop feedback and incremental update mechanisms. Each round of quality optimization typically requires rescanning all documents and all graph nodes, with iteration costs increasing linearly with data size, making efficient continuous iteration impossible. Furthermore, existing schemes mostly use fixed strategies for extraction and verification, making it difficult to dynamically adjust extraction priorities based on actual graph coverage gaps, thus hindering the continuous and efficient improvement of graph quality under limited computing resource constraints.

[0005] Therefore, how to leverage the gaps in knowledge graph evidence coverage to construct a hierarchical, budget-controllable, and incrementally iterative entity relationship extraction technology system, and efficiently improve the overall coverage quality of knowledge graphs with limited resources, is a technical problem that urgently needs to be solved in this field. Summary of the Invention

[0006] This invention provides a hierarchical document entity relationship extraction method based on graph evidence coverage gap feedback to solve the aforementioned technical problems, specifically adopting the following technical solution: A hierarchical extraction method for document entity relations based on graph evidence coverage gap feedback includes the following steps: S1. Slice the document set, use the initial extraction model to perform the first round of extraction on each segment, obtain candidate knowledge units and their evidence anchors in the original documents, and construct candidate graphs accordingly. S2. Generate gap nodes based on the difference between the evidence strength of knowledge units in the candidate graph and the target evidence strength, establish a reverse mapping relationship between each gap node and the document fragment that can fill the gap, and form an evidence coverage gap graph. S3. Based on the association between the types of gap nodes in the evidence coverage gap graph and the corresponding document fragments, predict the coverage probability and computational cost of the document fragments and the candidate fragment-model combinations formed by each augmentation extraction model to fill the corresponding gap nodes, and determine the coverage gain per unit cost accordingly. S4. Based on the total budget, select the re-extraction combination for this round from the candidate fragment-model combination according to the unit cost coverage gain, and call the corresponding enhanced extraction model to perform re-extraction to obtain supplementary candidate knowledge units and their evidence anchors. S5. Write the supplemented candidate knowledge units into the candidate graph. Based on the results of this round of re-extraction, locate the set of gap nodes affected in the evidence coverage gap graph. Only perform incremental updates on the coverage status of this set. If the updated overall coverage status containing multi-dimensional indicators does not meet the stopping condition, then proceed to the next round S3 with the updated candidate graph and evidence coverage gap graph.

[0007] Furthermore, the validity of the evidence anchors for candidate knowledge units is verified: When all verification items pass, the document is marked as valid and included in the strength of evidence calculation. When a relationship check lacks a clear trigger word and the head and tail entities and table structure can be confirmed, it is marked as weakly valid. After reducing the first-round calibration confidence of the candidate knowledge unit by a set ratio, it enters the candidate graph. If any of the range verification, readback verification, version verification, or entity verification fails, it is marked as invalid, not included in the evidence strength calculation, and enters the relocation queue, waiting to be re-entered into the candidate map after subsequent relocation and repair.

[0008] Furthermore, gap nodes include at least the following types: When the number of confirmed entity types in the target document fragment is lower than the preset target number in the candidate graph, an entity type gap node is generated; When the entity type satisfies the preset pattern compatibility constraint, there is relationship indication information, and the evidence strength of the current relationship is lower than the target evidence strength, a relationship pattern gap node is generated. When all the verified and valid evidence anchors corresponding to the candidate relationships in the candidate graph originate from the same independent group, and the number of independent groups does not reach the preset minimum number, a single-source gap node is generated. When multiple candidate relation contents under the same function key are incompatible, a conflict gap node is generated based on the conflict intensity of the multiple candidate relations. The function key is determined by at least one of the head entity, relation type and effective time range.

[0009] Furthermore, the source independent groups are generated in the following manner: After normalizing the documents, calculate the content hash value, locality-sensitive hash signature, and text vector for each document, and extract the reference links; Candidate recall is based on at least one of the following four criteria: identical content hash value, matching value of local sensitive hash signature, similarity of text vectors exceeding the recall threshold, or existence of a reference relationship. Based on text similarity, publication time difference, or core fact coverage, the recalled document pairs are determined to be from the same source, and documents that meet the criteria are grouped into independent groups from the same source. When merging, the new document must have a similarity to the cluster center text that is not lower than the merging threshold, and must satisfy the same source determination with a preset number of documents in the cluster; otherwise, merging is prohibited.

[0010] Furthermore, the reverse mapping relationship is established as follows: The first search entry point is the segment containing verified and valid evidence concerning the entities or relationships involved in the gap node; The second search entry is at least one of the following: a fragment containing the name of the entity involved in the gap node, a fragment containing the relationship trigger word involved in the gap node, a fragment adjacent to the fragment in the first search entry, or a fragment associated with the neighboring entity of the entity involved in the gap node. The document fragments retrieved by the first and second search entries are associated with the gap node.

[0011] Furthermore, the coverage probability and computational cost are determined in the following way: To enhance the extraction model, a capability profile is established for each gap node type. The capability profile is used to reflect the success rate of the enhanced extraction model in filling gaps in the corresponding types. For any candidate fragment-model combination and the corresponding gap node, based on the capability profile and the entity hit score, relation hit score, position score, conflict features, source novelty and first-round uncertainty of the candidate fragment relative to the gap node, predict the probability of the combination filling the gap node coverage. The computational cost required for this combination is predicted based on the input length of the candidate fragment, the expected output length, and the number of candidate knowledge units it contains.

[0012] Furthermore, the gain per unit cost is determined as follows: The independence reduction coefficient of the candidate fragment is determined based on the text similarity between the candidate fragment and each selected fragment belonging to the same independent group in the selected fragment-model combination set. The selected fragment-model combination set is the set of candidate fragment-model combinations that have been selected for execution in the current round. Based on the current coverage, coverage probability, independence reduction coefficient, and target evidence strength of each gap node, the updated coverage of each gap node after adding the candidate fragment-model combination is determined. The weighted increment of the coverage of each gap node before and after adding the candidate fragment-model combination is used as the marginal coverage gain of the combination. Based on the marginal coverage gain and the computational cost, the unit cost coverage gain is determined. The unit cost coverage gain is used to measure the gap coverage improvement corresponding to the unit computational cost.

[0013] Furthermore, based on the total budget, the current round of re-extraction combinations is selected from the candidate fragment-model combinations according to the gain covered by unit cost, including: The total budget is divided into a mandatory budget and a general budget. The mandatory budget is used to meet the minimum coverage requirement of the mandatory gap, while the general budget is used to make up for the remaining gaps. Within the mandatory budget, candidate fragment-model combinations that fill the mandatory gap are selected first, and then candidate fragment-model combinations are selected in descending order of gain coverage per unit cost within the normal budget. The selection process also satisfies the following conditions: the same candidate fragment corresponds to only one augmented extraction model per round; single-source gap nodes prioritize candidate fragments from independent groups of new sources; conflicting gap nodes retain candidate fragments from independent groups of different sources; and when a forced gap does not reach the minimum coverage, ordinary gaps are prohibited from occupying their reserved budget.

[0014] Furthermore, the selection of combinations for this round of re-sampling adopts a greedy approximation method, including: A max-heap is constructed by covering the upper bound of the gain with the unit cost of each candidate fragment-model combination. After popping the top of the heap in each round, the true gain of the combination under the selected fragment-model combination set is recalculated. If it is still not lower than the upper bound of the next top of the heap, it is selected; otherwise, it is put back into the heap. When selected, only the gain of other combinations with the same notch associated with this combination will be recalculated.

[0015] Furthermore, incremental updates include: The supplementary candidate knowledge units are aligned with the existing candidate knowledge units in the candidate graph. The fusion confidence of the same candidate knowledge unit is determined by a combination of the confidence of all its supporting evidence, source reliability, and source independence coefficient. Conflicting relationships retain multiple candidate values ​​and their chains of evidence. A candidate relationship is marked as resolved only when its fusion confidence exceeds that of other candidate relationships by a set margin and the number of independent groups from the source meets the requirement. Determine the set of gap nodes affected by the results of this round of re-extraction, and recalculate only the coverage status of this set. This set includes gap nodes directly associated with the re-extracted fragments, entity type gap nodes corresponding to newly added or merged entities, relational pattern gap nodes affected by changes in the head and tail entities, and conflicting gap nodes sharing the same function key.

[0016] Furthermore, the stopping conditions include: The overall weighted coverage rate reaches the preset target value and the unresolved conflict rate does not exceed the preset threshold. The cumulative actual computational costs have reached the budget limit; The increase in overall weighted coverage rate over multiple consecutive rounds has been lower than the dynamically determined minimum increase threshold; The unit cost coverage gain of all available candidate fragment-model combinations is below the stopping threshold; The maximum processing time limit stipulated by the business has been reached.

[0017] The advantages of this invention lie in the provision of a hierarchical document entity relationship extraction method based on graph evidence coverage gap feedback. This method uses graph evidence coverage gaps as the core driver to achieve hierarchical extraction, replacing the traditional full-scale extraction approach. This fundamentally reduces redundant computational overhead and improves overall execution efficiency. Furthermore, by accurately anchoring knowledge completion targets through the evidence coverage gap graph, it avoids indiscriminate full-document traversal processing, effectively improving the targeting of gap completion. Based on this, quantitative selection is achieved using the unit cost coverage gain of candidate fragment-model combinations, driving computational resources towards high-value completion tasks and significantly improving resource utilization efficiency. Simultaneously, an incremental update mechanism is used to complete graph state iteration, combined with multi-dimensional stopping conditions to form a controllable convergent closed-loop system. This allows for continuous optimization of graph coverage and conflict resolution under limited resource constraints, achieving the optimal balance between knowledge graph construction quality and operational efficiency. Attached Figure Description

[0018] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a schematic diagram of the document entity relationship hierarchical extraction method based on graph evidence coverage gap feedback of the present invention; Detailed Implementation

[0020] Embodiments of the present invention are described in detail below. Examples of these embodiments are illustrated in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention.

[0021] like Figure 1 The method for hierarchical extraction of document entity relations based on graph evidence coverage gap feedback, as described in this application, includes the following steps: S1. Slice the document set, use the initial extraction model to perform the first round of extraction on each segment, obtain candidate knowledge units and their evidence anchors in the original documents, and construct candidate graphs accordingly. S2. Generate gap nodes based on the difference between the evidence strength of knowledge units in the candidate graph and the target evidence strength, establish a reverse mapping relationship between each gap node and the document fragment that can fill the gap, and form an evidence coverage gap graph. S3. Based on the association between the types of gap nodes in the evidence coverage gap graph and the corresponding document fragments, predict the coverage probability and computational cost of the document fragments and the candidate fragment-model combinations formed by each augmentation extraction model to fill the corresponding gap nodes, and determine the coverage gain per unit cost accordingly. S4. Based on the total budget, select the re-extraction combination for this round from the candidate fragment-model combination according to the unit cost coverage gain, and call the corresponding enhanced extraction model to perform re-extraction to obtain supplementary candidate knowledge units and their evidence anchors. S5. Write the supplemented candidate knowledge units into the candidate graph. Based on the results of this round of re-extraction, locate the set of gap nodes affected in the evidence coverage gap graph. Only perform incremental updates on the coverage status of this set. If the updated overall coverage status containing multi-dimensional indicators does not meet the stopping condition, then proceed to the next round S3 with the updated candidate graph and evidence coverage gap graph.

[0022] This invention provides a hierarchical extraction method for document entity relationships based on graph evidence coverage gap feedback. Addressing the challenge of balancing extraction quality and computational efficiency in large-scale document knowledge graph construction, this method uses the evidence coverage status of the knowledge graph as the core control criterion. It constructs a hierarchical extraction mechanism combining lightweight initial extraction (achieved through an initial extraction model) with high-precision targeted completion and quality improvement (achieved through an enhanced extraction model), forming a closed-loop operation process that can iteratively converge and incrementally optimize. The method sequentially includes: constructing candidate graphs through initial extraction; generating evidence coverage gap graphs and establishing reverse mappings from fragments to gaps; calculating the unit cost coverage gain of candidate fragments combined with the enhanced model; optimizing the combination of re-extraction tasks under the constraint of the total computational budget; and writing the results back to the graph and incrementally updating the gaps. This gap feedback continuously guides resources towards high-value evidence completion tasks, steadily improving the overall quality of the graph within a limited cost. The specific implementation methods for each step are as follows.

[0023] For step S1, the document set is sliced, and the initial extraction model is used to perform the first round of extraction on each slice to obtain candidate knowledge units and their evidence anchors in the original documents, and a candidate graph is constructed accordingly.

[0024] As a preferred implementation, the method performs validity verification on the evidence anchor points of candidate knowledge units, including: marking the evidence anchor point as valid when all verification items pass and including it in the evidence strength calculation; marking it as weakly valid when only the relationship verification lacks a clear trigger word and the head and tail entities and table structure can be confirmed, reducing the first-round calibration confidence of the candidate knowledge unit by a set ratio before entering the candidate graph; marking it as invalid when any of the range verification, readback verification, version verification, or entity verification fails, not including it in the evidence strength calculation, and entering the relocation queue, to be re-entered into the candidate graph after subsequent relocation and repair.

[0025] Specifically, before performing the slicing operation, the input document set is first subjected to unified standardization processing, converting documents of different formats such as PDF, DOCX, HTML, TXT, and scanned text into unified structured text, fully preserving document identifiers, heading levels, paragraph numbers, table row and column positions, and page number information, providing a complete coordinate system for subsequent fragment location and evidence tracing.

[0026] After normalization, stable slicing is performed on the document set. The target length of the slices is adapted to the context window constraints of the subsequent augmented extraction model. This augmented extraction model is a high-precision semantic extraction model for knowledge graph gap completion, used to perform fine-grained entity recognition, relation extraction, and evidence location on the candidate fragments obtained from the slices, outputting structured candidate knowledge units that can be used to fill gap nodes. Let the maximum allowed context length of the augmented extraction model be W. mRead from the model deployment configuration, the proportion of prompt words and reserved outputs is ρ. p Then the target length L of the fragment f Determined according to the following formula: , Among them, L max The maximum fragment length allowed by the system is configured by the storage and retrieval services, ρ c A safety factor is used for the context, with a value ranging from [0.6, 0.9], to avoid exceeding the model context. This indicates rounding down. This slice length calculation method can dynamically adjust the granularity of the slice by combining the contextual capabilities of the enhanced extraction model. While ensuring that a single model call can fully carry the content of the slice, it can maximize the information density of a single slice, reduce unnecessary slice splitting, and reserve sufficient prompt words and output space to avoid result truncation due to context overflow during inference.

[0027] The above prompts and output reserve ratio ρ p Based on the actual word segmentation statistics of the model, for the same prompt template, the number of prompt word tokens T is counted on no less than 200 samples extracted stratified by document type, length, and table proportion. prompt And the actual number of output tokens T output And take the 95th percentile, while setting a safety margin T. margin : , final ρ p The calculation formula is: , Among them, C model To enhance the length of the extraction model context window, Q95 represents the 95th percentile. This percentage is updated every 1000 calls, and if the change is less than 0.02 for two consecutive calls, the original value is maintained. At the same time, percentile statistics and rolling updates can cover the output requirements of most scenarios without excessively occupying the input context space.

[0028] To reduce the probability of semantic units such as sentences or table rows being sliced, overlapping regions are set between adjacent segments, with a segment overlap length of O. f for: , in, The 95th percentile of sentence length in similar documents is obtained from historical document statistics, and ρ0 is the overlap ratio, which can be initially set to 0.1 to 0.2. Setting the sentence length corresponding to the 95th percentile as the lower limit of overlap length can adapt to most cross-segment sentence scenarios and effectively reduce the occurrence of semantic units being sliced ​​and truncated.

[0029] The overlap ratio ρ0 can be further optimized using a cross-boundary relationship validation set. Construct at least 500 labeled samples containing cross-sentence, cross-paragraph, cross-page, or cross-table relationships. With a fixed segment target length, slice and extract candidate overlap ratios ρ∈{0.05,0.10,...,0.30}, and calculate the corresponding cross-segment relationship recall R. cross (ρ) and the proportion of duplicate tokens D dup (ρ).

[0030] Among them, the cross-fragment relationship recall rate R cross (ρ)=N hit (ρ) / N cross N cross N represents the total number of relations in the cross-boundary relation validation set. hit (ρ) represents the number of cross-boundary relationships that are fully preserved and correctly extracted under the corresponding overlap ratio. D represents the duplicate token ratio. dup (ρ)=[T all (ρ)-T unique ] / T all (ρ), T all (ρ) represents the total number of tokens actually fed into the model after slicing, T unique This represents the number of unique tokens after deduplication based on their original position. All the above metrics are based on the same validation set, extracted from different tiers of slice results, and directly calculated through token statistics. This directly reflects the recall gain brought about by increasing the overlap ratio, as well as the additional computational overhead that comes with it.

[0031] The optimization process uses the lowest acceptable recall rate across boundary relationships as the objective, and denotes this objective as R. target Its value is determined based on downstream business requirements and the results of the complete document benchmark extraction. Let the minimum recall rate specified by the business be R. biz The recall rate is R when the same validation set is not sliced ​​or when sufficient context is used for complete extraction. full Then R target =max(R biz 0.95R full When R is not set biz Take 0.95R full In the cold start phase, when there is no complete document baseline, 0.90 can be temporarily used. After the cumulative number of cross-boundary relationships marked is no less than 500, the target value can be recalculated by substituting it into the above formula.

[0032] Based on this, two tuning parameters, ε1 and λ, are introduced. Parameter ε1 represents the threshold for the incremental recall rate across segments corresponding to the overlap ratio between two adjacent segments. It is used to determine whether further increasing overlap still yields effective returns. The default value of 0.005 indicates that a recall rate increase of less than 0.5 percentage points is considered to be stabilizing. This value can be adjusted according to the minimum recall change allowed by the business. Parameter λ represents the cost weight of duplicate tokens, used to evaluate the comprehensive indicator J(ρ)=R. cross (ρ)-λD dup (ρ), whose value is determined by the actual GPU time or model call cost on the validation set, is selected through a grid search on λ∈{0.01, 0.02, ..., 1.00}, making R... cross (ρ)≥R target Furthermore, λ, which has the lowest total inference cost, can be initially set to 0.10 when historical performance data is lacking.

[0033] The final choice satisfies R cross (ρ)≥R target Furthermore, the minimum ρ between two adjacent recall increments is less than ε1, and when multiple candidates are tied, the proportion with the largest comprehensive index J(ρ) is selected. Relying on this verification and optimization mechanism, the additional computational overhead introduced by duplicate tokens can be controlled within a reasonable range while ensuring cross-segment relationship recall capabilities, achieving a balance between recall effectiveness and computational cost. Each segment generated records at least the document identifier, segment identifier, source identifier, chapter path, start character position, end character position, page number, content hash value, token quantity, and adjacent segment identifiers; the content hash value is used to identify whether the document content has changed, providing a basis for subsequent incremental updates.

[0034] After document slicing, an initial extraction model is used to jointly extract entities, relations, and evidence locations for each segment. This initial extraction model can be a lightweight model with fewer parameters, supporting quantization and deployment, or customized for specific tasks. Compared to augmented extraction models, it has lower inference latency, less memory usage, and lower single-token call cost. It can complete the first round of full processing of massive segments with very low computational overhead, ensuring basic extraction recall while avoiding the waste of computational resources caused by directly calling the augmented extraction model. Each extracted candidate knowledge unit includes at least the entity name, entity type, and the start and end positions of the entity in the segment, including the head entity, relation type, tail entity, sentence, table cell, or paragraph position directly supporting the entity or relation, the first-round calibration confidence score, document source, and model version.

[0035] To uniformly measure the reliability of different extraction results, the system calculates the first-round calibration confidence score for each candidate knowledge unit. The calibration confidence score c of candidate knowledge unit z is... zCalculate using the following formula: c z =σ(a0+a1p span +a2p type +a3p rel +a4q anchor ), In the formula, p span p represents the probability of entity boundary recognition. type p represents the probability of entity type recognition. rel q represents the probability of relation type identification. anchor To output the consistency score between the relationship and the original evidence location, a0 is the confidence calibration bias term, a1 is the calibration weight coefficient corresponding to the entity boundary recognition probability, a2 is the calibration weight coefficient corresponding to the entity type recognition probability, a3 is the calibration weight coefficient corresponding to the relationship type recognition probability, and a4 is the calibration weight coefficient corresponding to the evidence location consistency score.

[0036] The initial extraction model described above can be divided into two implementation forms: classification model and generative model. Under the two implementation forms, the calculation logic of each recognition probability is different.

[0037] The above entity boundary recognition probability p span The calculation formula is: , Among them, P start (i) and P end (j) represent the probabilities that the initial extraction model classifies positions i and j as the start and end points of an entity, respectively. valid (i,j) is an entity boundary validity indicator function, which takes the value 1 when the start point is no later than the end point, the span does not exceed the preset maximum entity length, and it does not cross the prohibited structure boundary; otherwise, it takes the value 0. In this way, the classification model can directly output the start and end probabilities of each position, while the generative model, which does not directly output the position probability, can use the boundary consistency rate of the same entity in multiple controlled decodings instead.

[0038] The above entity type recognition probability p type This represents the normalized probability of the initial extraction model for the final entity type. Classification models directly read the Softmax probability, while generative models normalize it according to the logarithmic probability of the type label. When multiple label types exist, p... type The probability of a type being written into a candidate graph is taken, and if the type does not satisfy the domain pattern constraint, it is multiplied by the type constraint penalty coefficient, which ranges from 0 to 1.

[0039] The above relationship type identification probability p relGiven the head entity, tail entity, and evidence fragment, this represents the normalized conditional probability of the relation type. The classification model uses the relation classification Softmax probability, while the generative model uses the average token probability of the relation label sequence, multiplied by the head-tail direction consistency indicator.

[0040] Evidence anchor consistency score q anchor The calculation formula is: q anchor =0.30I head +0.30I tail +0.20I trigger +0.10I order +0.10I scope , Among them, I head I tail These respectively indicate whether the head entity and tail entity can be precisely or aliasedly matched and located within the anchor point range, I trigger Indicates whether a relation trigger word, table column name, or syntactic join is within the scope of the anchor point, I order This indicates that the direction of the beginning and end is consistent with the semantic direction of the original text. scope This indicates that the entity and trigger word are located in the same sentence, the same table row, or adjacent sentence windows. Each component ranges from 0 to 1; q indicates that the head or tail entity cannot be located. anchor Set to 0.

[0041] This confidence calibration mechanism integrates multi-dimensional confidence features such as entity boundaries, entity types, relationship types, and evidence anchors. Compared with the single-value confidence score output by the model, it can more accurately reflect the actual reliability of the extraction results, providing a more reliable quantitative basis for subsequent gap calculation and re-extraction scheduling.

[0042] The calibration coefficients are obtained by fitting a labeled validation set. The validation set used to fit the calibration coefficients is stratified and sampled according to document source, document type, entity type, relation type, fragment length, and extraction difficulty. The samples cover various scenarios, including normal positive examples, examples with ambiguous entity boundaries, examples with easily confused types, examples with incorrect relation directions, examples without evidence anchors, examples of negated relations, and examples of cross-fragment relations. Specifically, the calibration coefficients include a confidence calibration bias term a0, a calibration weight coefficient a1 corresponding to the entity boundary recognition probability, a calibration weight coefficient a2 corresponding to the entity type recognition probability, a calibration weight coefficient a3 corresponding to the relation type recognition probability, and a calibration weight coefficient a4 corresponding to the evidence position consistency score. In principle, each high-frequency entity or relation category should have no less than 50 positive examples and 50 negative examples, and rare categories should have no less than 20 positive examples. All samples are independently labeled by two annotators, and inconsistent samples are adjudicated by a third party. Therefore, the training set, calibration set, and test set are independently divided according to source and time, with ratios of 60%, 20%, and 20%, respectively, to prevent text leakage from the same source or adjacent time periods.

[0043] When fitting logistic regression, first set p span p type p rel and q anchor Normalized to 0 to 1 and regularized using the L2 norm. The intercept a0 is initialized to a0=ln(π / (1-π)) based on the basic accuracy π of the candidate knowledge units in the validation set. a1 to a4 are then obtained by maximum likelihood estimation. After source-independent group segmentation, the fitting results can reflect the cross-source generalization ability and ensure the universality of confidence calibration.

[0044] When an independent validation set is lacking, an equal-weight initialization strategy is adopted. This strategy does not directly cause the system's initial performance to collapse because the equal-weighted results are only used as candidate confidence ranking values ​​and are simultaneously constrained by the validity of evidence anchors, domain patterns, and acceptance thresholds. During cold start, a1=a2=a3=a4=0.25, a0 is calculated based on a conservative baseline accuracy π0, and when π0 is unknown, it is taken as 0.5, corresponding to a0=0. At the same time, the system raises the direct entry threshold to 0.85, and invalid anchor results are not allowed to be directly included in the evidence strength calculation. Results in the range of 0.50 to 0.85 only enter the candidate graph. After accumulating no less than 200 manually reviewed results, the coefficients are refitted, and after accumulating 500 results, they are switched to formal calibration parameters. Therefore, this phased mechanism ensures that the initial error only manifests as an increase in candidates, without causing a large number of erroneous relationships to enter the final graph, thus balancing recall and precision during the cold start phase.

[0045] The confidence calculations and subsequent verifications described above are all based on standardized evidence anchors. An evidence anchor is a verifiable location record between a candidate entity or relation and the smallest sufficient evidence region in the original document. Its standard fields include at least: doc_id (document identifier), fragment_id (fragment identifier), page_no (page number), section_path (section path), paragraph_id (paragraph identifier), table_cell (table row and column coordinates, optional), char_start and char_end (start and end positions of normalized text characters), sentence_id (sentence identifier), head_span (head entity position), trigger_span (relation trigger word or table field position), tail_span (tail entity position), quote_text (original evidence text), quote_hash (original evidence text hash), document_hash (document version hash), and anchor_version (anchor format version). Through these multi-dimensional standardized anchor fields, precise backtracking from the extracted results to specific locations in the original document is possible, providing a clear basis for subsequent item-by-item validity verification.

[0046] The validity verification of evidence anchors encompasses seven aspects: range verification, readback verification, entity verification, relation verification, scope verification, version verification, and OCR verification. Range verification ensures the start and end positions are within the corresponding segment, with the start position preceding the end position, guaranteeing the anchor's physical validity from a coordinate perspective. Readback verification then verifies that the text read back at each position is completely consistent with the normalized original evidence text or has the same hash, ensuring the accuracy of the anchor's correspondence to the original text from a content perspective. Entity verification further verifies that the beginning and end entity texts or registered aliases must be within the corresponding entity location range, ensuring the correspondence between entities and anchor positions. Relationship verification verifies that at least one relation basis—trigger words, syntactic connections, table column names, or structured fields—is present, ensuring that relation extraction is supported by the original text. Scope verification limits the scope semantically, verifying that the beginning and end entities and relation basis are within the same sentence, table row, or adjacent sentence windows allowed by domain rules, ensuring the semantic relevance of the evidence. Version verification verifies that the document version hash is consistent with the current document version, preventing anchor position shifts after document updates. For scanned and recognized documents, an additional OCR verification is set to ensure that the average OCR confidence of the anchor point area is not less than 0.80, thus guaranteeing the text quality of the anchor points in scanned documents.

[0047] When all validation items pass, the document is marked as valid and included in the evidence strength calculation, where evidence strength measures the degree to which a candidate knowledge unit is supported by valid evidence. If only the relation validation lacks a clear trigger word, but the head and tail entities and table structure can be confirmed, it is marked as weakly valid, and its initial calibration confidence is reduced by 0.5 to 0.8 before entering the candidate graph. If any of the range validation, readback validation, version validation, or entity validation fails, it is marked as invalid, not included in the evidence strength calculation, and enters the relocation queue, awaiting subsequent relocation and repair before re-entering the candidate graph. This multi-dimensional anchor point validity validation filters out low-quality evidence from the source, preventing it from interfering with subsequent evidence strength calculations and gap determination, while the weakly valid reduction mechanism balances evidence quality and candidate recall.

[0048] After extraction and anchor point validity verification, the system writes entities into the candidate graph after name normalization and alias alignment. Relationship edges simultaneously store corresponding evidence records. Each evidence record includes at least the original fragment identifier, original text location, source independent group, extraction confidence level, model identifier, model version, and extraction round. Binding evidence records to relationship edges ensures complete traceability for every piece of knowledge in the graph, facilitating subsequent evidence verification, conflict resolution, and compliance auditing.

[0049] The aforementioned source independent groups are generated as follows: After normalizing the documents, the content hash value, local sensitive hash signature, and text vector of each document are calculated, and the reference links are extracted; candidate recall is performed based on at least one of the following four criteria: identical content hash value, matching local sensitive hash signature, text vector similarity exceeding the recall threshold, or the existence of a reference relationship; based on text similarity, publication time difference, or core fact coverage, the recalled document pairs are judged to be from the same source, and documents that meet the conditions are assigned to the same source independent group; when assigning, the new document must have a text similarity with the cluster center not lower than the merging threshold, and must meet the same source judgment with a preset number of documents within the cluster, otherwise merging is prohibited.

[0050] Specifically, the source independence group is used to identify whether the evidence is truly independent, and to avoid miscounting repeated reposts of the same content as multi-source independent evidence. Its generation follows a complete process of document normalization, candidate recall, pairwise determination, connectivity clustering, and intra-cluster verification. The specific implementation process is as follows.

[0051] First, the documents undergo a template-free preprocessing step, removing common template content such as navigation bars, copyright notices, and fixed headers and footers one by one. Simultaneously, the use of full-width and half-width characters, number formats, date formats, and whitespace characters are standardized, while core information such as the main text, title, author, publication date, publishing organization, original link, and cited links are fully preserved. This eliminates the interference of format differences on same-origin determination, providing a clean text feature foundation for subsequent judgments. After preprocessing, multi-dimensional features are calculated for the main text of each document, including precise content hashing, locality-sensitive hash signatures, and text vectors, to construct a multi-dimensional feature system for same-origin determination. Precise content hashing is used to accurately match identical document content; locality-sensitive hash signatures include two types: 64-bit SimHash and 5-gramMinHash signatures, used to quickly measure text similarity between documents; and text vectors are used to calculate the similarity of documents at the semantic level.

[0052] After feature calculation, the process enters the candidate recall stage. This stage is used to quickly filter document pairs that may have a common source relationship, avoiding the waste of computational power caused by full pairwise comparisons. A document pair is sent to the subsequent pairwise determination stage if it meets any of the following five conditions: 1) identical exact content hashes; 2) Hamming distance of 64-bit SimHash not greater than 3; 3) Jaccard similarity estimated by 5-gramMinHash not less than 0.80; 4) cosine similarity of text vectors not less than 0.88; 5) direct reprint, citation, or normalized original link relationship between the documents. Simultaneously, the candidate recall stage uses source domain, publication time window, and local sensitive hash buckets to build a multi-level index, eliminating the need for pairwise comparisons of all documents and significantly improving the processing efficiency of common source identification.

[0053] After candidate recall is completed, the pairwise determination stage begins, where refined homology determination is performed on candidate document pairs. A document pair is considered a homologous document if it meets one of the following two conditions: first, the 5-gram Jaccard similarity of the main text is not less than 0.85 and the cosine similarity of the text vectors is not less than 0.92, and the publication time difference does not exceed 72 hours; second, there is a clear reprint or citation chain and the coverage rate of core fact sentences is not less than 0.80. Meanwhile, document pairs with identical exact content hashes do not require additional determination and are directly identified as homologous. The above determination thresholds are determined through grid search using manually labeled samples, aiming to maximize the F1 score for homology identification and ensure a false merging rate of less than 2% for independent source groups, thereby guaranteeing the accuracy of homology determination.

[0054] After pairwise identification, the process moves to the connectivity clustering stage. In this stage, individual documents are used as nodes, and the same-origin identification results are used as edges. A disjoint-set data structure (DFS) algorithm is employed to form connected components, initially dividing the data into independent source groups. To suppress chain-like erroneous merging, intra-cluster verification is performed before merging each new document: when assigned to the same independent source group, the new document must have a cosine similarity of at least 0.88 with the cluster center text, and must meet the pairwise identification threshold with at least 60% of the documents within the cluster. If these requirements are not met, merging is not performed, and a new independent source group is created. Furthermore, documents with links to a common original document are preferentially assigned to the group containing the corresponding original document, further improving the accuracy of clustering.

[0055] After clustering, the system generates a unique source independence group identifier for each source independence group and simultaneously saves the representative documents within the group, a list of member documents, the basis for the group's formation, intra-group similarity, citation chain relationships, and threshold version information. For documents describing the same fact but with independent wording, interview sources, and original materials, even if the extracted entities and relationships are identical, they are not grouped into the same source independence group, thus avoiding misjudging factual consistency as source independence. This refined division into source independence groups effectively distinguishes between independent reports and duplicate reprints, preventing the repeated counting of evidence from the same source and ensuring the accuracy of subsequent evidence strength calculations.

[0056] Step S1 above ultimately outputs the candidate map for round 0, the fragment library, and the evidence location index, providing a complete data foundation for subsequent gap generation and reverse mapping. The standardized evidence anchor point design is a core prerequisite for achieving minimal fragment re-extraction, enabling the system to locate specific evidence fragments without re-searching the entire document. The lightweight and low-cost characteristics of the initial extraction model make full-scale first-round extraction feasible in scenarios with massive amounts of documents, significantly reducing the computational cost of basic knowledge coverage and laying a dual foundation of data and cost for the subsequent closed-loop feedback mechanism.

[0057] For step S2, gap nodes are generated based on the difference between the evidence strength of knowledge units in the candidate graph and the target evidence strength. A reverse mapping relationship is established between each gap node and the document fragment that can fill the gap, forming an evidence coverage gap graph.

[0058] In this step, the gap nodes include at least the following types: When the number of confirmed entity types in the target document fragment is lower than the preset target number, an entity type gap node is generated; when the entity type meets the preset pattern compatibility constraints, has relationship indication information, and the evidence strength of the current relationship is lower than the target evidence strength, a relationship pattern gap node is generated; when the verified valid evidence anchors corresponding to the candidate relationships in the candidate graph all originate from the same source independent group, and the number of source independent groups does not reach the preset minimum number, a single source gap node is generated; when the contents of multiple candidate relationships under the same function key are incompatible with each other, a conflict gap node is generated according to the conflict strength of the multiple candidate relationships, and the function key is determined by at least one of the head entity, relationship type, and valid time range.

[0059] Furthermore, the reverse mapping relationship is established as follows: the first search entry is the fragment containing the verified evidence of the entity or relationship involved in the gap node; the second search entry is at least one of the following: the fragment containing the name of the entity involved in the gap node, the fragment containing the trigger word of the relationship involved in the gap node, the adjacent fragment of the fragment in the first search entry, or the fragment associated with the neighboring entity of the entity involved in the gap node; the document fragments retrieved by the first search entry and the second search entry are associated with the gap node.

[0060] Specifically, before generating gap nodes and constructing the evidence coverage gap graph, it is first necessary to quantify the evidence strength of each knowledge unit in the candidate graph, using this as a unified basis for gap determination. For a gap node g corresponding to a candidate entity, relation, or relation pattern, the evidence strength obtained in round t is... Calculate using the following formula: , In the formula, Let e ​​be the set of evidence associated with the gap node g in round t, and let c be any piece of evidence in this set. e The first-round calibration confidence level for the evidence-corresponding extraction results is taken from the calculation results of the initial extraction phase, s e The source reliability is determined by a weighted average of the source whitelist level, historical manual verification accuracy, and document completeness, with values ​​uniformly normalized to [0,1]. e The source independence coefficient is used to distinguish the difference in contribution of independent evidence and duplicated evidence from the same source to the strength of evidence.

[0061] Among them, the source reliability s e The calculation formula is: s e =w1L white +w2A hist +w3C doc +w4T trace +w5Ffresh , In the formula, L white The source-level score is calculated as follows: 1.0 for officially disclosed or signed original documents from regulatory agencies and enterprises, 0.85 for verified professional databases and mainstream media, 0.70 for ordinary institutional websites, and 0.40 for reprints from unknown sources. This mapping is maintained by the domain administrator.

[0062] A hist The historical accuracy rate of manual verification is calculated using Beta smoothing, and the formula is as follows: A hist =(N correct +2) / (N checked +4), In the formula, N correct N represents the number of pieces of evidence from this source that have been manually verified as correct. checked This represents the total number of verified samples to avoid extreme scores from a small number of samples.

[0063] C doc Document completeness is calculated using the following formula: , In the formula, I j The six metadata items are: title, publishing entity, publication date, body, attachments or signature, and original link. Each item has a value of 1, 0.5, or 0. A title that is not empty and is not a filename or placeholder text receives 1; if it can only be recovered from the first line of the body, it receives 0.5. A publishing entity that can be uniquely identified by a signature, signature, or site entity receives 1; if it can only be inferred from the domain name, it receives 0.5. A publication date that can be parsed and is logically consistent with the source time receives 1; if only the year and month are present, it receives 0.5. A body that can be completely parsed and is not navigation, template, or garbled text receives 1; if there is obvious truncation, it receives 0.5. Attachments referenced in the body that can be actually obtained or whose signatures can be verified receive 1; if only the attachment name or signature image exists but cannot be verified, it receives 0.5. An original link that is an official, compliant link and can be traced back receives 1; if only a secondary repost link exists, it receives 0.5. If an item is indeed inapplicable based on document type, that item is removed from the denominator. The weight of each item is w. j The data was obtained by normalizing the relative increase in the error rate of evidence verification caused by the absence of a certain item from historical samples with artificial integrity labels. The six items in the cold start phase were set with equal weights.

[0064] T trace For traceability, a score of 1 is assigned if the link leads back to the original publication page or digital signature file; 0.6 is assigned if there are only secondary reposting links; and 0.3 is assigned if there are no traceable links.

[0065] F fresh For timeliness scoring, the time-sensitive relationship is expressed as exp(-Δt / τ). f The calculation is performed, with non-time-sensitive facts taking the value 1. Where τ... fThis is the time constant for the decay of evidence, with units consistent with the time interval Δt, typically measured in days. When the time remaining between the evidence and the current task reaches τ... f At that time, the timeliness score decays to e. -1 Approximately 0.368, τ f Must be greater than 0, and can be configured according to relation type within 7 to 3650 days. When historical verification data is available, fit an exponential decay curve with the evidence age Δt and "currently valid, expired" labels, and select the τ that minimizes the prediction error. f However, in the absence of historical data, the validity half-life h is given by the business and calculated according to τ. f =h / ln2 is the conversion. This parameter is not used for non-time-sensitive facts, and F fresh Fixed at 1.

[0066] w1 is the weight coefficient corresponding to the source level score, w2 is the weight coefficient corresponding to the historical manual verification accuracy rate, w3 is the weight coefficient corresponding to the document completeness, w4 is the weight coefficient corresponding to the traceability, and w5 is the weight coefficient corresponding to the timeliness score. The sum of the weights is 1, and the values ​​of each weight are determined through a data-driven approach. On a validation set with source truth values, the weights are learned with the objective of minimizing the logarithmic loss of evidence correctness prediction. When historical data is missing, the weights can be directly preset as w1=0.30, w2=0.30, w3=0.15, w4=0.15, and w5=0.10. For newly emerging sources, A hist Reusing prior means from the same source category will affect the source reliability s e A temporary upper limit of 0.80 is set, and this upper limit constraint is lifted after a total of 20 manually verified records are completed.

[0067] Furthermore, the source independence coefficient d e To suppress the ineffective gain of duplicate evidence from the same source, the first piece of evidence in an independent group from the same source is assigned a value of 1, and subsequent evidence deemed highly duplicated is assigned a value of 1 / n, where n is the number of evidence already included in that independent group. Two pieces of evidence from the same independent group from the same source are considered highly duplicated if they meet any of the following conditions: First, the normalized hashes of the original evidence texts are exactly the same; second, the 5-gram Jaccard similarity is not less than 0.85, and the cosine similarity of the text vectors is not less than 0.92; third, both pieces of evidence come from the same reprint chain, and the coverage of the core fact sentences is not less than 0.80.

[0068] For evidence fragments with fewer than 30 Chinese characters, in addition to meeting the above similarity conditions, at least three of the following must be consistent: head entity, relation type, tail entity, and event time. This reduces misjudgments caused by accidental overlap. Evidence that does not reach the high repetition threshold but still belongs to the same source group, d eA suitable value is max(0.5, 1-sim), where sim is the cosine similarity of the text vectors of the two pieces of evidence, avoiding excessive penalty for documents from the same source that contain newly added information. This method of calculating evidence strength allows multiple independent sources to generate cumulative evidence gain, while duplicate content reprinted from the same source will not proportionally increase the evidence strength.

[0069] Based on the strength of evidence, the coverage of the gap node g. and the remaining gap ratio Calculate using the following formula: , , Among them, T g To achieve the required strength of evidence for full coverage, according to T g =max(T schema T risk T hist T min Confirmed. schema The minimum evidentiary requirement stipulated by the domain pattern or business rule, when the rule requires at least N g Each piece of evidence comes from an independent source, and the lowest calibrated confidence level for a single piece of evidence is c. min The minimum source reliability is s min At that time, T schema =N g ×c min ×s min T risk This is the risk adjustment value, i.e., T. risk =T schema ×(1+κR risk ), R risk T represents the normalized value of the risk level for an entity or relationship, where κ is the risk level correction strength coefficient, initially set to 0.2 to 0.5. hist The 25th percentile of the evidence strength distribution of manually verified knowledge units of the same category at the time of acceptance is used. A lower percentile is adopted to avoid setting the target too high, which could lead to non-convergence. min As the lower limit for cold start, the existence of ordinary entities can be taken as 0.8 to 1.0, the ordinary relationship can be taken as 1.0, the single-source supplementary evidence relationship is calculated based on the minimum strength of two independent sources, and the conflict resolution gap must meet the strength requirements and also satisfy at least two independent sources.

[0070] The target evidence strength value is determined by priority. When the domain graph model explicitly specifies the minimum number of evidence pieces, that specification is followed. For strong factual relationships, functional relationships, or high-risk relationships, the requirement is set to two or more independent sources, based on business rules. If the domain model does not specify this, the 25th percentile of the evidence strength of similar verified knowledge units is used as a conservative target. For newly built systems lacking historical data, a gap of 1 can be used for ordinary entity types, a gap of 2 for single-source relationship supplementary evidence, and a conflict resolution gap requires at least two independent sources. The system will test different T values ​​on the validation set. g The corresponding graph accuracy, recall, and enhanced extraction model cost are selected based on the lowest computational cost when the business accuracy threshold is reached. The model is recalibrated every 1,000 manually verified knowledge units or when the source accuracy drift exceeds 5 percentage points.

[0071] Based on the above calculation results of evidence strength and coverage, the system generates four types of gap nodes, each corresponding to different coverage insufficiency scenarios.

[0072] The first type of gap node is the entity type gap node. For entity roles required by document type or chapter template, an entity type gap node is generated when the number of confirmed entities is lower than the preset target number. Common scenarios include the contracting parties, contract dates, and subject matter in contract documents, and equipment, defects, and handling measures in equipment maintenance records.

[0073] For essential entity types with zero hits in the initial extraction model, the system can also generate zero-evidence entity type gaps by combining minimum cardinality constraints in domain patterns, document template fields, or statistical baselines of similar high-quality documents, and generate candidate fragments using anchor-free cold-start retrieval. During retrieval, the system constructs a retrieval expression using entity type definitions, synonyms, typical values, template field names, chapter titles, and regular features. Fragments are recalled using a combination of table column name matching, weak entity identification results, semantic vector retrieval, and fragment retrieval co-occurring with identified essential roles. Top-K fragments are retained for each source group independently. After these fragments are initially extracted by the enhanced extraction model and valid evidence anchors are generated, they are then transferred to the conventional gap-fragment reverse mapping process. Therefore, this candidate set does not depend on existing evidence fragments.

[0074] Furthermore, let the preset target number of entity type q in the target document fragment u be... Its minimum cardinality constraint comes from the domain template. When there is no template, the 25th percentile of the number of entities of that type in high-quality documents of the same type is used. The number of confirmed entities in the current candidate graph that meet the confidence and anchor point requirements is [number missing]. The quantity gap ratio The calculation formula is: .

[0075] The second type of gap node is the relation schema gap node. The system does not exhaustively enumerate relations for all entity pairs; it only generates a relation schema gap node when the entity type satisfies preset schema compatibility constraints, relation indication information exists, and the evidence strength of the current relation is lower than the target evidence strength. The preset schema compatibility constraints refer to the predefined set of head entity types, tail entity types, and relation direction constraints for each relation type within the domain knowledge graph schema. For example, relation r only allows type set H. r The entity in the middle is used as the head entity and type collection. The entity in the middle is used as the tail entity; when the candidate head and tail entities satisfy the above type constraints respectively, it is determined to be pattern compatible. For symmetric relations that allow bidirectional communication, either the positive or negative direction needs to be satisfied; otherwise, it is determined to be incompatible and no relation schema gap corresponding to the relation is generated.

[0076] Candidate head entity v i Candidate tail entity v j Relationship type r and relation hint score P ijr for: P ijr =b1x schema +b2x trigger +b3x near +b4x path , Where, x schema For schema compatibility, this is used to verify whether the entity type meets the preset schema compatibility constraints. If the candidate header entity v i and candidate tail entity v j If the highest probability types satisfy the head entity type and tail entity type constraints of relation r respectively, then x schema =min(p type (v i ), p type (v j ), where p type (v i ) is the candidate head entity v i The entity type recognition probability of the head entity type of the corresponding relation r, p type (v j ) is the candidate tail entity v j The entity type recognition probability of the tail entity type in relation r. If only the candidate type satisfies the constraint, the larger value of the corresponding candidate type probability is taken; if neither the head nor tail type satisfies the constraint, the probability is taken as 0. For symmetric relations that allow bidirectional relationships, the probability is calculated for both directions and the larger value is taken.

[0077] x trigger Textual relation cues, used to measure the strength of relational indication information, are calculated using the following formula: x trigger=0.40M lex +0.35M dep +0.25M table Among them, M lex This represents the maximum matching degree for relational trigger words, synonyms, or regular expressions. Precise trigger words are assigned a value of 1, synonyms 0.8, and fuzzy matching is based on edit similarity. M table This measures the degree of match between the column names, row headings, or key-value structures of the entity's table and the relational fields. If a certain type of structure is missing, the remaining weights are renormalized.

[0078] M dep This represents the matching degree between the shortest syntactic dependency path and the relation template between the candidate head entity and the candidate tail entity. Let v be the candidate head entity of the target relation r to be determined. i With candidate tail entity v j The shortest syntactic dependency path between them is p, and the set of dependency templates formed by relation r from manually verified positive examples or domain rules is T. r For any template q, calculate: Dependency label sequence similarity S label : S label =1-Lev(L p L q ) / max(|L p |,|L q |), Consistency rate S of dependent edge direction dir : S dir =N same / max(|p|,|q|,1), Path length similarity S len : S len =exp(-||p|-|q|| / max(|q|, 1)), Then M dep =max q∈Tr (S label ×S dir ×S len ) 1 / 3 Where Lev is the edit distance, L p For the dependency label sequence of candidate path p, L q For the dependency label sequence of template path q, |L p | represents the length of the candidate path label sequence, |L q | is the length of the template path label sequence, |p| is the number of edges in the candidate syntactic dependency path p, |q| is the number of edges in the template syntactic dependency path q, and N is the length of the template path label sequence. sameTo determine the number of dependent edges with consistent orientation at corresponding positions, template T r M is derived from the shortest dependency path of the real head and tail entities in the verified relation samples. dep The value ranges from 0 to 1.

[0079] x near For proximity, the minimum structural distance is used. 1 is assigned to the same table row or the same sentence; exp(-d) is assigned to different sentences within the same paragraph. s / 3), d s The sentence interval is 0.6 × exp(-d). f / 2), d f For segment intervals; take 0 for cross-chapter intervals. If there is an explicit referential chain connecting two entities, x near The lower limit has been raised to 0.6.

[0080] x path The formula for calculating the path support in a graph is: , in, Let P(v) be a relation edge inside candidate graph path p. i ,v j r) represents v in the candidate map i to v j A set of paths of length 2 or 3 that do not contain the relation r to be judged itself. M represents the confidence level of path edge fusion. pattern (p,r) represents the support probability of this path type sequence for relation r, calculated from the co-occurrence frequency in the verified graph, and γ is the path length penalty coefficient, initially set to 0.5. When no supporting path exists, x... path =0.

[0081] b1 is the weight coefficient corresponding to pattern compatibility, b2 is the weight coefficient corresponding to text relation hints, b3 is the weight coefficient corresponding to position proximity, and b4 is the weight coefficient corresponding to graph path support. Furthermore, b1 to b4 are used for grid search on the validation set with relation recall and candidate size as objectives. When the validation set is missing, b1, b2, b3, b4, b4, b5 ...5, b4, b5, b4, b5, b5, b4, b5, b5, b4, b5, b5, b5, b4

[0082] When P ijr Exceeding the threshold τ r When the current coverage of relation r is lower than the target coverage, a relation pattern gap is generated, where the threshold τ r By selecting a validation set, the accuracy of candidate relation hints entering the evidence coverage gap map is made to reach a preset lower limit, such as 0.75.

[0083] In the specific selection, for each relation type or risk level, a set of candidate thresholds τ∈{0.05, 0.10, ..., 0.95} is scanned on the validation set, and the number N of candidate relations entering the evidence coverage gap graph is counted. cand The parameters are: precision P(τ), recall R(τ), and the estimated total cost of the augmented extraction model K(τ). To avoid the accidental overestimation of precision in small samples, the quality constraint for threshold screening does not directly use the point estimate precision P(τ), but instead uses its 95% Wilson confidence interval lower bound P. L (τ). The optimal threshold finally selected for relation type r is denoted as τ. r *, The solution formula is τ r *=argmax τ R(τ), constraints include P L (τ)≥P min N cand (τ)≤N max And K(τ)≤K max That is, among all candidate thresholds that satisfy the lower limit of candidate quality, the upper limit of candidate quantity, and the upper limit of computational budget, the value with the highest recall rate is selected, and this optimal value τ is... r *This refers to τ, which is actually used in the gap determination criteria. r If multiple candidate thresholds have the same recall rate, the higher threshold with fewer candidates should be selected. When the sample size is insufficient, the threshold can be shared by major relational categories, and Bayesian smoothing can be used.

[0084] Wherein, recall R(τ) = TP(τ) / [TP(τ) + FN(τ)], precision P(τ) = TP(τ) / [TP(τ) + FP(τ)], where TP(τ) is the number of true positive samples corresponding to the candidate threshold τ, referring to the number of candidate relations in the validation set whose relation suggestion score is not lower than the threshold and matches the labeled true value; FN(τ) is the number of false negative samples corresponding to the candidate threshold τ, referring to the number of candidate relations in the validation set that actually exist as true relations but whose relation suggestion score is lower than the threshold and are not included in the evidence coverage gap map; FP(τ) is the number of false positive samples corresponding to the candidate threshold τ, referring to the number of candidate relations in the validation set that actually do not exist as true relations but whose relation suggestion score is not lower than the threshold and are incorrectly included in the evidence coverage gap map. min The number of candidate relations N represents the minimum allowed precision for this relation type. cand (τ) represents the total number of candidate relationships entering the evidence coverage gap map at the candidate threshold τ, N max This represents the upper limit of the number of candidates allowed in a single round, and its value is the system queue capacity plus... The smaller of the two.

[0085] Cost term K(τ)=Σ i minm C(i,m) represents the total prediction cost required to invoke at least one available augmented extraction model from this batch of candidates. C(i,m) comes from a pre-generated model cost profile, and K... max The calculation budget allocated to the relationship gap in this round.

[0086] Furthermore, P min The threshold is determined based on the downstream business costs of erroneous relationships and the ability of manual review. High-risk relationships such as those involving supervision, penalties, and transaction control are set at 0.85 to 0.95, general business relationships at 0.75 to 0.85, and exploratory graphs at 0.70 to 0.75. The default value is 0.75, which is derived from the engineering constraint of keeping the proportion of subsequent invalid model calls within 25% while maintaining the candidate recall level. The threshold is readjusted monthly or when the system detects a relationship distribution drift of more than 10%.

[0087] The third type of gap node is the single-source gap node. A single-source gap node is generated when all the verified valid evidence anchors corresponding to the candidate relationship originate from the same independent source group, and the number of independent source groups does not reach the preset minimum number. This type of gap will prioritize associating with fragments in other documents that contain the same entity alias, relationship trigger words, or related event times to supplement independent source evidence until the number of independent source groups reaches the preset minimum number.

[0088] The fourth type of gap node is the conflict gap node. When multiple candidate relations under the same function key are incompatible, a conflict gap node is generated based on the conflict intensity of the candidate relations. The function key is determined by at least one of the following: head entity, relation type, and valid time range. Conflict intensity. Calculate using the following formula: , Among them, Z g z is the set of candidate relations under the same function key. i z j For any two distinct candidate relations in the candidate relation set, For candidate relation z i The confidence level of the first round of calibration, For candidate relation z j The first round of calibration confidence, I incomp (z i ,z j The value is an incompatibility indicator; it is 1 for incompatibility and 0 for compatibility. Therefore, the higher the confidence of both parties in the conflict, the higher the priority of the conflict gap.

[0089] To support subsequent scheduling and sorting, the importance w is calculated for each gap node. g The formula is: , Where h g To determine business importance, entity types or relationship types are assigned levels 1 to 5 by the domain schema and then normalized. This represents the remaining gap ratio. This represents the conflict intensity; it is set to 0 when there is no conflict. λ1 represents the dependency level, indicating how many downstream relational patterns, queries, or rules depend on the gap node. It can be obtained by normalizing the number of outgoing edges in the pattern graph or the number of historical query hits. λ1 is the weight coefficient corresponding to business importance, λ2 is the weight coefficient corresponding to the proportion of remaining gaps, λ3 is the weight coefficient corresponding to conflict intensity, and λ4 is the weight coefficient corresponding to dependency. The sum of λ1 to λ4 is 1. The parameter tuning is aimed at maximizing the amount of correct knowledge obtained per unit computational cost on the offline validation set, but when there is no historical data, they can be set to 0.30, 0.30, 0.25, and 0.15 respectively.

[0090] After generating the gap nodes, a reverse mapping relationship is established between the gap nodes and document fragments. The first search entry point is the fragment containing verified evidence of the entity or relationship involved in the gap node. Based on this, a multi-dimensional second search entry point is expanded, including other document fragments containing the standard name or alias of the entity corresponding to the gap, fragments containing relationship trigger words or attribute column names, adjacent fragments before and after existing evidence fragments, fragments within the same chapter, and evidence fragments of the gap-related entity's one- or two-hop neighbors in the graph. For single-source gaps, fragments from independent groups of the same source are excluded to supplement the search results. For conflict gaps, fragments containing conflicting parties, time-limited terms, negative terms, or revision terms are additionally searched. The document fragments retrieved from all the above entry points are associated with the corresponding gap nodes, ultimately forming an evidence coverage gap graph.

[0091] Evidence Coverage Gap Map H t It can be represented as: , in, Let t be the set of gap nodes in round t. A is a set of fragment nodes that can be used to fill the gap. t This is the set of edges connecting gap nodes and fragment nodes. Each edge stores at least the gap types that the fragment can cover, the retrieval criteria, the source independent group, and a list of candidate augmentation extraction models.

[0092] Step S2 above ultimately outputs an evidence coverage gap graph and a gap-to-fragment inverse index. Subsequent step S3 only calculates the coverage probability and cost of the fragment-model combination on the associated edges of this graph, without scanning all document fragments. This step transforms the extraction quality problem into a schedulable task on the graph. Gap nodes record the target evidence strength and the current remaining gaps, while fragment nodes store the original input position. The associated edges between them directly constrain the model input range based on the graph state. This application, on the one hand, achieves targeted filtering of candidate fragments through the gap-to-fragment inverse index, avoiding a full traversal of massive document fragments, significantly reducing the computational overhead of subsequent steps and lowering model call costs. On the other hand, it encapsulates multi-dimensional information such as business importance, conflict intensity, dependency, and evidence strength into the gap nodes, providing a unified quantitative scheduling basis for downstream tasks, facilitating priority ranking of completion and conflict resolution tasks. This gap-driven graph structure differs from traditional quality evaluation methods that simply calculate graph completeness and conflict rate. It is no longer limited to outputting static quality indicators, but directly outputs task carriers that can drive subsequent model execution, opening up the link from quality assessment to incremental extraction and completion, and taking into account both knowledge extraction effect optimization and engineering execution efficiency.

[0093] For step S3, based on the type of gap nodes in the evidence coverage gap graph and the association with the corresponding document fragments, the coverage probability and computational cost of the candidate fragment-model combination formed by the document fragment and each augmentation extraction model to fill the corresponding gap node are predicted, and the coverage gain per unit cost is determined accordingly. Here, the candidate fragment-model combination refers to the smallest task execution unit formed by pairing each candidate document fragment with each augmentation extraction model. When the same document fragment is paired with different augmentation extraction models, different gap-filling effects and computational costs will be achieved.

[0094] The coverage probability and computational cost of this method are determined as follows: A capability profile of the augmented extraction model is established for each gap node type. This capability profile reflects the success rate of the augmented extraction model in filling gaps in the corresponding types. For any candidate fragment-model combination and the gap node corresponding to that combination, the coverage probability of the combination filling the gap node is predicted based on the capability profile and the entity hit score, relation hit score, position score, conflict features, source novelty, and first-round uncertainty of the candidate fragment relative to the gap node. The computational cost required for the combination is predicted based on the input length, expected output length, and number of candidate knowledge units contained in the candidate fragment.

[0095] Furthermore, the unit cost coverage gain is determined as follows: Based on the text similarity between the candidate fragment and each selected fragment belonging to the same independent group in the selected fragment-model combination set, the independence reduction coefficient of the candidate fragment is determined, where the selected fragment-model combination set is the set of candidate fragment-model combinations selected for execution in the current round; based on the current coverage, coverage probability, independence reduction coefficient, and target evidence strength of each gap node, the updated coverage of each gap node after adding the candidate fragment-model combination is determined; the weighted increment of the coverage of each gap node before and after adding the candidate fragment-model combination is taken as the marginal coverage gain of the combination, and based on the marginal coverage gain and computational cost, the unit cost coverage gain is determined, which measures the gap coverage improvement corresponding to a unit computational cost.

[0096] Specifically, the first step is to establish categorized capability profiles for each augmentation extraction model. The system can configure multiple augmentation extraction models, including domain entity models, domain relation models, medium-sized generative models, and large-scale inference models. For each model m, its historical performance on various gap node types k is statistically analyzed to obtain the capability profile value q. m,k The calculation formula is: , in, The number of times model m successfully fills in the gap nodes in the test samples of gap node type k. This represents the total number of test samples for the model on gap node type k. The capability profile is derived from the offline validation set. When a new model lacks validation records, the F1 score of the publicly available validation set for the same task can be normalized as the initial value, and it will be continuously updated by manual sampling results after running.

[0097] Based on this, for any candidate segment f, model m, and gap node g in the evidence coverage gap map, the coverage probability p of the candidate segment-model combination filling the gap node can be predicted. fmg The coverage probability is based on the capability profile of the model corresponding to the gap node type, and then weighted and adjusted by the six feature scores of the candidate segment relative to the gap node. The calculation formula is as follows: , Where k(g) is the type of the gap node g, q m,k (g) To enhance the ability profile value of the extraction model m on the type to which the gap node g belongs, σ is the sigmoid function used to map the weighted score to the interval between 0 and 1.

[0098] The entity hit score y in the above formula entityThe maximum score for relevance matching between the target entity and candidate segments is taken. A score of 1.0 is used for exact match of the standard name, 0.85 for normalized alias match, and 0.70 for parsed pronoun match. Text vector similarity retrieval results are normalized using cosine similarity. When both the first and last entities are matched, the geometric mean of their scores is used. For entity type gap nodes without specific entity names, the maximum score from the entity type dictionary, field name, and weak candidate span is used as the entity hit score.

[0099] The relationship hit score y in the above formula relation Used to measure the indication strength of target relations in candidate segments, calculated as y relation =0.40M lex +0.35M dep +0.25M table Relationship with text hints x trigger The trigger words, syntactic paths, and table structures in the code use the same computational logic, but are limited to the specific relationships required by the current gap node.

[0100] The position score y in the above formula position The structural distance between the candidate fragment and the existing evidence fragment is calculated using the formula y. position =exp(-d pos / λ pos ). Calculate the structural distance d pos First, establish a global index idx for fragments of the same document according to the stable slice order. The structural distance between a candidate fragment f and an existing evidence fragment e is defined as d(f,e)=|idx f -idx e |+N sec (f,e), where N sec (f,e) represents the number of chapter boundaries crossed between the two segments, idx f Let idx be the global index of the candidate fragment f. e d is the global index of the existing evidence fragment e. pos Take the minimum value among all existing evidence segments corresponding to d(f,e), and within the same segment, d pos Take 0, take 1 for adjacent segments within the same chapter, and when crossing chapters, accumulate the difference in sequence number and the number of chapter boundaries. If there is a lack of existing evidence, use the target chapter segment in the domain template where the entity or relation usually appears as the reference segment to calculate the structural distance.

[0101] λ pos This is the location decay scale, used to indicate how much an increase in structural distance results in a significant decrease in the location score. This parameter can be calculated by plotting the median distance d of true evidence across historically confirmed supplementary evidence fragments. 50 Press λ again pos =d 50 / ln2 calculation, this allows for the calculation of y at typical real evidence distances. position Approximately 0.5, λ during the cold start phase pos Option 3 is acceptable; a reassessment should be conducted after accumulating at least 100 genuine supplementary evidence fragments.

[0102] The conflict feature y in the above formula conflict The measure of the degree to which candidate fragments contain conflict resolution-related information is obtained by weighting four sub-features, and the calculation formula is as follows: y conflict =0.30V value +0.25V neg +0.25V time +0.20V pair , Among them, V value V represents the matching degree of whether a candidate fragment contains conflicting values. neg V represents the hit rate of negation or withdrawal words in the candidate segment. time V measures the hit rate of time-related revision terms such as "revision," "expiration," "change," and "expiration" and their validity periods in the candidate segments. pair The matching degree is determined by whether the candidate segments contain both conflicting elements. All values ​​range from 0 to 1.

[0103] The source novelty y in the above formula source The formula used to measure the source independence of a candidate fragment relative to the evidence already used at the current gap node is as follows: y source =1-max e∈Eused [I same_group [(f,e)×sim(f,e)], Among them, E used I is the set of evidence already used for the current gap node. same_group (f,e) is the same group indicator function. It takes the value 1 if candidate fragment f and used evidence fragment e belong to the same independent source group, and 0 otherwise. When y comes from a completely new, unused independent source group... source The score is 1, close to 0 when the text is completely repeated in the same group, and partially retained based on similarity when there is new content in the same group.

[0104] The first-round uncertainty y in the above formula uncertain It is negatively correlated with the confidence score of the first round of candidate fragment extraction, and the calculation formula is: y uncertain =1-c z , Among them, c z The confidence score of the first-round extraction result corresponding to the candidate segment. When there are multiple relevant first-round results for the same segment, take 1 - max(c). z), a mode-triggered gap node with no initial results, where the initial uncertainty is 1.

[0105] Furthermore, θ0 is the bias coefficient, θ1 is the weight coefficient corresponding to the entity hit score, θ2 is the weight coefficient corresponding to the relation hit score, θ3 is the weight coefficient corresponding to the position score, θ4 is the weight coefficient corresponding to the conflict feature, θ5 is the weight coefficient corresponding to the source novelty, and θ6 is the weight coefficient corresponding to the first-round uncertainty. The weight coefficients θ0 to θ6 of the six features are obtained by training the logistic regression model through historical re-extraction records. When the system is first deployed, 0.25, 0.20, 0.15, 0.15, 0.10, and 0.10 can be used as initial weights, respectively, and the model can be refitted after accumulating at least 500 re-extraction feedbacks. This method of calculating coverage probability takes into account the attributes of the gap nodes in the graph, the original text position features, the source independence, and the model capability, avoiding the bias caused by selecting segments solely based on the first-round confidence.

[0106] In addition to predicting the coverage probability, the computational cost of each candidate fragment-model combination is also predicted. The computational cost K for model m to process candidate fragment f is also predicted. fm It consists of fixed overhead and multiple variable costs, and the calculation formula is as follows: , in, Input the number of tokens for candidate segments and prompt words. The expected number of output tokens can be obtained by regression analysis of the number of candidate entities and relationships in the fragment. α is the number of candidate entities, candidate relations, and gap nodes provided in the prompt. 0m To fix the startup cost of the model, α 1m α 2m and α 3m These are the model performance coefficients corresponding to the input length, output length, and number of candidates, respectively. Each coefficient is obtained by performing linear regression on each model using at least 100 to 500 inference calls, with the actual GPU milliseconds as the dependent variable. These coefficients need to be recalibrated when deploying on different hardware to ensure that the cost reflects the actual computational resource consumption. If an external model service is used, K can also be used. fm It is converted into the sum of input token fee, output token fee, and fixed call fee.

[0107] After obtaining the coverage probability and computational cost, it is necessary to calculate the actual marginal gain of candidate combinations by considering the homologous repetition of the selected fragment-model combination set. Let the candidate fragment-model combinations selected for execution in the current round constitute the selected fragment-model combination set S. To reduce the spurious gain caused by homologous repetition evidence, it is necessary to first calculate the independence reduction coefficient D(f,S) of candidate fragment f relative to set S. The calculation formula is as follows: , Where group(f) represents the source independent group to which candidate segment f belongs, group(x) represents the source independent group to which selected segment x belongs, sim(f,x) represents the text similarity between candidate segment f and selected segment x, and ξ is the duplicate evidence penalty coefficient, determined through the validation set, which can be initially set to 0.8. If there are no selected segments from the same source group in set S, the maximum similarity is treated as 0, and the independence reduction coefficient is 1.

[0108] The formula for calculating the text similarity sim(f,x) between candidate segment f and selected segment x is: , Among them, v f and v x Let f and x be the text embedding vectors for the candidate fragment f and the selected fragment x, respectively, and ||*||2 be the L2 norm of the vectors. Before calculating similarity, the fragments need to undergo uniform text normalization, removing navigation, copyright information, and fixed headers and footers, while retaining entities, values, times, and negation words. Then, a fixed version of the text embedding model is used for encoding. If the length does not exceed the token limit, the vector is obtained directly. For very long fragments, they are divided into multiple sub-blocks according to sentence boundaries, encoded separately, and then weighted and averaged according to the sub-block length before L2 normalization. Because the vectors have already been L2 normalized, the actual similarity calculation can be simplified to vector dot product operation. The model identifier, model version, normalization rule version, and vector hash result are saved with the results. Vectors generated by different model versions should not be directly compared. When the embedding service is unavailable, TF-IDF vector cosine similarity can be used as a downgrade scheme, and the similarity confidence flag is set to low.

[0109] Based on the independence reduction factor, the coverage of the gap node g in set S can be predicted. : , in, T represents the current coverage of the gap node g. g The strength of the target evidence corresponding to the gap node.

[0110] Based on this, add candidate fragment f and use the marginal cover gain after model m. The sum of the coverage increments of all gap nodes associated with the candidate segment, weighted by the importance of the gap nodes, is calculated using the following formula: , Where Γ(f) is the set of gap nodes connected to candidate fragment f in the evidence coverage gap graph, w g This represents the importance of the corresponding gap node.

[0111] Finally, the coverage gain per unit computational cost U(f,m|S) is the marginal coverage gain divided by the computational cost, as shown in the formula: , Where ε is a very small positive number to prevent the denominator from being zero, it can be taken as 10. -6 Unit cost coverage gain measures the improvement in gap node coverage corresponding to a unit computational cost and is a core indicator for subsequent scheduling selection.

[0112] Step S3 above ultimately supplements each gap node-segment association edge with coverage probabilities, computational costs, and unit cost coverage gains under different models. This allows subsequent steps to directly reuse the pre-calculated data for screening and re-extraction tasks, eliminating the need to repeatedly perform feature calculations and probabilistic inference. This mechanism unifies segment selection and model selection into a combinatorial optimization problem of candidate segment-model combinations, no longer treating segment screening and model selection as two independent processing steps. Although a longer candidate segment has a higher computational cost, if it can simultaneously cover multiple high-weight gap nodes, its unit cost coverage gain may still be better than that of a shorter segment. Conversely, even if some short segments have low individual processing costs, if the number of gaps they can cover is small or the importance of the gaps is low, the overall benefit will be suppressed. For model selection, even if a model has high overall accuracy, if its capability-cost ratio is low at a specific gap node, it will not be prioritized for scheduling and invocation, avoiding the ineffective consumption of extracting enhancement models. This design integrates the convergence and completion objective of missing nodes in the knowledge graph with the allocation and configuration of limited computing resources into a unified optimization objective. While ensuring the knowledge completion effect, it achieves fine-grained scheduling of document fragments and extraction models, balancing the recall benefits of re-extraction tasks with engineering computational overhead.

[0113] For step S4, based on the total budget, select the current round of re-extraction combination from the candidate fragment-model combination according to the unit cost coverage gain, and call the corresponding enhanced extraction model to perform re-extraction to obtain supplementary candidate knowledge units and their evidence anchors. As a preferred implementation, based on the total computational budget, the current round of re-extraction combinations are selected from candidate fragment-model combinations according to the coverage gain per unit cost, including: dividing the total computational budget into a mandatory budget and a general budget; the mandatory budget is used to meet the minimum coverage requirement of the mandatory gap, and the general budget is used to fill the remaining gaps; within the mandatory budget, candidate fragment-model combinations that fill the mandatory gaps are selected first, and then candidate fragment-model combinations are selected in descending order of coverage gain per unit cost using the general budget; the selection also satisfies the following: the same candidate fragment corresponds to only one enhanced extraction model per round; single-source gap nodes prioritize candidate fragments from independent groups of new sources; conflicting gap nodes retain candidate fragments from independent groups of different sources; when the mandatory gap does not reach the minimum coverage, the general gap is prohibited from occupying its reserved budget.

[0114] Furthermore, the selection of combinations for this round of re-extraction adopts a greedy approximation method, including: constructing a max-heap with the unit cost of each candidate fragment-model combination covering the upper bound of the gain; after popping the top of the heap in each round, recalculating the true gain of the combination under the selected fragment-model combination set; if it is still not lower than the upper bound of the next heap top, it is selected; otherwise, it is re-entered into the heap; after selection, only the gain of other combinations with the same gap associated with the combination is recalculated.

[0115] Specifically, an optimization problem is constructed with maximizing the weighted coverage gain as the core objective. The goal is to select a set of candidate fragment-model combinations within the total computational budget B, such that the sum of the weighted coverage of all gap nodes is maximized. The corresponding objective function is: , The constraint is the sum of the computational costs of the selected combinations: , Where S is the final selected fragment-model combination set.

[0116] In addition to calculating the total budget constraint, the selection process must also meet several business rule constraints. Among them, a candidate fragment can only select one primary extraction model in the same round; secondly, single-source gap nodes prioritize candidate fragments from independent groups of new sources; thirdly, conflict gap nodes retain at least candidate fragments from independent groups of different sources; and fourthly, for candidate fragments containing sensitive data, only enhanced extraction models that meet data localization and permission requirements are selected.

[0117] The system pre-defines mandatory gaps and sets minimum coverage requirements. Gap nodes marked as mandatory in the domain schema, gap nodes requiring confirmation under legal, regulatory, contract review, or security rules, gap nodes directly dependent on downstream key queries, decision rules, or model features, gap nodes with high business importance and unresolved conflicts, and gap nodes manually designated as requiring supplementary verification are all uniformly designated as mandatory gaps. The system sets a flag M(g)=1 for mandatory gaps and configures a dedicated minimum coverage C for each. min (g) Minimum coverage is set according to the type of gap node: 1.0 for gap nodes with mandatory entity existence, 0.90 for ordinary strong factual relationships, and 0.90 or higher for high-risk relationships with at least two independent sources, with no fewer than two independent sources. In addition to meeting the conflict condition, conflict gap nodes require the fusion confidence difference between the winning candidate and the second-best candidate to reach a set threshold. Specific values ​​are configured by the domain administrator and verified through historical manual acceptance rates.

[0118] To ensure the supply of resources to cover the mandatory shortfall, the total budget B will be broken down into a mandatory budget B. force And ordinary budget B normalTwo parts. The mandatory budget is used to meet the minimum coverage requirement for the mandatory gap, and the general budget is used to fill all other gap nodes. The formula for calculating the mandatory budget is: , Wherein, β is the minimum proportion reserved in the total budget B for mandatory gaps, used to prevent ordinary gap nodes from consuming all resources prematurely. Its value ranges from 0 to 1, and this scheme constrains it to the range of 0.30 to 0.50. This coefficient is derived from the actual cost percentage of mandatory gaps in historically completed tasks, and C is denoted as the cumulative consumption before the mandatory gaps reach the minimum coverage. force The total task cost is C. total The proportion of forced gap cost per single task is q. force =C force / C total Statistics on similar tasks (q) force The 75th percentile is then used to limit the range to a specified interval using a clipping function, i.e., β=clip[Q]. 75 (q force [0.30, 0.50], where clip indicates that the value is restricted to a given interval. β is 0.40 when there is no historical data, and is re-estimated every 20 similar tasks.

[0119] K min (g) The minimum prediction cost required to achieve the minimum coverage for a single forced gap g. Let the current coverage of the forced gap g be C. g The minimum requirement is C. min (g), and the set of executable candidate fragment-model combinations connected to g is A. g The coverage probability of combination i is p ig The computational cost is K. i For any subset S, the predicted coverage C after execution g (S) is: C g (S)=1-(1-C g )∏ i∈S (1-p ig ), The solution for minimum cost is then in the form of: The constraint condition is C g (S)≥C min (g) In scenarios with a large number of candidate scenarios, it is not necessary to exhaustively search all subsets; you can press p. ig (1-C g ) / K i Add each element to the combination from highest to lowest and update the coverage C. g (S), reaching C for the first time min The cumulative cost at time (g) is used as K. min The online approximation of (g).

[0120] During scheduling, tasks are first selected within the mandatory budget based on priority: "Mandatory Gap Importance × Remaining Gap Proportion / Cost". Then, the regular budget is used to optimize all gaps. Unused budget in the mandatory queue can be lent to the regular queue, but the regular queue cannot use reserved budget for mandatory gaps that have not yet met their targets. When the calculated total budget is insufficient to cover all mandatory gaps, the system prioritizes high-risk gaps based on business risk level, legal deadlines, downstream dependency frequency, and unit cost gain. A list of incomplete mandatory gaps and the estimated additional budget are also output. Tasks are not made to appear complete by lowering the minimum coverage.

[0121] Before the formal scheduling and selection, the system compresses the candidate size through a three-layer pruning mechanism to improve computational efficiency. The first layer is gap node-candidate fragment recall pruning, where each gap node retains only the Top-K candidate fragments obtained by merging BM25 retrieval, vector retrieval, structural location retrieval, and neighbor retrieval. K is initially set to 50 to 200, while ensuring the diversity of independent groups from the source. The second layer is candidate fragment-model pruning, where combinations whose model capability profile is below the capability threshold, whose model does not meet data permissions, whose prediction coverage probability is below the minimum threshold, or whose prediction cost exceeds the remaining budget are directly deleted. The third layer is dominance pruning and redundancy pruning. For the same candidate fragment, if combination A has a coverage probability of no less than that of combination B for all connected gaps and a lower cost, then combination B is directly eliminated. Combinations that highly overlap with the selected evidence and whose marginal gain after independence reduction is below the retention threshold are also deleted.

[0122] The minimum threshold for predicted coverage probability is used to filter combinations that are almost impossible to fill the gap. Its value is determined by scanning the validation set with actual re-extraction results. The goal is to remove the most invalid candidates while ensuring that the recall rate of combinations that successfully fill the gap is not less than 95%. A value of 0.10 can be used during the cold start phase. The marginal gain retention threshold has the same dimension as the weighted coverage gain. When there is sufficient historical data, it is taken as the 5th percentile of the marginal gain distribution of actual successfully re-extracted combinations. When there is no historical data, it is taken as 1% of the maximum gain of all positive gain combinations in the current round.

[0123] Considering the high cost of directly solving integer programming with a massive number of candidate fragments, the system adopts an incrementally executable greedy approximation method. This method uses a lazy max-heap as its core data structure, initially constructing a max-heap based on the upper bound of the unit cost coverage gain for each candidate fragment-model combination in an empty set. Each time a top-of-the-heap combination is popped, only the true marginal gain of that combination in the currently selected fragment-model combination set is recalculated. If the recalculated gain is still not lower than the upper bound of the next top-of-the-heap and there is sufficient remaining budget, the combination is selected; otherwise, it is re-added to the heap with the new gain value. After selecting a combination, only other combinations that share the same gap node are marked as awaiting recalculation; all unrelated combinations are not recalculated, thus significantly reducing computational load. The evidence coverage gap graph can be processed in parallel by connected components or business partitions, and finally merged and sorted in the global budget queue.

[0124] From a computational complexity perspective, let the total number of candidate fragment-model combinations be M, the total number of gap-combination related edges be E, and Q combinations be ultimately selected. The maximum number of adjacent combinations affected by each selected combination is D. If a naive full recalculation method is used, updating the gain of all combinations after each selection results in a time complexity of O(QM). Using an inverted adjacency list and a lazy max-heap mechanism, the initial heap construction complexity is O(E+MlogM), the expected time complexity of the selection process is O(QDlogM), and the memory complexity is O(E+M). Under the Top-K truncation constraint at each level, the number of related edges E is constrained by the product of the total number of gap nodes, the number of candidate fragments recalled per gap K, and the number of available models. This avoids the Cartesian product operation between all candidate fragments and models in scenarios with hundreds of millions of candidates.

[0125] To prevent approximate selection from missing mandatory gaps, the lowest-cost feasible combination with a mandatory gap is checked separately before each round of heap selection. This combination is not affected by the normal candidate Top-K truncation and is given priority in the mandatory budget queue.

[0126] The complete greedy approximation execution process can be divided into five steps: First, initialize the selected combination set to empty and the used budget to 0; Second, calculate the upper bound of the unit cost coverage gain for all optional combinations and construct a max-heap; Third, pop the top combination of the heap and calculate its true marginal coverage gain and unit cost coverage gain under the current selected set; Fourth, if the gain is still not lower than the upper bound of the next heap top and the remaining budget is sufficient, add the combination to the selected set and update the used budget, while marking other combinations associated with the same gap to be recalculated; Fifth, terminate the selection process when the budget is insufficient, all forced gaps have reached the target, the maximum unit cost gain is lower than the stopping threshold, or the upper limit of the number of candidate fragments in this round is reached.

[0127] The maximum unit cost gain stopping threshold is derived from historical tasks. When the number of correct new knowledge generated by continuing to invoke the model falls below the system's minimum unit computational cost, the corresponding unit gain value becomes the stopping threshold. For initial deployment, this threshold can be set to 5% to 10% of the median unit gain of the top 100 candidate combinations.

[0128] After selecting a combination, the system generates corresponding gap constraint prompts for each candidate fragment to be executed, writing relevant gap information into the prompts to limit the augmented extraction model to extract content that needs to be filled. The prompts must contain at least six pieces of information: 1) the entity type or relation type to be confirmed; 2) the known entity's standard name and alias; 3) the existing evidence and its deficiency type; 4) the values, source dates, or validity periods of the conflicting parties; 5) the format constraints for the output entity, relation, and original evidence positions; and 6) constraints prohibiting the generation of relationships without evidence from the candidate fragment. When the same candidate fragment is associated with multiple gap nodes, the system will merge the constraints of multiple gaps into a single model call, reducing the need for repeated token input.

[0129] The system then calls the corresponding augmented extraction model to perform high-precision re-extraction. After the augmented extraction model outputs the locations of the supplementary new entities, relations, and evidence, the system performs structural format verification, original text location verification, and entity type constraint verification on the output. Results where evidence cannot be located in the original text are marked as candidates without anchor points and are not directly included in the effective coverage.

[0130] Step S4 above ultimately outputs the knowledge units, evidence locations, model call costs, and task execution records obtained through re-extraction, which are then provided to step S5 to update the candidate graph and evidence coverage gap graph based on these results. The core function of this step is to transform the graph quality improvement problem into a combinatorial optimization problem under budget constraints. It selects the candidate fragment-model combination required to cover the gaps, rather than a fixed proportion of fragments or all low-confidence fragments. Gap constraint prompts ensure that the augmented extraction model's processing range aligns with the current gaps in the graph. The same candidate fragment can simultaneously complete entity completion, relation completion, and conflict verification, effectively increasing the amount of effective knowledge generated in a single model call.

[0131] For step S5, the supplemented candidate knowledge units are written into the candidate graph. Based on the results of this round of re-extraction, the set of affected gap nodes in the evidence coverage gap graph is located. Only the coverage status of this set is incrementally updated. If the overall coverage status containing multi-dimensional indicators after the update does not meet the stopping condition, the updated candidate graph and evidence coverage gap graph will be used to enter the next round S3.

[0132] The updated overall coverage status, encompassing multiple dimensions, is a set of indicators representing the completeness of candidate graph knowledge, conflict convergence, and iteration gains. It is incrementally updated based on the results of this round of re-extraction and serves as the core basis for determining iteration termination conditions and scheduling tasks in the next round. It includes a quality dimension corresponding to the overall weighted coverage rate, a conflict dimension corresponding to the unresolved conflict rate, and a gain dimension covering coverage improvement, conflict rate reduction, new knowledge additions, unit cost benefits, and improvement trends, respectively measuring coverage progress, knowledge consistency, and input-output efficiency. This status is only incrementally calculated for gap nodes affected in this round; unaffected nodes retain the status from the previous round, ensuring accuracy while avoiding the overhead of full recalculation. Each indicator corresponds to stopping condition branches such as quality achievement, conflict convergence, and gain decay, collectively constituting the basis for determining iteration termination.

[0133] As a preferred implementation, incremental updates include: aligning the supplemented candidate knowledge units with existing candidate knowledge units in the candidate graph; the fusion confidence of the same candidate knowledge unit is determined by comprehensively considering the confidence of all supporting evidence, source reliability, and source independence coefficient; retaining multiple candidate values ​​and their evidence chains for conflicting relationships, marking a conflicted relationship as resolved only when the fusion confidence of a candidate relationship leads other candidate relationships by a set margin and the number of source independence groups meets the requirements; determining the set of gap nodes affected by the results of this round of re-extraction, and recalculating only the coverage status of this set, which includes gap nodes directly associated with the re-extracted fragments, entity type gap nodes corresponding to newly added or merged entities, relational pattern gap nodes affected by changes in the head and tail entities, and conflicting gap nodes sharing the same function key.

[0134] Furthermore, the stopping conditions for incremental updates include: the overall weighted coverage rate reaches the preset target value and the unresolved conflict rate does not exceed the preset threshold; the cumulative actual computational cost consumed reaches the budget limit; the increase in the overall weighted coverage rate for multiple consecutive rounds is lower than the dynamically determined minimum increase threshold; the unit cost coverage gain of all optional candidate fragment-model combinations is lower than the stopping threshold; and the maximum processing time limit stipulated by the business is reached.

[0135] Specifically, after the system completes high-precision re-extraction, it first aligns and fuses the supplemented candidate knowledge units with the existing candidate knowledge units in the candidate graph. For the same candidate knowledge unit z, its fusion confidence Conf(z) is calculated by integrating the attributes of all supporting evidence, and the calculation formula is as follows: , Where, ε zThe evidence set supports knowledge unit z. Multiple independent, high-quality pieces of evidence will continuously increase the fusion confidence. However, duplicate evidence from the same source has a low independence coefficient and will not bring an equivalent increase in confidence, thus avoiding artificially high confidence caused by duplicate counting of content from the same source.

[0136] For conflict-related relationships, the system retains multiple candidate values ​​and their complete evidence chains, rather than simply retaining the result with the highest current confidence. Only when the fusion confidence of a candidate relationship is higher than the conflict resolution confidence difference threshold δ of other candidate relationships will the system retain that value. conf Only when the number of independent groups from the source meets the corresponding requirement is the candidate relation marked as resolved, and δ conf The ablation rate of incorrect solutions on the validation set is determined, and can initially be taken as 0.15 to 0.25.

[0137] After the knowledge unit fusion is completed, the system quickly locates the set A of gap nodes affected in this round through the gap node-fragment reverse index. t : , This set consists of four types of gap nodes. The first type is gap node A, which is directly related to the fragment being re-extracted in this round. direct The second category is entity type gap node A, which has the same standard identifier as the newly added or merged entity. entity The third type is the gap node A in the relational pattern that is affected after the head or tail entity changes. relation The fourth type is conflict gap node A that shares the same function key. conflict The system only recalculates set A. t The evidence strength, coverage, candidate fragments, and associated edges of each gap node are updated. Unaffected gap nodes directly retain the state from the previous round, thereby achieving incremental updates and reducing computational overhead.

[0138] Based on this, the system calculates the overall weighted coverage Cov in round t. t This is used to measure the overall completion status of all missing nodes, and the calculation formula is: , in, This is the set of all gap nodes in the current round. This represents the remaining uncovered percentage of the gap node.

[0139] Simultaneously calculate the unresolved conflict rate. This is used to measure the convergence degree of conflict-type gaps, and the calculation formula is: , in, The number of unresolved conflict gap nodes. This represents the total number of candidate relationships in the candidate graph. The increase in overall weighted coverage ΔCov in this round. t ΔCov is the difference between the coverage rate of the current round and the coverage rate of the previous round. t =Cov t -Cov t-1 .

[0140] The system determines whether to terminate the re-extraction iteration based on multi-dimensional stopping conditions; it stops when any one of these conditions is met. The first type is the quality achievement condition, where the overall weighted coverage reaches a preset target value τ. cov And the unresolved conflict rate does not exceed the preset threshold τ conf The second type is the budget exhaustion condition, where the cumulative actual computational cost reaches the upper limit of the total computational budget; the third type is the revenue decay condition, where the increase in the overall weighted coverage rate for J consecutive rounds is lower than the dynamically determined minimum increase threshold τ. gain The fourth type is the candidate exhaustion condition, where the unit cost coverage gain of all available candidate fragment-model combinations is lower than the stopping threshold τ. U The fifth category is time-limit conditions, which refer to the maximum processing time limit stipulated by the business.

[0141] Where, τ cov The map coverage is determined by the business requirements, and is typically taken as 0.90 to 0.98. conf The value of J is determined by the acceptable conflict ratio in downstream operations; J can be 2 to 3, and τ U The minimum boost threshold τ corresponds to the maximum unit cost gain stopping threshold in step S4. gain The value is determined using a dynamic adaptive approach, rather than a fixed value.

[0142] The system records the coverage improvement ΔCov for each round of historical tasks. t Conflict rate decrease ΔConf t The number of newly added correct knowledge units ΔN t And the cost ΔK calculated in this round t Then calculate the unit cost revenue r. t : r t =(ω1ΔCov t +ω2ΔConf t +ω3ΔN t / N target ) / ΔK t , Where ω1 is the weighting coefficient corresponding to the coverage improvement, ω2 is the weighting coefficient corresponding to the conflict rate reduction, and ω3 is the weighting coefficient corresponding to the number of newly added correct knowledge units. The sum of the three is 1, and their specific values ​​are determined by business objectives. N targetThe target total number of knowledge units for the task is used to normalize the number of newly added knowledge units to a dimension consistent with the coverage rate.

[0143] To reduce the interference of single-round fluctuations on the judgment of inflection points, the original unit cost-benefit series is smoothed using a three-round moving median or exponential moving average to obtain a smoothed benefit series. .

[0144] Inflection point of returns t * The segmentation point is determined by piecewise linear regression. Two straight lines are fitted before and after the inflection point for each candidate segmentation point. The earliest segmentation point with the smallest sum of squared residuals and an absolute slope after the inflection point not exceeding 50% of the slope before the inflection point is selected. Simultaneously, the smoothed return is required to be lower than η times the average return of the previous two rounds for J consecutive rounds. η is initially set to 0.20 to 0.30. When the sample size is small, a simplified criterion can be used: a decrease in unit cost return for three consecutive rounds with a cumulative decrease exceeding 70%.

[0145] τ gain The final value takes into account both the statistical inflection point and the minimum business impact, and the calculation formula is as follows: τ gain =max(τ biz ,median(ΔCov t* ,...,ΔCov t*+J-1 )), Where, τ biz For the minimum perceptible weighted coverage improvement that the business deems worth continuing to calculate, ΔCov is configured based on downstream accuracy requirements or manual review tolerance. t* To ΔCov t*+J-1 τ represents the coverage improvement over J consecutive rounds near the historical task inflection point, where median represents the median. Without a business threshold, τ... biz The value can be determined based on the maximum change in coverage upgrades during historical manual reviews that did not alter the final acceptance conclusion. When there is no historical data for a new task, τ... gain It can be initialized to improve the median by 5% to 10% in the first round of coverage, and after accumulating no less than 20 similar tasks, the results can be switched to segmented regression.

[0146] Budgetary conditions ensure that the system has a defined upper bound on resources in scenarios with massive amounts of documents. If the stopping condition is not met, the system proceeds to the next round (S3) with the updated candidate graph and evidence coverage gap graph.

[0147] The final output of step S5 is the updated knowledge graph, evidence chain, unresolved gap list, model call records, and resource consumption statistics. If iteration continues, the updated evidence coverage gap graph is directly used as the input for the next round S3. This step uses an incremental affected subgraph update mechanism to avoid rescanning all documents and all gap nodes in each round. The re-extraction results will eliminate, reduce, transfer, or generate new gap nodes. The gap changes then control the model input for the next round, forming a convergent computational closed loop.

[0148] This invention achieves incremental quality improvement of knowledge graphs through a gap-driven closed-loop iterative mechanism, significantly reducing redundant computational overhead compared to full re-extraction schemes. The system organizes candidate fragments and model combinations around gap nodes, achieving efficient allocation of computing resources through quantitative evaluation of gain coverage per unit cost and greedy optimization scheduling, while ensuring mandatory gap priority through a hierarchical budget mechanism. Multi-source evidence fusion and conflict retention mechanisms balance the accuracy of confidence calculation with the rigor of conflict resolution, avoiding erroneous conclusions caused by premature merging. Incremental updates of affected subgraphs and adaptive stopping conditions jointly guarantee controllable convergence of the iterative process, achieving an optimal balance between business quality goals and computational costs, demonstrating high engineering practical value and resource efficiency advantages.

[0149] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the above embodiments do not limit the present invention in any way, and all technical solutions obtained by equivalent substitution or equivalent transformation fall within the protection scope of the present invention.

Claims

1. A hierarchical extraction method for document entity relations based on graph evidence coverage gap feedback, characterized in that, Includes the following steps: S1. Slice the document set, use the initial extraction model to perform the first round of extraction on each segment, obtain candidate knowledge units and their evidence anchors in the original documents, and construct candidate graphs accordingly. S2. Generate gap nodes based on the difference between the evidence strength of knowledge units in the candidate graph and the target evidence strength, establish a reverse mapping relationship between each gap node and the document fragment that can fill the gap, and form an evidence coverage gap graph. S3. Based on the type of the gap node in the evidence coverage gap diagram and the association of the corresponding document fragment, predict the coverage probability and computational cost of the document fragment and the candidate fragment-model combination formed by each enhancement extraction model to fill the corresponding gap node, and determine the coverage gain per unit cost accordingly. S4. Based on the total budget, select the current round of re-extraction combination from the candidate fragment-model combination according to the unit cost coverage gain, and call the corresponding enhanced extraction model to perform re-extraction to obtain supplementary candidate knowledge units and their evidence anchors. S5. Write the supplemented candidate knowledge units into the candidate graph. Based on the results of this round of re-extraction, locate the set of affected gap nodes in the evidence coverage gap graph. Only perform incremental updates on the coverage status of this set. If the updated overall coverage status containing multi-dimensional indicators does not meet the stopping condition, then proceed to the next round S3 with the updated candidate graph and the evidence coverage gap graph.

2. The document entity relationship hierarchical extraction method based on graph evidence coverage gap feedback according to claim 1, characterized in that, The validity of the evidence anchor points of the candidate knowledge units is verified: When all verification items pass, the document is marked as valid and included in the calculation of the strength of evidence. When a relationship verification lacks a clear trigger word and the head and tail entities and table structure can be confirmed, it is marked as weakly valid. The first-round calibration confidence of the candidate knowledge unit is reduced by a set ratio before it enters the candidate graph. If any of the range verification, readback verification, version verification, or entity verification fails, it is marked as invalid, not included in the evidence strength calculation, and enters the relocation queue, to be re-entered into the candidate map after subsequent relocation and repair.

3. The document entity relationship hierarchical extraction method based on graph evidence coverage gap feedback according to claim 2, characterized in that, The gap nodes include at least the following types: When the number of confirmed entity types in the target document fragment is lower than the preset target number in the candidate graph, an entity type gap node is generated; When the entity type satisfies the preset pattern compatibility constraint, there is relationship indication information, and the evidence strength of the current relationship is lower than the target evidence strength, a relationship pattern gap node is generated; When all the verified and valid evidence anchor points corresponding to the candidate relationships in the candidate graph originate from the same independent source group, and the number of the independent source groups does not reach the preset minimum number, a single source gap node is generated. When multiple candidate relationship contents under the same function key are incompatible, a conflict gap node is generated based on the conflict intensity of the multiple candidate relationships. The function key is determined by at least one of the head entity, relationship type, and effective time range.

4. The document entity relationship hierarchical extraction method based on graph evidence coverage gap feedback according to claim 3, characterized in that, The source independent group is generated in the following manner: After normalizing the documents, calculate the content hash value, locality-sensitive hash signature, and text vector for each document, and extract the reference links; Candidate recall is based on at least one of the following four criteria: the content hash values ​​are the same, the local sensitive hash signatures meet the matching value, the similarity of the text vectors exceeds the recall threshold, or there is a reference relationship. Based on text similarity, publication time difference, or core fact coverage, the recalled document pairs are determined to be from the same source, and documents that meet the criteria are grouped into the same independent group of sources. When merging, the new document must have a similarity to the cluster center text that is not lower than the merging threshold, and must meet the same source determination with a preset number of documents within the cluster; otherwise, merging is prohibited.

5. The document entity relationship hierarchical extraction method based on graph evidence coverage gap feedback according to claim 1, characterized in that, The reverse mapping relationship is established as follows: The first search entry point is the segment containing verified and valid evidence of the entities or relationships involved in the gap node; The second search entry is at least one of the following: a fragment containing the name of the entity involved in the gap node, a fragment containing the relationship trigger word involved in the gap node, a fragment adjacent to the fragment in the first search entry, or a fragment associated with the neighboring entity of the entity involved in the gap node. The document fragments retrieved by the first search entry and the second search entry are associated with the gap node.

6. The document entity relation hierarchical extraction method based on graph evidence coverage gap feedback according to claim 1, characterized in that, The coverage probability and the computational cost are determined in the following ways: A capability profile is established for the enhanced extraction model for each of the gap node types, and the capability profile is used to reflect the success rate of the enhanced extraction model in filling gaps in the corresponding types; For any candidate fragment-model combination and the corresponding gap node, based on the capability profile and the entity hit score, relation hit score, position score, conflict features, source novelty and first-round uncertainty of the candidate fragment relative to the gap node, predict the probability that the combination will fill the gap node's coverage. The computational cost required for the combination is predicted based on the input length of the candidate fragment, the expected output length, and the number of candidate knowledge units it contains.

7. The document entity relation hierarchical extraction method based on graph evidence coverage gap feedback according to claim 6, characterized in that, The unit cost coverage gain is determined as follows: The independence reduction coefficient of the candidate segment is determined based on the text similarity between the candidate segment and each selected segment belonging to the same source independent group in the selected segment-model combination set, wherein the selected segment-model combination set is the set of candidate segment-model combinations that have been selected for execution in the current round; Based on the current coverage of each gap node, the coverage probability, the independence reduction coefficient, and the target evidence strength of each gap node, the updated coverage of each gap node after adding the candidate fragment-model combination is determined; The weighted increment of the coverage of each gap node before and after adding the candidate fragment-model combination is used as the marginal coverage gain of the combination. Based on the marginal coverage gain and the computational cost, the unit cost coverage gain is determined. The unit cost coverage gain is used to measure the gap coverage improvement corresponding to the unit computational cost.

8. The document entity relation hierarchical extraction method based on graph evidence coverage gap feedback according to claim 1, characterized in that, The step of selecting the current round of re-extraction combination from the candidate fragment-model combination according to the calculated total budget and the unit cost coverage gain includes: The total budget is divided into a mandatory budget and a general budget. The mandatory budget is used to meet the minimum coverage requirement of the mandatory gap, and the general budget is used to make up for the remaining gaps. Within the mandatory budget, the candidate fragment-model combination that fills the mandatory gap is selected first, and then the candidate fragment-model combination is selected in descending order of the unit cost coverage gain within the ordinary budget; The selection process also satisfies the following: the same candidate fragment corresponds to only one enhanced extraction model per round; single-source gap nodes prioritize the candidate fragments from independent groups of new sources; conflict gap nodes retain the candidate fragments from independent groups of different sources; and when the forced gap does not reach the minimum coverage, ordinary gaps are prohibited from occupying their retention budget.

9. The document entity relation hierarchical extraction method based on graph evidence coverage gap feedback according to claim 8, characterized in that, The selection of the combination for this round of re-sampling adopts a greedy approximation method, including: A max-heap is constructed using the unit cost of each candidate fragment-model combination to cover the upper bound of the gain. After popping the top of the heap in each round, the true gain of the combination under the selected fragment-model combination set is recalculated. If it is still not lower than the upper bound of the next heap top, it is selected; otherwise, it is put back into the heap. When selected, only the gain of other combinations with the same notch associated with this combination will be recalculated.

10. The document entity relation hierarchical extraction method based on graph evidence coverage gap feedback according to claim 1, characterized in that, The incremental update includes: The supplemented candidate knowledge units are aligned with the existing candidate knowledge units in the candidate graph. The fusion confidence of the same candidate knowledge unit is determined by a combination of the confidence of all its supporting evidence, source reliability, and source independence coefficient. Conflicting relationships retain multiple candidate values ​​and their chains of evidence. A candidate relationship is marked as resolved only when its fusion confidence exceeds that of other candidate relationships by a set margin and the number of independent groups from the source meets the requirement. The set of gap nodes affected by the results of this round of re-extraction is determined, and the coverage status of only the set is recalculated. The set includes the gap nodes directly associated with the re-extracted fragments, the entity type gap nodes corresponding to the newly added or merged entities, the relational pattern gap nodes affected after the head and tail entities change, and the conflicting gap nodes that share the same function key.

11. The document entity relation hierarchical extraction method based on graph evidence coverage gap feedback according to claim 1, characterized in that, The stopping conditions include: The overall weighted coverage rate reaches the preset target value and the unresolved conflict rate does not exceed the preset threshold. The cumulative actual computational costs have reached the budget limit; The increase in the overall weighted coverage rate in multiple consecutive rounds is lower than the dynamically determined minimum increase threshold; The unit cost coverage gain of all the optional candidate fragment-model combinations is below the stopping threshold; The maximum processing time limit stipulated by the business has been reached.

Images