Semantic depth proofreading and man-machine rechecking method and system based on large model and knowledge graph traceability

Abstract

The invention relates to the technical field of electric digital data processing, and discloses a semantic depth proofreading and man-machine rechecking method and system based on a large model and knowledge graph traceability, and the method comprises the steps: constructing a knowledge field vector index composed of authoritative facts; calculating the vector of the fact assertion to be corrected and the gravitational gradient thereof in the knowledge field; according to the method, a preposed risk quantification mechanism is established, so that a traditional post-verification mode is avoided, computing resources can be dynamically allocated to high-risk assertion, hidden errors which do not accord with a fact basis are recognized in a semantic level, and the risk classification is carried out according to the gradient, so that the subsequent deep analysis process is differentially started or skipped. And the proofreading efficiency and quality are improved.

Description

TECHNICAL FIELD

[0001] The application relates to a semantic deep proofreading and man-machine rechecking method and system based on a large model and a knowledge graph traceability, and belongs to the technical field of electronic digital data processing. BACKGROUND

[0002] In current text processing applications, a common technical solution is to combine a large language model for deep semantic understanding and use a knowledge graph for fact tracing to realize automatic proofreading. Compared with traditional rule-based or shallow matching methods, this solution has obvious technical advantages in dealing with problems such as context contradictions, logical errors and inconsistent terms. However, when this technical solution is applied to large-scale text processing, there is an inherent technical contradiction between operation cost and processing efficiency in the work process. The existing process usually adopts a post-validation work mode, that is, the system first performs unified deep scanning on all text content through a large language model to identify potential errors, and then calls external resources such as a knowledge graph to verify the identified potential errors. This fixed work process causes the high computational load of deep semantic analysis to be applied to all text information to be processed without distinction, including a large amount of correct information.

[0003] Therefore, simply improving the reasoning performance of the large language model itself or optimizing the query speed of the knowledge graph cannot change the work process itself in which computing resources are uniformly allocated to all information, and cannot fundamentally solve the problem of excessive overall operation overhead. Analysis shows that the existing technical solution has the following deficiencies in process design: 1. The processing flow lacks a calculation link that can pre-judge the deviation degree of text assertions from factual basis before deep semantic analysis; 2. The allocation of computing resources is not associated with the probability of information error, and is equally allocated to all information to be processed by the previous link; 3. The preliminary screening mainly based on semantic recognition is difficult to generate effective distinguishing features for errors that meet the grammar and logic but do not conform to the facts, resulting in the possibility of being missed.

[0004] Moreover, even some existing technologies aimed at optimizing the reasoning process have not been able to break away from this way in the macro flow of data processing, for example, the Chinese invention patent with the publication number CN118606487B discloses a knowledge graph reasoning engine data processing method based on logical reasoning. Although this method attempts to ensure the accuracy of the results by analyzing the logical pathway characteristics and the cross-relation of the pathways, and combining subsequent traceability verification, it is essentially still a serial mode of performing unified and complex logical verification on all data entering the processing flow. It does not set a preliminary risk assessment link and cannot divert according to the degree of deviation of the data from the existing knowledge system, so it also faces the problem of wasting a lot of computing resources on verifying correct information that does not need to be analyzed in depth. This makes it difficult to fundamentally avoid the inherent defect of excessive computational overhead when processing massive data. Therefore, how to establish a new data processing method that can quantify the internal risk of each fact assertion in the text through an efficient preliminary calculation link before performing high-cost semantic depth analysis, and divert the information processing according to the quantification results, and allocate computing resources to assertions with high risk first, has become a technical problem to be solved by the present invention. SUMMARY

[0005] The present application provides a semantic depth proofreading and man-machine review method and system based on large model and knowledge graph traceability, which mainly aims to solve the problems of fixed computing resource allocation mode, high proofreading cost and hidden fact error omission risk caused by the lack of a preliminary risk quantification link in the prior art.

[0006] To achieve the above-mentioned purpose, the present application provides a semantic depth proofreading and man-machine review method based on large model and knowledge graph traceability, which performs a preliminary risk assessment and processing path decision step before a semantic depth analysis step is executed. The method comprises: Step a, constructing a knowledge field vectorization index: calling a first sentence encoding model, calculating each triple in a knowledge graph composed of multiple (entity, relationship, attribute) triples to generate a knowledge vector, and storing all generated knowledge vectors in a vector database, establishing a vector index supporting nearest neighbor search for all knowledge vectors, and the index is defined as a knowledge field; Step b, performing pre-examination based on the assertion vector and the knowledge field gravity gradient: calling a sentence encoding model identical to the first sentence encoding model to calculate a fact assertion extracted from the text to be proofread to generate an assertion vector; performing a nearest neighbor search in the knowledge field to find a knowledge vector closest in geometric distance to the assertion vector, and calculating the Euclidean distance between the assertion vector and the found nearest neighbor knowledge vector, defining the distance value as the knowledge field gravity gradient; Step c, performing a differentiated processing path based on the comparison result of the knowledge field gravity gradient: when the gradient value is less than a preset first threshold value, the fact assertion is determined as a high-confidence assertion, and the subsequent semantic depth analysis step and the provenance step performed by the knowledge graph are skipped; when the gradient value is greater than a preset second threshold value, the fact assertion is determined as a high-risk abnormal assertion, and the semantic depth analysis step and the provenance step are started.

[0007] Preferably, the fact assertion extracted from the text to be proofread is a structured expression containing a core entity, a core relationship and a core attribute, which is atomized from an unstructured sentence of the text to be proofread through natural language processing technology.

[0008] Preferably, the method further comprises a closed-loop feedback self-evolution step based on the human-computer review result, and the closed-loop feedback self-evolution step comprises: structurally archiving the final decision of a human expert on the output result of the semantic depth analysis step and the provenance step; when the final decision is to confirm a new fact or to modify an existing fact, updating the confirmed or modified fact in the form of a (entity, relationship, attribute) triple to the knowledge graph, and calling the first sentence encoding model to recalculate the knowledge vector of the updated triple to update the knowledge field vectorization index.

[0009] Preferably, the method further comprises a step of calculating an implicit risk score of the high-risk abnormal assertion for determining the processing priority of the high-risk abnormal assertion in the semantic depth analysis step; the calculation rule of the implicit risk score is limited to: wherein, is the implicit risk score; is the knowledge field gravity gradient of the fact assertion; is the contextual credibility score of the fact assertion in the context paragraph of the text to be proofread; is historical adoption rate data representing the authority degree of the source associated with the fact assertion; , , is a preset weight coefficient, and the preset weight coefficient , , and 1.

[0010] Preferably, the human-machine review is implemented through a human-machine interaction interface, which is configured to: for a high-confidence assertion, display the high-confidence assertion side by side with the original triple corresponding to the nearest neighbor knowledge vector found in the knowledge field, and provide a confirmation option and a correction option; for a high-risk abnormal assertion, display the high-risk abnormal assertion side by side with the correction suggestion generated by the semantic depth analysis step performed by the large model, and the traceability basis provided by the traceability step performed by the knowledge graph, and provide an option to accept the correction suggestion, an option to reject the correction suggestion, and an option to edit the correction suggestion.

[0011] Preferably, a preset third threshold is further set between the preset first threshold and the preset second threshold, wherein the preset first threshold is less than the preset third threshold, and the preset third threshold is less than the preset second threshold; when the knowledge field gravitational gradient is greater than the preset first threshold and less than the preset third threshold, the fact assertion is determined as a potential risk assertion, and the semantic depth analysis step and the traceability step are started with a priority lower than that of the high-risk abnormal assertion.

[0012] Preferably, the data sources of the knowledge graph include at least one of authoritative literature, legal provisions, industry standards, and historical correction records.

[0013] Preferably, the first sentence encoding model and the same sentence encoding model as the first sentence encoding model are a distillation BERT model pre-trained on a medium-sized general corpus and then fine-tuned using a domain corpus in the data source.

[0014] Preferably, the context credibility score The generation step includes: based on a preset semantic pattern library and context dependency relationship analysis, determining whether the fact assertion is a sentiment-dependent assertion; if it is determined to be yes, a context sentiment analysis model is called to analyze the context paragraph where the fact assertion is located to extract its sentiment polarity, subjective intensity, and context consistency features, and to generate a context credibility score based on the extracted features The method further includes a step of dynamically calibrating the preset weight coefficient , , The dynamic calibration step includes: comparing the priority ranking results of a human expert on a plurality of high-risk abnormal assertions with the priority ranking results calculated by the system based on the implicit risk scores to calculate a ranking consistency deviation; and adjusting the value of the preset weight coefficient , , according to the ranking consistency deviation.

[0015] A semantic depth proofreading and man-machine rechecking system based on a large model and a knowledge graph traceability, the system comprising: A knowledge field construction module, the knowledge field construction module being configured to: call a first sentence encoding model, calculate each triple in a knowledge graph composed of a plurality of (entity, relation, attribute) triples to generate a knowledge vector, and store all generated knowledge vectors in a vector database, and establish a vector index supporting nearest neighbor search for all knowledge vectors, the index being defined as a knowledge field; A pre-examination module, the pre-examination module being configured to: call a sentence encoding model identical to the first sentence encoding model, calculate a fact assertion extracted from the text to be proofread to generate an assertion vector; perform a nearest neighbor search in the knowledge field to find a knowledge vector closest in geometric distance to the assertion vector, and calculate the Euclidean distance between the assertion vector and the found nearest neighbor knowledge vector, and define the distance value as the knowledge field gravitational gradient; A processing path decision module, the processing path decision module being configured to: based on the comparison result of the knowledge field gravitational gradient, execute a differentiated processing path, when the gradient value is less than a preset first threshold value, determine the fact assertion as a high confidence assertion, and control the system to skip the subsequent semantic depth analysis step and traceability step performed by the large model and the knowledge graph; when the gradient value is greater than a preset second threshold value, determine the fact assertion as a high-risk abnormal assertion, and control the system to start the semantic depth analysis step and the traceability step.

[0016] Compared with the prior art, the beneficial effects of the present application are: 1、Before starting the depth semantic analysis, a knowledge field vector index composed of a large number of facts is first constructed, and by calculating the distance between the assertion vector in the text to be proofread and the nearest neighbor knowledge vector in the knowledge field, a gravitational gradient value representing the consistency degree of the assertion with the existing knowledge system is obtained; this way establishes a preliminary risk assessment mechanism based on the internal correlation degree of data, so that the subsequent computing resources are no longer indiscriminately invested in all text information, but are guided to different processing paths according to the gradient value, high-risk abnormal assertions are preferentially sent to the depth analysis process, and high-confidence assertions are directly sent to the artificial confirmation link, thereby changing the previous proofreading system's inherent serial processing and post-validation of all information.

[0017] 2. This method places the construction of the knowledge field vectorized index, the real-time calculation of assertion vectors, and the subsequent deep analysis based on the large model into a unified closed-loop feedback system. The final decision of human experts on the results of the deep analysis is structured and archived and used in reverse to update the facts or relationships in the knowledge graph. This directly supplements or strengthens the vector index of the knowledge field and provides high-quality labeled data for the subsequent fine-tuning of the large model. In this way, a continuous and mutually reinforcing data coupling relationship is formed between the benchmark knowledge field of the pre-examination and the core large model of the subsequent deep analysis. The overall verification capability of the system evolves spontaneously as its processing tasks increase.

[0018] 3. By decomposing the text to be proofread into atomic factual assertions and calculating the gravitational gradient of each assertion in the knowledge field, the system can effectively identify implicit errors that are semantically grammatically correct and logically consistent but deviate from the factual basis. This is because the assertion vectors of such errors will be in a relatively sparse position in the knowledge field vector space composed of objective facts due to the lack of sufficient gravitational pull from relevant knowledge vectors, and their calculated gravitational gradient values ​​will be significantly different from those assertions that are consistent with the facts. This provides a direct identification path based on the data space distribution for dealing with key factual errors that are easily missed by traditional proofreading methods based solely on semantic analysis. Attached Figure Description

[0019] Fig. 1 This is a flowchart of the closed-loop proofreading and knowledge graph self-evolution process of human-machine collaboration in this invention. Fig. 2 This is a graph showing the multi-dimensional performance improvement of the system's self-evolution. Fig. 3 This is a diagram illustrating the risk pre-screening and closed-loop feedback framework of the present invention based on the knowledge field gravitational gradient. Detailed Implementation

[0020] To make the objectives, technical solutions, and advantages of the present invention clearer, the technical solutions of the present invention will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0021] The application provides a semantic depth proofreading and man-machine review method and system based on a large model and a knowledge graph traceability, which is configured to perform a preliminary risk assessment and processing path decision step before a semantic depth analysis step is executed. The system includes a knowledge field construction module, a pre-examination module, a processing path decision module, and a man-machine review and closed-loop feedback module in physical implementation. The knowledge field construction module is used to convert an authoritative fact-based knowledge graph into a vectorized index supporting efficient retrieval, which is defined as a knowledge field in the technical solution. The pre-examination module is used to receive the text to be proofread, disassemble it into atomized fact assertions, and calculate the gravitational gradient of each fact assertion in the knowledge field. The processing path decision module classifies the risk of each fact assertion according to the calculation result of the gravitational gradient, and controls the system to execute differentiated subsequent processing paths, thereby dynamically allocating computing resources to high-risk assertions. The operation of the system aims to solve the technical problems of fixed computing resource allocation and hidden fact error omission caused by the lack of a preliminary risk quantification link in existing electronic digital data processing processes. In a specific application scenario, that is, when automatically proofreading a contract draft containing a large number of technical specifications and legal clauses, an objective obstacle of the existing processing process is that it must uniformly call a high-cost large language model for deep scanning of all clauses in the contract, regardless of their correctness or incorrectness, resulting in the consumption of computing resources. To address this challenge, the knowledge field construction module of the technical solution is configured to perform the construction step of the knowledge field vectorized index in the system initialization stage. The initial state of the module includes a knowledge graph composed of entity, relationship, and attribute triples, and the data sources include authoritative literature, legal provisions, and industry standards, as well as a distilled BERT model pre-trained on a medium-sized general corpus and fine-tuned on the aforementioned domain corpus as the first sentence encoding model. The controller calls the first sentence encoding model to calculate each triple in the knowledge graph to generate a 768-dimensional knowledge vector, and stores all generated knowledge vectors in a vector database, establishing a vector index supporting nearest neighbor search for all knowledge vectors, which is defined as the knowledge field. This process establishes a vector space benchmark based on authoritative facts that can be efficiently calculated, providing a data basis for subsequent risk prediction.

[0022] Further, in order to avoid indiscriminately imposing the computational load of deep semantic analysis on all text information to be processed when performing the proofreading task, the pre-examination module of the present technical solution is configured to perform a pre-examination step based on the assertion vector and the knowledge field gravitational gradient; when receiving the contract draft text to be proofread, the processor first uses natural language processing technology to disassemble the unstructured sentences in the text into one or more atomized structured fact assertions containing a core entity, a core relationship and a core attribute, for example, disassembling the sentence "the warranty period of equipment A is three years" into the fact assertion (equipment A, warranty period, three years); then, the processor calls a sentence encoding model identical to the first sentence encoding model used when building the knowledge field to calculate each extracted fact assertion to generate an assertion vector of the same dimension; for each generated assertion vector, the processor performs a k-nearest neighbor search operation in the knowledge field, where k is equal to 1, to find a knowledge vector closest to the current assertion vector in the vector space, and calculates the Euclidean distance between the assertion vector and the nearest neighbor knowledge vector it finds, and defines the calculated distance value as the knowledge field gravitational gradient; for example, if the assertion vector generated by the aforementioned fact assertion is the nearest neighbor knowledge vector corresponding to the known fact (equipment A, warranty period, three years) found in the knowledge field is and the calculated knowledge field gravitational gradient is which may be 0.05, this step generates a quantitative risk indicator for each fact assertion that can represent the degree of consistency with the existing knowledge system through a vector neighbor search and distance calculation.

[0023] The process of decomposing unstructured sentences in text into atomic fact assertions is performed by one of two configurable technical paths, the first of which is a pipeline processing mode that sequentially calls a named entity recognition model based on the BERT-CRF structure and a relationship extraction model based on the attention mechanism, the former is used to identify the core entities and attributes contained in the sentence, and the latter is used to judge the pre-defined relationship types between entities and attributes based on the identification results of the former; the second path is a unified generation mode, which takes the unstructured sentence as input, calls a generative large language model with a parameter of not less than 7 billion through a structured prompt containing specific instructions and output format examples, and directly generates a JSON formatted text string containing (entity, relationship, attribute) key-value pairs; when the system is deployed, for the scene with high requirement on processing speed in batch processing task, the system is configured to adopt the first path, while for the scene with high requirement on complex sentence and implicit relationship extraction accuracy in online processing task, the system is configured to adopt the second path; after obtaining the knowledge field gravitational gradient of each fact assertion, in order to realize the differentiated allocation of computing resources, the processing path decision module of the technical scheme is configured to execute differentiated processing paths based on the comparison results of the gradient values; the execution of the module depends on two preset thresholds, a first threshold and a second threshold , wherein is less than , these two thresholds can be determined by offline calibration experiments, for example, setting to 0.1 and setting to 0.7; when the gradient value of a fact assertion is less than the first threshold , such as 0.05 in the aforementioned example, the assertion is determined as a high-confidence assertion, and the processing path decision module will control the system to skip the subsequent semantic depth analysis step performed by the large model and the tracing step performed by the knowledge graph, and directly display the assertion and the matching original triple in the knowledge field in the human-computer interaction interface for the operator to confirm; when the gradient value is greater than the second threshold , it indicates that the assertion is significantly deviated from all known facts in the knowledge field, and is determined as a high-risk abnormal assertion, and the module will control the system to start the subsequent semantic depth analysis and tracing step; when the gradient value is between and , it is determined as a potential risk assertion, and the depth analysis and tracing step is started with a lower priority than the high-risk abnormal assertion, this differentiated processing path decision mechanism ensures that computing resources can be focused on text content with a higher error probability.

[0024] Moreover, to prioritize the processing of high-risk abnormal assertions and make the system's risk assessment model adaptive, the technical solution further includes an implicit risk score calculation step and a closed-loop feedback self-evolution step based on the results of human-computer review; for the fact assertions determined to be high-risk abnormal assertions, the system will calculate their implicit risk scores according to the defined calculation rules ; wherein, is the calculated knowledge field gravity gradient; is the context credibility score, and its generation step includes calling a context sentiment analysis model to analyze the context paragraph where the assertion is located to extract features such as sentiment polarity and subjective intensity, and generating a score value based on these features; is a historical adoption rate data representing the authority degree of the source associated with the fact assertion; and , , is a preset weight coefficient, and the sum is 1, and can be adjusted through a dynamic calibration step; in the human-computer review link, the final decision of a human expert on the system's deep analysis result is structured and archived, and when the decision is to confirm a new fact or modify an existing fact, the updated fact will be updated in the form of a triple to the knowledge graph, and the knowledge vector will be recalculated to update the knowledge field vector index; at the same time, the actual sorting results of the human expert on the processing priority of multiple high-risk abnormal assertions will be compared with the sorting results calculated by the system to calculate the sorting consistency deviation, and the values of the weight coefficients will be adjusted according to the deviation, and the values of the weight coefficients , , will be adjusted according to the deviation, and the values of the weight coefficients will be adjusted according to the deviation, and the values of the weight coefficients

[0025] Embodiment 1: This embodiment is a specific operation example of the technical solution described in a specific scenario. In the final review process of a new drug marketing application file submitted by a large pharmaceutical company to the drug supervision administration, the application file is huge, containing thousands of pages of clinical trial data, production batch number, pharmacology and toxicology research reports, and chemical composition description. There are hidden fact errors that cannot be found by traditional proofreading methods, i.e. the text is grammatically and semantically correct, but the facts stated are inconsistent with the project knowledge base or other parts of the file. For example, the content of a key component is described as 0.5% in the main text, but recorded as 0.45% in the appendix test report. Such errors, if missed, may delay the entire approval process. When the system using the technical solution processes the new drug marketing application file, the system first starts the pre-review step based on the assertion vector and the knowledge field gravitational gradient, rather than immediately invoking a large model with high computational cost for comprehensive semantic analysis. The processor in the system disassembles the entire application file into tens of thousands of atomized fact assertions, such as the molecular formula of compound A , 350 subjects were enrolled in the phase II clinical trial, and the environmental temperature for product stability testing was 25 °C. Using the same sentence encoding model as when building the knowledge field, each fact assertion is calculated as an assertion vector. At the same time, the knowledge field of the system has pre-constructed a vectorized index containing tens of millions of knowledge vectors through the knowledge field construction module, based on the entire historical project data accumulated by the pharmaceutical company, all standard operating procedures, and relevant pharmacopoeia standards. For the vast majority of fact assertions extracted as described above, such as the molecular formula of a compound or the number of subjects enrolled in a clinical trial, because their content is highly consistent with the authoritative facts already existing in the knowledge field, the calculated knowledge field gravitational gradient value is less than the first preset threshold , for example, less than 0.1. The system accordingly determines these assertions as high-confidence assertions and directly skips the subsequent deep analysis process. This mechanism enables the system to release computational resources from a large amount of correct information, thereby resolving the contradiction between the high computational cost and the need for comprehensive proofreading in traditional data processing processes.

[0026] Furthermore, the system focuses computational resources on a small number of assertions determined to be at risk. During processing, a fact assertion that the environmental temperature for product stability testing is 25 °C has a calculated knowledge field gravitational gradient value of 0.82, which is greater than the second preset threshold . The system immediately determines it as a high-risk abnormal assertion because the standard operating procedure triple stored in the knowledge field related to the product explicitly records the environmental temperature as 30 , this comparison based on vector space distance enables the system to identify implicit errors deviating from the factual basis before the semantic understanding level; at this time, the screening of the pre-preliminary review step and the accurate analysis of the post-depth analysis step form a synergistic effect, and the processing path decision module immediately starts the semantic depth analysis step performed by the large model and the traceability step performed by the knowledge graph, but the target of analysis is no longer the full text, but only the environmental temperature of 25 This high-risk assertion and its context; the large model combines the context for logical reasoning, while the knowledge graph provides traceability evidence, clearly showing that the assertion conflicts with the specific provisions of the standard operating procedure document; in this way, the technical solution no longer attempts to directly answer the question of how to more quickly perform semantic analysis on the full text, but rather converts it into how to pre-judge which parts of the text are worth performing semantic analysis on, by changing the initial entry point of data processing, so that the original technical limitations can be circumvented in the new processing framework; ultimately, the system submits a focused proofreading report to the pharmacist responsible for the review, which does not contain a large number of text polishing suggestions, but only lists a few key factual errors verified, including the environmental temperature of the product stability test, and for each error, it displays its location in the original text, the correction suggestion given by the large model, and the authoritative traceability evidence provided by the knowledge graph; the pharmacist completes the confirmation and correction of the key errors within a few minutes according to the report, ensuring the accuracy of the submitted file, and the entire data processing task, compared to the traditional process, has improved in processing efficiency and the detection rate of implicit factual errors; it should be noted that in the final human-computer review link of the aforementioned embodiment 1, the human-computer interaction interface operated by the pharmacist is configured to provide differentiated display and operation methods for assertions of different risk levels; for the environmental temperature of the product stability test of 25 This high-risk abnormal assertion, the interface displays it together with the correction suggestion of environmental temperature of 30 provided by the traceability step performed by the knowledge graph, and the traceability evidence of standard operating procedure SOP-B-12 section 3.4: the environmental temperature of the stability test should be maintained at 30 ±2 The interface displays them side by side and provides an option to accept the correction suggestion, an option to reject the correction suggestion, and an option to edit the correction suggestion for the pharmacist to make a decision; for assertions with high confidence determined by the system, such as the molecular formula of A compound is , the interface displays the assertion together with the original triple (A compound, molecular formula, ) and provide a confirmation option and a correction option, so that pharmacists can quickly compare the consistency of the two, thereby greatly improving the decision-making efficiency and reliability of the manual review link.

[0027] To further verify the beneficial effects of the technical solutions of the present application in solving the problems of the prior art through objective data comparison, the following Comparative Example 1 is provided.

[0028] Comparative Example 1: This comparative example aims to simulate the scenario of using the existing conventional technical path in the background art section to automatically proofread the same new drug marketing application file as in Example 1. The system used in this comparative example has the same hardware configuration (including processor, memory, and graphics processing unit), called large language model, knowledge graph as factual benchmark, and natural language processing technology for text decomposition into factual assertions as the system in Example 1. The only essential difference is that the system used in this comparative example does not configure the pre-examination module based on knowledge field gravitational gradient and the processing path decision module claimed in the present application, and it follows a fixed, non-differentiated serial processing flow, i.e., all the approximately 15,000 atomized factual assertions decomposed from the application file are submitted to the large language model for deep semantic analysis without exception, and the knowledge graph is called for factual tracing and verification. To quantitatively evaluate the performance difference between the two technical solutions in processing the same large-scale professional document, the key performance indicators of the experimental group (i.e., the execution process of Example 1) using the method of the present application and the control group using the conventional method of this comparative example are recorded and compared, and the results are shown in Table 1.

[0029] Table 1: Comparison of performance indicators between experimental group and control group.

[0030]

[0031] From the data of Table 1, in the core quality index of proofreading, that is, the detection rate of hidden fact errors, both methods perform equally, and can effectively identify most of the key errors. However, in terms of operation efficiency and resource consumption, there are fundamental differences between the two. The control group applies high computational load to all information indiscriminately due to its post-validation mode, resulting in a total processing time that is 7.5 times that of the experimental group. During processing, the resource occupancy rate of its graphics processing unit is continuously saturated, which in practical engineering applications means higher hardware and energy costs, and makes it difficult to handle multiple proofreading tasks concurrently. The experimental results show that without sacrificing proofreading accuracy, the conventional technical path cannot avoid the technical defects of high computational cost and low processing efficiency when dealing with large-scale, high-sensitivity technical documents, which also proves that the core technical features of the present application can improve the overall operation efficiency of the automated proofreading system.

[0032] Example 2: To objectively verify the effectiveness of the pre-examination step based on the knowledge field gravitational gradient in identifying hidden fact errors and optimizing the allocation of computing resources in the technical solution, the following comparative test was designed and performed. The purpose of this test is to quantitatively compare the differences in key performance indicators between the test group using the technical solution and the control group using the prior art by running on a standardized text corpus containing known error types and distributions; the test platform is based on a server equipped with a general-purpose processor, 64 GB of memory, and a graphics processing unit with 40 GB of video memory; both the test group and the control group use the same BERT model based on the specific implementation as the sentence encoding model, and call the same large language model to perform deep semantic analysis; the data source used in the test is a publicly available legal document dataset. First, 10,000 documents were randomly selected from the dataset to build the knowledge field for this test, ensuring that the facts in the knowledge field are authoritative and unmodified original facts; then, another 5,000 documents were taken as the basis for the test corpus, and an automated script was used to inject two types of hidden fact errors into them at a predetermined rate: the first type is high-risk abnormal assertions, totaling 500, characterized by subtle modifications to key numbers or entities in the original text, such as changing the date May 10, 2023 to May 20, 2023; the second type is potential risk assertions, totaling 500, characterized by introducing new facts that do not exist in the knowledge field but are correct in themselves; the remaining facts, which account for more than 99% of the total, remain unchanged as high-confidence assertions; the technical solution of the control group is set to directly call the large language model to perform complete deep semantic analysis on the entire test corpus; the test group uses the technical solution, i.e., first performs the pre-examination step, and then processes the fact assertions according to the calculation results of the knowledge field gravitational gradient; after the test starts, the pre-examination module of the test group first calculates the knowledge field gravitational gradient for all fact assertions in the test corpus, and classifies them according to the pre-set first threshold of 0.1 and second threshold of 0.7, the statistical results show that 99.1% of the fact assertions are determined to be high-confidence assertions and skip deep analysis because their gradient values are less than 0.1, while among the 1000 injected error or new assertions, 988 have gradient values greater than 0.1 and are sent to the subsequent processing flow; then, the two systems are processed in parallel until the proofreading task for the entire test corpus is completed, and the key performance indicators are recorded, see Table 2 for specific data.

[0033] Table 2: Comparison of performance indicators between test group and control group.

[0034]

[0035] From the data in Table 2, in terms of the detection rate of hidden fact errors, the performance of the sample group and the control group is at the same level, which shows that the pre-examination step does not lead to the omission of key errors due to the filtering of a large number of high-confidence assertions; however, in terms of processing efficiency, the total processing time of the sample group is only 13.2% of that of the control group, because the pre-examination step reduces the amount of data that requires in-depth semantic analysis from 100% to less than 1%, which directly confirms the core mechanism of the technical solution, that is, the pre-quantitative risk through the knowledge field gravity gradient can effectively separate the computing resources from a large amount of information that is probably correct, and direct them to a small number of high-risk information; the test results show that by introducing a pre-examination step before in-depth semantic analysis, the technical solution can reduce the data processing amount of subsequent high-cost computing steps without reducing the ability to detect hidden fact errors.

[0036] Embodiment 3: This embodiment combines Figs. 1 to 3 the semantic depth proofreading and man-machine review method and system based on large models and knowledge graph tracing, as shown in Fig. 1 , the process starts with the original text to be proofread, after step a: fact assertion analysis and intent structure extraction to generate factual assertions, enters step b: classification, sentiment and context analysis of factual assertions, divides the assertions into simple fact type assertions and sentiment dependent type assertions, for the former, the system verifies efficiently through step c1 using the knowledge graph, or through step c2 to make an automatic auxiliary decision on the conflicting verification evidence, for the latter, it enters step d, and the sentiment large model and the knowledge graph generate RAG in the way of retrieval enhancement for in-depth analysis, the processing results of the two paths are collected in step e: man-machine collaborative review and closed-loop reconstruction, for the final decision of the experts, the decision result of the experts finally triggers step z1: the system performs self-evolution of the knowledge graph based on the arbitration result, and updates the knowledge graph and the vertical large model based on the arbitration data.

[0037] As shown in Fig. 2 , the horizontal coordinate of the graph is the training time from the initial state to the 20th week, and the vertical coordinate is the detection rate%, the graph shows the evolution curves of three key performance indicators, which are the detection rate of hidden fact errors, the detection rate of logical errors and the detection rate of term inconsistency, all three curves show a steady upward trend with the passage of training time, which shows that the system can continuously enhance its recognition and detection ability for different types of errors through continuous learning and iteration; as shown in Fig. 3As shown, it begins with unstructured text from a source text to be proofread. This text is first broken down into multiple atomic fact assertions by the fact assertion extraction module. Subsequently, each fact assertion is fed into the gravity gradient calculation module. This module uses a B1: knowledge field vectorization index, constructed from knowledge graph triples, as a benchmark to generate an assertion vector for the assertion and compare it with the knowledge field, thereby outputting a quantified knowledge field gravity gradient value. This gradient value is passed to the decision processing path module, which performs risk triage based on the gradient value, selecting high-confidence assertions with lower gradient values ​​along with their... The original triples matched in the knowledge field are directly submitted to human experts for review, while the high-risk / potential-risk assertions with high gradient values ​​are sent to the deep analysis and tracing module. This module calls the large model and knowledge graph for analysis, and submits the generated correction suggestions and tracing evidence to human experts. Finally, the final decision made by the human experts is not only used to generate the final proofreading report to complete this proofreading task, but also serves as a closed-loop feedback signal to update the triples in B2: the knowledge graph, and then update the vectorized index of B1: the knowledge field through the updated knowledge vector.

[0038] Example 4: This example aims to illustrate the calibration procedure for key parameters in the preceding technical solution. In a specific application scenario, when the verification system constructed by this technical solution is deployed for the first time in a new professional field, such as the review of prospectuses in the financial securities field, one of the objective technical challenges it faces is the first threshold used to distinguish factual assertions of different risk levels. With the second threshold And implicit risk score calculation rules for refined ranking of high-risk anomaly assertions. Weighting coefficients in , , All parameters are currently unknown. Setting these parameters based solely on experience will lead to deviations in risk assessment and handling path decisions during subsequent system operation. To address this issue, a standardized offline calibration process must be executed before the system is officially deployed. The initial state of this process is defined as: a corpus containing 10,000 authoritative documents in the field, used to construct an initial knowledge field; and a validation set containing 1,000 documents pre-annotated by human experts in the field, where each factual assertion is labeled as one of three categories: correct assertion, incorrect assertion, or newly added factual assertion. First, to determine the first threshold... With the second threshold The system calculates the knowledge field gravitational gradient for each fact asserted in the validation set, taking the value of . ; through statistical analysis of the calculation results, the gradient values of different categories of assertions show different distribution intervals, the gradient values of correct assertions are concentrated in a lower interval, the gradient values of factually incorrect assertions are concentrated in a higher interval, and the gradient values of newly added fact assertions are between the two; accordingly, the determination procedure of the first threshold value is limited to: within a preset gradient value interval, such as 0.01 to 0.30, iterating with a step size of 0.01, calculating the precision and recall of correctly classifying correct assertions at each candidate threshold value, and selecting the gradient value that maximizes the harmonic mean of the two, i.e. the F1 score, as the final value of the first threshold value ; similarly, the determination procedure of the second threshold value is limited to selecting the gradient value that maximizes the F1 score of classifying factually incorrect assertions within another preset interval; through this procedure, the and a set of parameters based on data statistics can be obtained.

[0039] Further, to calibrate the weight coefficients , , in the implicit risk score calculation rule, the system selects all samples labeled as factually incorrect assertions and newly added fact assertions from the verification set, and for each sample, calculates its context credibility score ; the calculation of this value is determined as an algorithmic process, i.e. calling a preset context sentiment analysis model to analyze the context paragraph where the assertion is located to extract its sentiment polarity , subjective intensity and context consistency three characteristic values, and according to to calculate , where is the sentiment polarity value, is the subjective intensity value, is the context consistency value, and this calculation method makes objective, neutral and context consistent statements obtain higher credibility scores; after obtaining the gradient value , the context credibility score and the historical adoption rate data (initial value uniformly set to 0.5) of each sample, the system presents a subset of 20 samples in random order to a human expert, and records the expert's handling priority ranking results based on his professional judgment; then, the system executes an optimization algorithm to find a set of under the constraint of The weight coefficient is combined to reach the maximum Spearman rank correlation coefficient between the calculated ranking result and the ranking result of the human expert; through the process, an initial weight coefficient is determined , , Through the above-mentioned whole set of standardized calibration procedures, the originally uncertain multiple core parameters obtain the initial set values based on objective data and clear optimization targets, so that a parameter setting link depending on experience is converted into a logically closed and reproducible engineering implementation step, which ensures that in the initial deployment stage of the correction system in different professional fields, the core risk assessment and decision mechanism can be adapted to the data characteristics of the current field, providing an initial benchmark for subsequent stable operation and self-evolution.

[0040] Embodiment 5: This embodiment aims to explain the dynamic maintenance procedure of the knowledge field and related parameters in the foregoing technical solutions, which is the operation basis thereof. In a specific application scenario, when the correction system constructed by the technical solution is applied to review technical standard documents in a continuously evolving technical field, such as the communication industry, the field will periodically issue new standards, introduce new technical terms and parameter indicators. If the knowledge field of the system is not updated synchronously, it will determine the assertions that have become facts in the new standard as high-risk abnormal assertions, thereby generating unnecessary in-depth analysis overhead. To cope with this working condition, the technical solution is configured with a time effectiveness guarantee and reconstruction mechanism for the knowledge field, which is set to be automatically executed at a preset period. When executed, the system first accesses a plurality of pre-set external authoritative data sources, such as the official release library of the industry standard organization, and automatically obtains all the documents newly added or revised in this period. The system processes these documents, extracts the triples (entity, relationship, attribute) contained therein, and stores these triples in a temporary staging area. Subsequently, the system calculates the knowledge field gravitational gradient of each triple in the temporary staging area in the current formal knowledge field, and identifies the triples with a gradient value greater than a pre-set second threshold as potential newly added or changed knowledge, and generates a difference report by aggregating them, which is submitted to a human expert for final confirmation. After being confirmed by the human expert, these newly added or changed triples are formally updated to the knowledge graph, and the knowledge field construction module recalculates the knowledge vectors thereof to update the knowledge field vectorization index. At the same time, the replaced old version triples are time-stamped and archived for subsequent version tracing. Through this procedure, the knowledge field serving as the pre-examination benchmark can keep pace with the evolution of the external knowledge system.

[0041] Further, to ensure that the historical adoption rate data representing the authority degree of the source in the calculation of the implicit risk score This technical solution is also equipped with a feature that can reflect the reliability of information from different sources. The dynamic update procedure for values; in this procedure, each information source in the system is assigned a unique source identifier and an initial value is set. Values, such as 0.5; In the human-machine review process of the system, when a human expert makes a final decision on an assertion originating from a specific source identifier, whether it is acceptance, modification, or rejection, the decision result is associated with that source identifier and recorded; the system periodically counts the total number of all associated decisions under each source identifier. And the number of times it was ruled acceptable. and based on The calculation rules are used to update the historical acceptance rate data of the information source. In addition, in order to make The value can better reflect recent changes in the credibility of the information source. A time decay factor can be introduced into the update calculation, assigning higher weight to recent rulings. This procedure will... The value has been transformed from a static initial setting into a data-driven reputation score that can be dynamically adjusted based on actual verification results.

[0042] Embodiment 6: This embodiment aims to supplement the specific selection, construction and fine-tuning procedures of the first sentence encoding model and the sentence encoding model identical to the first sentence encoding model in the foregoing technical solutions, in order to eliminate the model black box that may exist in specific engineering implementation; in a specific application scenario, when the proofreading system constructed by the technical solution needs to be applied to a field with a highly specialized terminology system, such as the aircraft design specification review in the aerospace field, a general sentence encoding model that has not been adapted may not be able to accurately capture the fine semantic differences of the professional terms in the field (such as flutter boundary or static aeroelastic divergence) when mapping them to the vector space, resulting in deviations in the calculated knowledge field gravity gradient, affecting the accuracy of the pre-examination step; to cope with this boundary condition, the sentence encoding model used in the technical solution has an initial state of a distilled BERT model pre-trained on a medium-sized general corpus, which aims to balance the computational efficiency and model performance; before applying it to a specific professional field, a standardized domain adaptation fine-tuning process needs to be performed; the initial state of this process is defined as a training dataset containing at least 1 million sentence pairs from authoritative literature in the target professional field (such as aerospace), where each sentence pair is labeled as semantically equivalent or semantically non-equivalent; the fine-tuning process uses a contrastive learning framework, which passes the two input sentences through the distilled BERT model with shared weights to obtain their corresponding sentence vectors, and minimizes the vector distance of semantically equivalent sentence pairs while maximizing the vector distance of semantically non-equivalent sentence pairs as the optimization objective to iteratively update the model parameters; the process is complete when the classification accuracy of the model on the reserved validation set converges to a stable value, for example, higher than 98%.

[0043] Further, to ensure the performance stability of the sentence encoding model in the long run and to adapt to the continuous evolution of domain knowledge, the technical solution is also configured with an incremental fine-tuning mechanism based on human-computer review feedback; in this mechanism, all factual assertions that have been finally decided by human experts in the human-computer review link and their corresponding original triples corresponding to the nearest neighbor knowledge vectors in the knowledge field are automatically constructed as new training sentence pairs; specifically, if a high-confidence assertion is confirmed as correct by a human expert, the assertion and its corresponding original triple are constructed as a semantically equivalent sentence pair; if a high-risk abnormal assertion is updated to the knowledge graph after being modified by a human expert, the assertion before modification and the fact triple after modification are constructed as a semantically inequivalent sentence pair; the system periodically accumulates these high-quality labeled data generated by actual business, and when the data amount reaches a preset threshold, such as 10,000 new sentence pairs, an incremental fine-tuning process is automatically triggered, and the model after the previous fine-tuning is used as the basis for further training using these new data; through this procedure, it is ensured that the understanding ability of the sentence encoding model, which is the basis for the semantic representation of the entire system, for professional domain terminology can be continuously optimized.

[0044] It is obvious to those skilled in the art that the present application is not limited to the details of the above exemplary embodiments, and can be implemented in other specific forms without departing from the spirit or essential characteristics of the present application.

[0045] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application and are not limiting, and although the present application has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present application can be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present application.

Claims

1. A semantic deep proofreading and human-computer verification method based on large model and knowledge graph source tracing, characterized in that, This method performs a preliminary risk assessment and processing path decision-making step before executing a semantic deep analysis step. The method includes: Step a, construct a knowledge field vectorized index: call a first sentence encoding model to calculate each triple in a knowledge graph consisting of multiple triples to generate a knowledge vector, and store all generated knowledge vectors in a vector database. Build a vector index that supports nearest neighbor search for all knowledge vectors. This vector index is defined as a knowledge field. Step b, perform pre-screening based on assertion vector and knowledge field gravitational gradient: call a sentence encoding model that is the same as the first sentence encoding model, calculate an assertion vector for a factual assertion extracted from the text to be proofread; perform a nearest neighbor search in the knowledge field to find a knowledge vector that is geometrically closest to the assertion vector, and calculate the Euclidean distance between the assertion vector and the found nearest neighbor knowledge vector, and define the Euclidean distance value as the knowledge field gravitational gradient; Step c: Based on the comparison results of the knowledge field gravity gradient, execute differentiated processing paths: when the gradient value is less than a preset first threshold, the factual assertion is judged as a high-confidence assertion, and the subsequent semantic deep analysis step performed by the large model and the source tracing step performed by the knowledge graph are skipped; when the gradient value is greater than a preset second threshold, the factual assertion is judged as a high-risk abnormal assertion, and the semantic deep analysis step and the source tracing step are initiated.

2. The semantic deep proofreading and human-computer verification method based on large model and knowledge graph tracing as described in claim 1, characterized in that, The factual assertion extracted from the text to be proofread is an atomized, structured representation that is decomposed from the unstructured sentence of the text to be proofread using natural language processing technology. It contains a core entity, a core relation, and a core attribute.

3. The semantic deep proofreading and human-computer verification method based on large model and knowledge graph tracing as described in claim 1, characterized in that, The method also includes a closed-loop feedback self-evolution step based on the human-machine review results. The closed-loop feedback self-evolution step includes: archiving the final decision made by a human expert on the output results of the semantic deep analysis step and the source tracing step in a structured manner; when the final decision is to confirm a new fact or to correct an existing fact, the confirmed or corrected fact is updated to the knowledge graph in the form of a triple, and the first sentence encoding model is called to recalculate the knowledge vector for the updated triple.

4. The semantic deep proofreading and human-computer verification method based on large model and knowledge graph tracing as described in claim 1, characterized in that, The method also includes a step of calculating implicit risk scores for high-risk anomaly assertions to determine their processing priority in the semantic deep analysis step; the rules for calculating the implicit risk scores are defined as follows: ,in, This is an implicit risk score; Gravitational gradient of knowledge fields that assert facts; Score the contextual credibility of factual assertions within the surrounding paragraphs of the text to be proofread; This is historical adoption rate data that characterizes the authority of the source associated with a factual assertion; , , The preset weighting coefficients are used, and the preset weighting coefficients are used... , , The sum of is 1.

5. The semantic deep proofreading and human-computer verification method based on large model and knowledge graph tracing as described in claim 1, characterized in that, Human-computer review is implemented through a human-computer interaction interface, which is configured as follows: for high-confidence assertions, the high-confidence assertion is displayed side by side with the original triples corresponding to the nearest neighbor knowledge vectors found in the knowledge field, and a confirmation option and a correction option are provided; for high-risk anomaly assertions, the high-risk anomaly assertion is displayed side by side with the correction suggestions generated by the semantic deep analysis step performed by the large model, and the source tracing evidence provided by the source tracing step performed by the knowledge graph, and an option to accept the correction suggestion, an option to reject the correction suggestion, and an option to edit the correction suggestion are provided.

6. The semantic deep proofreading and human-computer verification method based on large model and knowledge graph tracing as described in claim 1, characterized in that, Between the preset first threshold and the preset second threshold, a preset third threshold is also set, wherein the preset first threshold is less than the preset third threshold, and the preset third threshold is less than the preset second threshold. When the knowledge field gravity gradient is greater than a preset first threshold but less than a preset third threshold, the factual assertion is judged as a potentially risky assertion, and the semantic deep analysis step and the source tracing step are initiated with a lower priority than high-risk abnormal assertions.

7. The semantic deep proofreading and human-computer verification method based on large model and knowledge graph tracing as described in claim 1, characterized in that, The data sources for knowledge graphs include at least one of the following: authoritative documents, legal provisions, industry standards, and historical proofreading records.

8. The semantic deep proofreading and human-computer verification method based on large model and knowledge graph tracing as described in claim 1, characterized in that, The first sentence encoding model is the same as the first sentence encoding model. It is a distilled BERT model that is pre-trained on a medium-sized general corpus and then fine-tuned using domain corpus from the data source.

9. The semantic deep proofreading and human-computer verification method based on large model and knowledge graph tracing as described in claim 4, characterized in that, Contextual credibility score The generation steps include: based on a pre-defined semantic pattern library and context dependency analysis, determining whether the factual assertion is an emotion-dependent assertion; if so, invoking a contextual sentiment analysis model to analyze the contextual paragraph containing the factual assertion to extract its sentiment polarity, subjective intensity, and contextual consistency features, and generating a contextual credibility score based on the extracted features. The method also includes a preset weighting coefficient. , , The dynamic calibration process includes: prioritizing the processing of multiple high-risk anomaly assertions by a human expert and comparing the results with the system's implicit risk scores. The calculated priority ranking results are compared to determine a ranking consistency deviation; and based on this deviation, the preset weighting coefficients are adjusted. , , The value is adjusted.

10. A semantic deep proofreading and human-computer review system based on large model and knowledge graph tracing, characterized in that, The system is configured to perform the method of claim 1, the system comprising: A knowledge field construction module is configured to: call a first sentence encoding model to compute each triple in a knowledge graph consisting of multiple triples to generate a knowledge vector, store all generated knowledge vectors in a vector database, and build a vector index that supports nearest neighbor search for all knowledge vectors. This vector index is defined as a knowledge field. A pre-review module is configured to: invoke a sentence encoding model identical to the first sentence encoding model to compute a fact assertion extracted from the text to be reviewed, thereby generating an assertion vector; perform a nearest neighbor search in the knowledge field to find a knowledge vector that is geometrically closest to the assertion vector, and compute the Euclidean distance between the assertion vector and the found nearest neighbor knowledge vector, defining the Euclidean distance value as the knowledge field gravity gradient; A processing path decision module is configured to: execute differentiated processing paths based on the comparison results of the knowledge field gravity gradient; when the gradient value is less than a preset first threshold, the factual assertion is judged as a high-confidence assertion, and the system controls the system to skip the subsequent semantic deep analysis steps performed by the large model and the source tracing steps performed by the knowledge graph; when the gradient value is greater than a preset second threshold, the factual assertion is judged as a high-risk abnormal assertion, and the system controls the system to start the semantic deep analysis steps and the source tracing steps.

Images