Text factuality proofreading method, system, electronic device and medium

Abstract

The application provides a text factual correction method, system, electronic equipment and medium, the method comprises the following steps: obtaining the user input text to be corrected; based on the retrieval enhancement generation technology, recalling the reference information related to the text to be corrected from the pre-constructed multi-knowledge base; filling the text to be corrected and the reference information into the target template; using the trained large language model, according to the reference information filled into the target template, correcting the factual errors existing in the text to be corrected. The application uses the reference information supplemented by the multi-knowledge base and the excellent semantic understanding and analysis ability of the large language model to significantly improve the accuracy of text correction; through the cooperation of the knowledge resources of the multi-knowledge base, the accuracy and comprehensiveness of the fact retrieval are improved; get rid of the dependence on rules, can intelligently and accurately correct the text facts, effectively reduce the false positive rate, improve the efficiency and quality of text correction.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of text processing, and relates to a text factual correction method and system, an electronic device and a medium. BACKGROUND

[0002] With the development of Internet technology and the improvement of social informatization, the amount of text data is growing explosively. Text data, with its rich information content and wide application field, has become an important form for people to acquire knowledge, express opinions and convey information. However, incorrect factual statements can lead to misinformation.

[0003] Traditional text factual correction methods are mainly based on information extraction and rule matching. The specific operation process is to extract triples related to facts from the text, and then correct the extracted triples according to pre-set hard rules to determine whether the text content conforms to the facts.

[0004] However, this method, although simple and direct, has obvious limitations in practical application. First, the traditional rule-based method relies on hard rules, and the correction logic is complex and the development and maintenance cost is high, and it lacks generalization and is difficult to adapt to diversified scenarios. Second, the traditional method cannot utilize contextual semantic information and lacks semantic understanding and reasoning ability, resulting in a high false positive rate, especially when dealing with texts involving entity ambiguity, complex semantics or requiring deep reasoning, the error correction effect is not good. SUMMARY

[0005] The purpose of the present application is to provide a text factual correction method, system, electronic device and medium, which solves the technical problems of strong rule dependence, high maintenance cost and insufficient semantic understanding of the prior art.

[0006] In a first aspect, the present application provides a text factual correction method, comprising: obtaining a user inputted text to be corrected; based on a retrieval enhancement generation technology, recalling factual description information related to the text to be corrected from a pre-constructed multi-knowledge base as reference information; the multi-knowledge base includes a sentence and entity joint knowledge base, a triple knowledge base and a sentence vector knowledge base; according to the semantic type of the text to be corrected, selecting a target template from a plurality of pre-set text correction prompt word templates, and filling the text to be corrected and the reference information into the target template; using a trained large language model, according to the reference information filled into the target template, correcting the factual errors existing in the text to be corrected.

[0007] In an implementation form of the first aspect, the recalling, based on the retrieval enhancement generation technique, factual descriptive information related to the text to be corrected from the pre-constructed multi-knowledge base comprises: splitting the text to be corrected into a plurality of sentences to be corrected; extracting entities in each of the sentences to be corrected based on an entity extraction model to obtain a first entity extraction result; screening, according to the first entity extraction result, the sentences to be corrected including at least two entities as candidate sentences to be corrected; retrieving, based on a cross-knowledge base collaborative retrieval strategy, knowledge related to the candidate sentences to be corrected from the multi-knowledge base; and taking the retrieved knowledge as the factual descriptive information related to the text to be corrected.

[0008] In an implementation form of the first aspect, the constructing process of the multi-knowledge base comprises: splitting factual descriptive articles stored in a document resource base into a plurality of sentences; extracting entities in each of the sentences based on an entity extraction model to obtain a second entity extraction result; screening, based on the second entity extraction result, the sentences including at least two entities as candidate sentences; storing the candidate sentences and corresponding entities in a database to form a sentence-entity joint knowledge base; extracting relationship triplets of each of the candidate sentences based on a relationship extraction model; storing the extracted relationship triplets in the database to form a triplet knowledge base; and performing vectorization processing on each of the candidate sentences based on a vectorization model to obtain a sentence vector of each of the candidate sentences; storing the sentence vectors of each of the candidate sentences in the database to form a sentence vector knowledge base.

[0009] In an implementation form of the first aspect, the retrieving, based on the cross-knowledge base collaborative retrieval strategy, knowledge related to the candidate sentences to be corrected from the multi-knowledge base comprises: calculating semantic similarity between the candidate sentences to be corrected and sentence vectors of each of the candidate sentences in the sentence vector knowledge base; recalling, from the sentence-entity joint knowledge base, candidate sentences having high semantic similarity with the candidate sentences to be corrected; calculating structured similarity between the candidate sentences to be corrected and relationship triplets of each of the candidate sentences in the triplet knowledge base; and recalling, from the triplet knowledge base, relationship triplets having high structured feature similarity with the candidate sentences to be corrected.

[0010] In an implementation form of the first aspect, the correcting, by the large language model trained, a factual error existing in the text to be corrected according to the reference information filled into the target template comprises: performing semantic conflict detection on the text to be corrected and the reference information, and quantifying a semantic conflict degree between the text to be corrected and the reference information by a confidence score; if the confidence score is lower than a preset threshold, it is determined that there is a factual error in the text to be corrected; a conflict source is marked in the target template; the factual error existing in the text to be corrected is corrected, and a fact-corrected text is output; otherwise, it is determined that there is no factual error in the text to be corrected.

[0011] In an implementation form of the first aspect, after the factual error existing in the text to be corrected is corrected, the method further comprises: verifying whether an entity type in the fact-corrected text is within a specified range; if yes, it is determined that the verification is passed, and the fact-corrected text is taken as new reference information and is refilled into the target template for secondary correction; otherwise, it is determined that the verification fails, and interception processing is performed in a timely manner.

[0012] In an implementation form of the first aspect, the training process of the large language model comprises: pre-training the large language model based on a general domain dataset; and fine-tuning training the pre-trained large language model based on a vertical domain dataset.

[0013] In a second aspect, the present application provides a text factual correction system, comprising: a text acquisition module configured to acquire a text to be corrected input by a user; an information acquisition module configured to recall, based on a retrieval enhancement generation technology, factual description information related to the text to be corrected from a pre-constructed multi-element knowledge base as reference information; the multi-element knowledge base comprises a sentence and entity joint knowledge base, a triple knowledge base, and a sentence vector knowledge base; a template filling module configured to select a target template from a plurality of preset text correction prompt word templates according to a semantic type of the text to be corrected, and fill the text to be corrected and the reference information into the target template; and a fact comparison module configured to correct, by a large language model trained, a factual error existing in the text to be corrected according to the reference information filled into the target template.

[0014] In a third aspect, the present application provides an electronic device, comprising: a processor and a memory; the memory is configured to store a computer program; and the processor is configured to execute the computer program stored in the memory, so that the electronic device executes any of the above-mentioned methods.

[0015] In a fourth aspect, the present application provides a computer readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any of the above-mentioned methods.

[0016] As described above, the text factual correction method, system, electronic device and medium described in the present application have the following beneficial effects:

[0017] (1) The reference information supplemented by the multi-knowledge base and the excellent semantic understanding and analysis ability of the large language model are used to significantly improve the text correction accuracy;

[0018] (2) By coordinating the knowledge resources of the multi-knowledge base, the accuracy and comprehensiveness of fact retrieval are improved;

[0019] (3) The dependence on rules is eliminated, and the text factual correction can be more intelligent and accurate, effectively reducing the false positive rate and improving the efficiency and quality of correction. BRIEF DESCRIPTION OF DRAWINGS

[0020] Figure 1 A structural schematic diagram of a mobile terminal according to an embodiment of the present application is shown.

[0021] Figure 2 A flowchart of a text factual correction method according to an embodiment of the present application is shown.

[0022] Figure 3 A principle diagram of a retrieval enhancement generation technique according to an embodiment of the present application is shown.

[0023] Figure 4 A construction flowchart of a multi-knowledge base according to an embodiment of the present application is shown.

[0024] Figure 5 A flowchart of a text factual correction method according to an embodiment of the present application is shown.

[0025] Figure 6 A structural schematic diagram of a text factual correction system according to an embodiment of the present application is shown.

[0026] Figure 7 A structural schematic diagram of an electronic device according to an embodiment of the present application is shown. DETAILED DESCRIPTION

[0027] The embodiments of the present application will be described in detail below with specific reference to specific examples. Those skilled in the art can easily understand other advantages and effects of the present application from the content disclosed in the present specification. The present application can also be implemented or applied by different specific embodiments, and the details in the present specification can be modified or changed based on different views and applications without departing from the spirit of the present application. It should be noted that the following embodiments and features in the embodiments can be combined with each other without conflict.

[0028] It should be noted that the diagrams provided in the following embodiments only illustrate the basic concepts of the present application in a schematic manner, and only the components related to the present application are shown in the diagrams, not the number, shape and size of the components when actually implemented. The actual implementation of each component can be a random change, and the component layout pattern can be more complex.

[0029] In addition, the descriptions such as "first", "second" and the like in the present application are only for the purpose of description, and cannot be understood as indicating or implying the relative importance of the indicated technical features or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first", "second" can explicitly or implicitly include at least one of the features. In addition, the technical solutions of each embodiment can be combined with each other, but it must be based on the implementation of a person skilled in the art, and when the combination of technical solutions contradicts each other or cannot be implemented, it should be considered that the combination of technical solutions does not exist and is not within the protection scope required by the present application.

[0030] Text factual correction, as a key link of information quality control, aims to ensure that the text content is highly consistent with the objective facts through systematic checking and accurate correction. The existing text factual correction method mainly relies on structured hard rule base when performing rule matching. Although this verification mechanism based on fact database has clear judgment standard, it is easy to cause false positives in actual application scenarios due to the deviation of triple extraction. For example, when correcting the text "The poet of the Yuan Dynasty, Sallar, commented on Li Bai's "Quiet Night Thoughts", the ideal state should accurately analyze the correct triples such as "Yuan Dynasty-Poet-Sallar", "Tang Dynasty-Poet-Li Bai", and "Quiet Night Thoughts-Author-Li Bai". However, due to the limitations of natural language processing technology, errors may occur. For example, the dynasty attribute is incorrectly bound as "Yuan Dynasty-Poet-Li Bai", or the "Yuan Dynasty-Commentator- "Quiet Night Thoughts" "such abnormal triples that violate the space-time logic are generated during the analysis of the comment relationship.

[0031] The following embodiments of the present application provide a text factual correction method, system, electronic device and medium. The present application utilizes the reference information supplemented by the multi-knowledge base and the excellent semantic understanding and analysis capability of the large language model, realizes the significant improvement of the text correction accuracy; through the cooperation of the knowledge resources of the multi-knowledge base, the accuracy and comprehensiveness of the fact retrieval are improved; get rid of the dependence on rules, can more intelligently and accurately perform text factual correction, effectively reduce the false positive rate, improve the efficiency and quality of correction.

[0032] The text factual correction method provided by the embodiments of the present application can run in mobile terminals, computer terminals and the like. Taking the running on the mobile terminal as an example, Figure 1A hardware structure block diagram of a mobile terminal for the text factual correction method is shown in Figure 1 The mobile terminal can include a processor, which can be a central processing unit, and a memory for storing data. Figure 1 The mobile terminal in the above embodiment is only used for example and does not limit the specific structure of the mobile terminal.

[0033] Optionally, the mobile terminal can further include a communication transmission device and an input and output device.

[0034] Optionally, the memory can be used to store computer programs, such as software programs of application software and modules. The memory can include a high-speed random access memory and can further include a non-volatile memory, such as one or more magnetic storage devices, flash memories, or other non-volatile solid-state memories. In some examples, the memory can further include a memory remotely arranged relative to the processor, which can be connected to the mobile terminal through a network. Examples of the network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0035] Optionally, the communication transmission device can be used to receive or send data via a network, which can include a wireless network provided by a communication provider of the mobile terminal. The communication transmission device can include a NIC (Network Interface Controller) which can be connected to other network devices through a base station so as to communicate with the Internet.

[0036] The technical solutions in the embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0037] Please refer to Figure 2 , which shows a flowchart of the text factual correction method in an embodiment of the present application. As shown in Figure 2 The present application provides a text factual correction method including the following steps S100 to S400.

[0038] In step S100, a user-inputted text to be corrected is acquired.

[0039] In the present embodiment, the input source of the text to be corrected can include user-created texts, network-crawled contents, academic literature fragments, and the like. Such texts usually contain explicit factual errors, implicit logical contradictions, and ambiguities caused by ambiguous expressions.

[0040] Specifically, the text to be corrected can contain the following six typical factual biases:

[0041] Error in time and space coordinates. For example, "Zheng He's expedition to Africa reached the Cape of Good Hope."

[0042] Misplacement of personage. For example, “Su Shi and Lu You jointly advocated literary innovation.”

[0043] Inaccurate numerical precision. For example, “The total length of the Yangtze River is 5400 kilometers.”

[0044] Reversal of event order. For example, “The An Lushan Rebellion broke out before the establishment of the Tang Dynasty.”

[0045] Conceptual category confusion. For example, classifying photosynthesis as a chemical reaction rather than a biological process.

[0046] Misattribution of sources. For example, the Analects records the thoughts of Mencius.

[0047] At step S200, based on the retrieval augmented generation technology, factual description information related to the text to be corrected is recalled from a pre-constructed multi-knowledge base as reference information; the multi-knowledge base includes a sentence and entity joint knowledge base, a triple knowledge base, and a sentence vector knowledge base.

[0048] Retrieval-Augmented Generation (RAG) is a model that combines retrieval and generation techniques. In this embodiment, RAG provides rich background knowledge and contextual information for the text correction process, improving the quality and accuracy of the text correction results and effectively preventing the occurrence of knowledge hallucinations.

[0049] Please refer to Figure 3 , which shows the principle diagram of the retrieval augmented generation technology in an embodiment of the present application. As shown in Figure 3 , based on the retrieval augmented generation technology, factual description information related to the text to be corrected is recalled from a pre-constructed multi-knowledge base, including the following steps S201 to S205.

[0050] At step S201, the text to be corrected is split into several sentences to be corrected.

[0051] Specifically, the splitting operation of the text to be corrected can be completed with the help of a sentence segmentation model. For example, the sentence segmentation model used can be constructed based on punctuation rules, or implemented using the Natural Language Processing (NLP) tool Spacy. This model can reasonably disassemble complex long sentences into key semantic units while preserving the complete subject-predicate-object structure of the sentence.

[0052] At step S202, based on an entity extraction model, entities in each of the sentences to be corrected are extracted to obtain a first entity extraction result.

[0053] In this embodiment, the entity extraction model has multi-type entity extraction capability, and can extract a wide range of entities, including not only common types such as person entities, work entities, and time entities, but also domain-specific entities. The domain-specific entities here specifically refer to entity content with specific domain attributes, such as historical events and scientific and technical terms.

[0054] In step S203, according to the first entity extraction result, the sentence to be corrected including at least two entities is screened out as a candidate sentence to be corrected.

[0055] In this embodiment, the filtering operation can filter out semantic units without entities or containing only single entities. For example, for non-factual related expressions such as "Today is really a good weather", since such sentences lack verifiable factual relevance, filtering out such sentences can significantly improve retrieval efficiency and reduce invalid knowledge base query operations.

[0056] In step S204, based on the cross-knowledge base collaborative retrieval strategy, knowledge related to the candidate sentence to be corrected is retrieved from the multi-knowledge base.

[0057] In an embodiment of the present application, based on the cross-knowledge base collaborative retrieval strategy, the knowledge related to the candidate sentence to be corrected is retrieved from the multi-knowledge base, including: calculating the semantic similarity between the candidate sentence to be corrected and the sentence vector of each candidate sentence in the sentence vector knowledge base; recalling the candidate sentence with high semantic similarity from the sentence and entity joint knowledge base; calculating the structured similarity between the candidate sentence to be corrected and the relationship triple of each candidate sentence in the triple knowledge base; and recalling the relationship triple with high structured feature similarity from the triple knowledge base.

[0058] In step S205, the retrieved knowledge is used as factual description information related to the text to be corrected.

[0059] In this implementation, the limitations of a single knowledge base are broken, and the accuracy and comprehensiveness of fact retrieval are improved by collaborative knowledge resources of multi-knowledge base.

[0060] In an embodiment of the present application, based on the retrieval enhancement generation technology, the factual description information related to the text to be corrected is recalled from the pre-constructed multi-knowledge base, which further includes step S206.

[0061] In step S206, the retrieved factual description information related to the text to be corrected is cleaned, filtered, and integrated.

[0062] Please refer to Figure 4, shows the construction flowchart of the multi-element library in an embodiment of the present application. As shown in Figure 4 The construction process of the multi-element library includes: splitting factual description articles stored in the document repository into a plurality of sentences; extracting entities in each of the sentences based on an entity extraction model to obtain a second entity extraction result; screening the sentences including at least two entities as candidate sentences based on the second entity extraction result; storing the candidate sentences and the corresponding entities in a database to form a sentence-entity joint knowledge base; extracting relationship triples of each of the candidate sentences based on a relationship extraction model; storing the extracted relationship triples in the database to form a triple knowledge base; and performing vectorization processing on each of the candidate sentences based on a vectorization model to obtain a sentence vector of each of the candidate sentences; and storing the sentence vectors of each of the candidate sentences in the database to form a sentence vector knowledge base.

[0063] In actual application, the sentence-entity joint knowledge base can adopt an LMDB library, the triple knowledge base can adopt a Neo4j library, and the sentence vector knowledge base can adopt a Milvus library. The LMDB library can provide accurate entity-sentence pairs to provide accurate entity and corresponding sentence association information for fact verification; the Neo4j library has the function of supporting relationship reasoning, which helps to mine the potential logical relationship between entities, thereby more comprehensively verifying the facts; the Milvus library can realize semantic generalization, which can to some extent cope with the changes and expansion of the semantic level, and ensure effective fact verification under different semantic expressions. The three knowledge bases cooperate with each other, which can cover various complex scenarios in the fact verification process.

[0064] It should be noted that the multi-element knowledge base involved in the present embodiment can complete the construction work in an offline state, so as to effectively avoid the overhead brought by real-time processing and improve the efficiency and stability of the entire system.

[0065] In the present implementation, by introducing the retrieval enhancement generation technology and the multi-element knowledge base, the complex maintenance work of the traditional rule base is avoided. The multi-element knowledge base covers multi-dimensional fact information from fine-grained entities to coarse-grained semantics, can quickly recall reference information related to the text to be corrected through semantic retrieval, reduces the dependence on customized rules, improves the generalization of the method, and reduces the development and maintenance cost.

[0066] In step S300, a target template is selected from a plurality of preset text correction prompt word templates according to the semantic type of the text to be corrected, and the text to be corrected and the reference information are filled into the target template.

[0067] Specifically, the semantic categories of the text to be corrected include time category, location category, and character relationship category. Each semantic category corresponds to a predefined text proofreading prompt word template.

[0068] The present application can intelligently match or dynamically select the most suitable prompt word template according to the semantic type of the text to be corrected, breaking the limitations of traditional fixed templates, achieving problem-oriented template adaptation, greatly improving the accuracy of text proofreading, and optimizing the user experience.

[0069] At step S400, using the trained large language model, the factual errors in the text to be corrected are proofread according to the reference information filled into the target template.

[0070] Large Language Model (LLM) refers to a Transformer architecture model containing tens of billions (or more) of parameters. Its large parameter size enables it to learn complex patterns and semantic relationships in massive amounts of text data, thereby possessing strong semantic understanding and reasoning capabilities. For example, when processing natural language text, it can understand the logical relationships between words and sentences, accurately capture the meaning of the context, and further analyze and process the text content in depth.

[0071] By combining the semantic understanding and reasoning capabilities of the large language model, the present application can fully utilize contextual information for error correction. For example, when filling in the target template, the model can infer a more contextually appropriate correction result based on the reference information and the semantic type of the text to be corrected. Compared to traditional rule-based methods, this semantic reasoning-based error correction method can effectively address entity ambiguity (such as "Li Bai" referring to different characters) and complex semantic errors (such as factual contradictions in negative sentences), significantly reducing false positive rates.

[0072] Large language models are essentially generative models. When given a paragraph as input, it will generate a piece of text based on the learned knowledge and patterns. However, without reference information input, the content generated directly by the large language model may not be very accurate. This is because its generation results mainly rely on its own prior knowledge, which may be affected by the limitations or ambiguity of the training data. For example, on some issues involving specific fields or time sensitivity, the model's own memory may not be able to provide accurate answers.

[0073] To make the large language model generate content more accurately, the present application fills the reference information in the target template. The filling of reference information provides a more specific and targeted context for the large language model, guiding the large language model to generate answers that are more in line with the actual situation. For example, in the field of literary knowledge, if the large language model is directly asked "Li Bai is a poet of the Song Dynasty", due to the possibility of knowledge confusion or inaccurate memory of some details, it may give a correct answer. When the reference information "Li Bai is known to be of the Tang Dynasty" is supplemented and the same question is asked again, the model will make a judgment based on the supplemented reference information, thereby giving a correct answer.

[0074] In addition, due to the stability and authority of the reference information, even if the text to be corrected has some ambiguity, non-standard language expression or is interfered by a certain degree of noise, as long as its core factual content is consistent with the reference information, the model can accurately judge, thereby improving the robustness and anti-interference ability of the model. For example, there may be some colloquial expressions or a small number of typos in an article, but as long as the facts conveyed are consistent with the reference information, the model will not incorrectly determine that the text has factual errors due to these minor problems, ensuring that the model can make accurate judgments in different situations.

[0075] In an embodiment of the present application, the training process of the large language model includes: pre-training the large language model based on a general domain dataset; fine-tuning the pre-trained large language model based on a vertical domain dataset.

[0076] The general domain dataset in this embodiment generally covers a wide range of topics and rich language expression forms, such as large-scale network texts, books, news reports, etc. The large language model automatically mines statistical rules and semantic information in language through unsupervised learning.

[0077] The vertical domain dataset refers to a focus on a specific discipline or application scenario, for example, in the medical field, the vertical domain dataset may include a large number of medical literature, case reports, professional term explanations, etc.; in the legal field, there will be relevant data such as legal regulations, judicial case analysis, etc. The vertical domain data can provide more specific and accurate knowledge information for the model, making the model more familiar with specific concepts, terms and facts in the field.

[0078] In this implementation, a two-stage training paradigm is adopted. By pre-training the model on a large amount of data, the world knowledge can be compressed into the parameters of the large language model, so that the large language model has good general basic ability and downstream task migration ability; by fine-tuning the model on the vertical domain data, the parameters of the large language model can be best suited to solve the factual correction task based on reference information in the current application scenario.

[0079] Referring to Figure 5 , a flowchart showing the text factual correction method described in an embodiment of the present application in another embodiment is shown.

[0080] As Figure 5 shown, using the trained large language model, according to the reference information filled into the target template, the factual error existing in the text to be corrected is corrected, which includes: performing semantic conflict detection on the text to be corrected and the reference information, and quantifying the semantic conflict degree between the text to be corrected and the reference information through a confidence score; if the confidence score is lower than a preset threshold, it is determined that there is a factual error in the text to be corrected; the conflict source is marked in the target template; the factual error existing in the text to be corrected is corrected, and the text after factual correction is output; otherwise, it is determined that there is no factual error in the text to be corrected.

[0081] In the use of reference information, the large language model shows its wide applicability and strong generalization ability. It can not only refer to structured triple information, but also flexibly deal with the input of related sentences. In contrast, the triple extraction model in the prior art is relatively rigid and needs to define a strict framework in advance to determine which rules can be extracted. If some key information is not defined in the framework, such as the dynasty in which the poet lived or the representative works, the model cannot extract relevant triples to support factual correction. The large language model is different. As long as the reference information contains relevant content, whether these contents are defined in the framework in advance, the model can make accurate judgments based on its strong semantic understanding and reasoning ability, thereby providing users with more comprehensive and accurate factual correction services.

[0082] In an embodiment of the present application, after correcting the factual error existing in the text to be corrected, it further includes: verifying whether the entity type in the text after factual correction is within the specified range; if yes, it is determined that the verification is passed, and the text after factual correction is taken as new reference information and refilled into the target template for secondary correction; otherwise, it is determined that the verification fails, and timely interception processing is performed.

[0083] In the present implementation, through effective verification and interception mechanism, the error judgment caused by the mismatch of entity type is avoided, the accuracy and reliability of the correction result are guaranteed, the stability and credibility of the entire correction system are improved, and more accurate and high-quality text correction services are provided for users.

[0084] It should be noted that the protection scope of the text factual correction method described in the embodiments of the present application is not limited to the execution order of the steps listed in the embodiments. Any scheme achieved by increasing, reducing or replacing steps of the prior art according to the principle of the present application is included in the protection scope of the present application.

[0085] Referring to Figure 6 , a structural schematic diagram of the text factual correction system described in the embodiments of the present application in an embodiment is shown. As Figure 6 indicated, the embodiments of the present application provide a text factual correction system, which includes a text acquisition module, an information acquisition module, a template filling module and a fact comparison module.

[0086] The text acquisition module is configured to acquire a user-inputted text to be corrected.

[0087] The information acquisition module is configured to recall, based on a retrieval enhancement generation technology, factual description information related to the text to be corrected from a pre-constructed multi-knowledge base as reference information; the multi-knowledge base includes a sentence and entity joint knowledge base, a triple knowledge base and a sentence vector knowledge base.

[0088] The template filling module is configured to select a target template from a plurality of preset text correction prompt word templates according to a semantic type of the text to be corrected, and fill the text to be corrected and the reference information into the target template.

[0089] The fact comparison module is configured to use a large language model after training to correct factual errors existing in the text to be corrected according to the reference information filled into the target template.

[0090] It should be noted that the structure and principle of the text acquisition module, the information acquisition module, the template filling module and the fact comparison module in the embodiments of the present application correspond to the steps in the text factual correction method described above one by one, and therefore will not be described here.

[0091] The text factual correction system provided in the embodiments of the present application can implement the text factual correction method described in the present application, but the implementation device of the text factual correction method described in the present application includes but is not limited to the structure of the text factual correction system listed in the embodiments. Any structure deformation and replacement of the prior art according to the principle of the present application is included in the protection scope of the present application.

[0092] Referring to Figure 7 , a structural schematic diagram of the electronic device described in the embodiments of the present application in an embodiment is shown.

[0093] As Figure 7 indicated, the embodiments of the present application provide an electronic device, which includes a processor and a memory.

[0094] The memory is configured to store a computer program.

[0095] The processor is configured to execute the computer program stored in the memory, so that the electronic device executes the method described in any one of the preceding embodiments.

[0096] Preferably, the processor can be a general processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; or a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA) or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component. The memory can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as a Static Random Access Memory (SRAM), an Electrically Erasable Programmable Read-Only Memory (EEPROM), an Erasable Programmable Read-Only Memory (EPROM), a Programmable Read-Only Memory (PROM), a Read-Only Memory (ROM), a magnetic storage, a flash memory, a magnetic disk or an optical disk.

[0097] The embodiment also includes one or more of a multimedia component, an input / output (I / O) interface, and a communication component.

[0098] The multimedia component can include a screen and audio components. The screen can be, for example, a touch screen, and the audio components can be for output and / or input of audio signals. For example, the audio components can include a microphone for receiving an external audio signal. The received audio signal can be further stored in the memory or transmitted through the communication component. The audio components also include at least one speaker for outputting audio signals. The I / O interface provides an interface between the processor and other interface modules, which can be a keyboard, a mouse, a button, etc. These buttons can be virtual buttons or physical buttons. The communication component is used for wired or wireless communication between the timer and other devices. Wireless communication, such as Wi-Fi, Bluetooth, Near Field Communication (NFC), 2G, 3G or 4G, or a combination of one or more of them, so the corresponding communication component can include a Wi-Fi module, a Bluetooth module, an NFC module.

[0099] In several embodiments provided in the present application, it should be understood that the disclosed system, device or method can be implemented in other ways. For example, the above-described device embodiments are only illustrative, for example, the division of modules / units is only a logical function division, and actual implementation can have another division manner, for example, a plurality of modules or units can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the displayed or discussed modules can be indirect coupling or communication connection through some interfaces, devices or modules, which can be electrical, mechanical or other forms.

[0100] The modules / units described as separate components can or can not be physically separated, and the components shown as modules / units can or can not be physical modules, i.e. they can be located in one place or distributed on multiple network units. Part or all of the modules / units can be selected according to actual needs to achieve the purpose of the embodiments of the present application. For example, the functional modules / units in each embodiment of the present application can be integrated into a processing module, or each module / unit can be physically separated, or two or more modules / units can be integrated into one module / unit.

[0101] Those skilled in the art should further understand that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be realized in electronic hardware, computer software or a combination of both. In order to clearly illustrate the interchangeability of hardware and software, the components and steps of each example have been described in the above description in a general manner. Whether the functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. A person skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0102] The embodiments of the present application also provide a computer readable storage medium, which stores a computer program. The program is executed by a processor to implement the method described in any of the above embodiments. Those skilled in the art can understand that all or part of the steps of the method described in the above embodiments can be instructed by a program to complete by a processor. The program can be stored in a computer readable storage medium. The storage medium is a non-transitory medium, such as a random access memory, a read only memory, a flash memory, a hard disk, a solid state disk, a magnetic tape, a floppy disk, an optical disc and any combination thereof. The storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server, data center and the like, which includes one or more available media sets. The available medium can be a magnetic medium (for example, a floppy disk, a hard disk, a magnetic tape), an optical medium (for example, a digital video disc (DVD)) or a semiconductor medium (for example, a solid state disk (SSD)) and the like.

[0103] The embodiments of the present application can also provide a computer program product, which includes one or more computer instructions. When the computer instructions are loaded and executed on a computing device, all or part of the processes or functions described in the embodiments of the present application are generated. The computer instructions can be stored in a computer readable storage medium or transmitted from one computer readable storage medium to another, for example, the computer instructions can be transmitted from one website, computer or data center to another website, computer or data center through a wired (for example, coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (for example, infrared, wireless, microwave and the like) manner.

[0104] The computer program product is executed by a computer, and the computer executes the method in the foregoing method embodiment. The computer program product can be a software installation package, and in a case where the foregoing method needs to be used, the computer program product can be downloaded and executed on the computer.

[0105] The descriptions of the corresponding flows or structures of the various drawings are each focused on, and parts not described in detail in a certain flow or structure can be referred to the related descriptions of other flows or structures.

[0106] The above embodiments only exemplarily illustrate the principles and effects of the present application, and are not used to limit the present application. Any person skilled in the art can modify or change the above embodiments without departing from the spirit and category of the present application. Therefore, all equivalent modifications or changes completed by those skilled in the art without departing from the spirit and technical thought of the present application should be covered by the claims of the present application.

Claims

1. A method for factual proofreading of a text, characterized in that: include: Get the text to be corrected entered by the user; Based on the retrieval enhancement generation technology, factual descriptive information related to the text to be corrected is recalled from a pre-built multi-knowledge base as reference information; the multi-knowledge base includes a sentence and entity joint knowledge base, a triple knowledge base, and a sentence vector knowledge base; Selecting a target template from a plurality of preset text proofreading prompt word templates according to the semantic type of the text to be corrected, and filling the text to be corrected and the reference information into the target template; Using the trained large language model, based on the reference information filled into the target template, proofread the factual errors in the text to be corrected; Based on the retrieval enhancement generation technology, the factual descriptive information related to the text to be corrected is recalled from the pre-built multi-knowledge base, including: Splitting the text to be corrected into a plurality of sentences to be corrected; Extracting entities from each of the sentences to be corrected based on the entity extraction model to obtain a first entity extraction result; According to the first entity extraction result, the sentences to be corrected that include at least two entities are screened out as candidate sentences to be corrected; Retrieving knowledge related to the candidate sentence to be corrected from the multi-knowledge base based on a cross-knowledge base collaborative retrieval strategy; Using the retrieved knowledge as factual descriptive information related to the text to be corrected; Based on the cross-knowledge base collaborative retrieval strategy, the knowledge related to the candidate sentence to be corrected is retrieved from the multi-knowledge base, including: Calculating the semantic similarity between the candidate sentence to be corrected and the sentence vectors of each candidate sentence in the sentence vector knowledge base; Recalling candidate sentences with high semantic similarity to the candidate sentence to be corrected from the sentence and entity joint knowledge base; Calculating the structural similarity between the candidate sentence to be corrected and the relation triples of each candidate sentence in the triple knowledge base; Recalling relation triplets having high structural feature similarity with the candidate sentence to be corrected from the triple knowledge base.

2. The method according to claim 1, characterized in that The construction process of the multivariate library includes: Split the factual description articles stored in the document resource library into several clauses; Based on the entity extraction model, extract the entities in each of the sentences to obtain a second entity extraction result; Based on the second entity extraction result, screening out the sentences including at least two entities as candidate sentences; Storing the candidate sentences and corresponding entities in a database to form a joint knowledge base of sentences and entities; Extracting relation triples of each candidate sentence based on a relation extraction model; Storing the extracted relationship triples in a database to form the triple knowledge base; Based on the vectorization model, vectorization processing is performed on each candidate sentence to obtain a sentence vector of each candidate sentence; The sentence vector of each candidate sentence is stored in a database to form the sentence vector knowledge base.

3. The method according to claim 1, characterized in that Using the trained large language model and based on the reference information filled into the target template, proofreading the factual errors in the text to be corrected includes: Performing semantic conflict detection on the text to be corrected and the reference information, and quantifying the degree of semantic conflict between the text to be corrected and the reference information through a confidence score; If the confidence score is lower than a preset threshold, it is determined that there are factual errors in the text to be corrected; the source of the conflict is marked in the target template; the factual errors in the text to be corrected are corrected, and a factually proofread text is output; Otherwise, it is deemed that there are no factual errors in the text to be corrected.

4. The method according to claim 1, wherein After proofreading the factual errors in the text to be corrected, the method further includes: Verify whether the entity types in the proofread text of the facts are within the prescribed range; If so, the verification is deemed to be passed, and the fact-proofed text is used as new reference information and re-filled into the target template for secondary proofreading; Otherwise, the verification will be deemed to have failed and the request will be intercepted and processed promptly.

5. The method according to claim 1, wherein The training process of the large language model includes: Pre-training the large language model based on a general domain dataset; Based on the vertical field dataset, the pre-trained large language model is fine-tuned.

6. A text factual proofreading system, characterized in that: include: The text acquisition module is used to obtain the text to be corrected input by the user; An information acquisition module is used to recall factual descriptive information related to the text to be corrected from a pre-built multi-knowledge base based on retrieval enhancement generation technology as reference information; the multi-knowledge base includes a sentence and entity joint knowledge base, a triple knowledge base, and a sentence vector knowledge base; A template filling module is used to select a target template from a plurality of preset text proofreading prompt word templates according to the semantic type of the text to be corrected, and fill the text to be corrected and the reference information into the target template; A fact comparison module is used to use the trained large language model to proofread factual errors in the text to be corrected based on the reference information filled in the target template; Based on the retrieval enhancement generation technology, the factual descriptive information related to the text to be corrected is recalled from the pre-built multi-knowledge base, including: Splitting the text to be corrected into a plurality of sentences to be corrected; Extracting entities from each of the sentences to be corrected based on the entity extraction model to obtain a first entity extraction result; According to the first entity extraction result, the sentences to be corrected that include at least two entities are screened out as candidate sentences to be corrected; Retrieving knowledge related to the candidate sentence to be corrected from the multi-knowledge base based on a cross-knowledge base collaborative retrieval strategy; Using the retrieved knowledge as factual descriptive information related to the text to be corrected; Based on the cross-knowledge base collaborative retrieval strategy, the knowledge related to the candidate sentence to be corrected is retrieved from the multi-knowledge base, including: Calculating the semantic similarity between the candidate sentence to be corrected and the sentence vectors of each candidate sentence in the sentence vector knowledge base; Recalling candidate sentences with high semantic similarity to the candidate sentence to be corrected from the sentence and entity joint knowledge base; Calculating the structural similarity between the candidate sentence to be corrected and the relation triples of each candidate sentence in the triple knowledge base; Recalling relation triplets having high structural feature similarity with the candidate sentence to be corrected from the triple knowledge base.

7. An electronic device, characterized in that: include: processor and memory; The memory is used to store computer programs; The processor is configured to execute the computer program stored in the memory, so as to enable the electronic device to perform the method according to any one of claims 1 to 5.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 5 is implemented.

Images