Text processing method, text processing model training method, article correction method, computing device, storage medium and computer program product

Abstract

Embodiments of the present specification provide a text processing method, a text processing model training method, an article correction method, a computing device, a storage medium and a computer program product wherein the text processing method comprises: determining a to-be-processed text, and determining a reference correction text associated with the to-be-processed text, the confidence coefficient of the reference correction text is greater than that of the to-be-processed text; inputting the to-be-processed text and the reference correction text into a text processing model to obtain a target text corresponding to the to-be-processed text; wherein the text processing model is obtained by training according to a label text, a label correction text associated with the label text and a sample text, and the sample text is constructed according to the label text and the label correction text. The text processing model can learn more errors possibly occurring in the text, so that the error correction capability of the text processing model is improved, and the processing performance of the text processing model is ensured.

Description

Technical Field

[0001] The embodiments of this specification relate to the field of computer technology, and in particular to a text processing method, a text processing model training method, an article correction method, a computing device, a storage medium, and a computer program product. Background Art

[0002] With the development of computer technology, artificial intelligence has been widely used in many fields. For example, in the field of article creation, artificial intelligence can also be used to complete article creation. For example, large models or neural network models can be used to create articles based on input content. However, due to the sequence decoding method of the model in the output stage, it is randomly generated according to probability. The model focuses on adjacent text and ignores contextual information, which creates knowledge illusions. As a result, the articles generated by the model may contain factual errors or may be inconsistent with the input content. The accuracy of the model's generation results is poor, which affects the user experience of using artificial intelligence. Based on this, an effective technical solution is urgently needed to solve the above problems. Summary of the Invention

[0003] In view of this, embodiments of this specification provide a text processing method. One or more embodiments of this specification also relate to a text processing device, a text processing model training method, a text processing model training device, an article correction method, an article correction device, a computing device, a computer-readable storage medium, and a computer program product to address technical deficiencies in the prior art.

[0004] According to a first aspect of the embodiments of this specification, a text processing method is provided, including:

[0005] Determining a text to be processed, and determining a reference revised text associated with the text to be processed, wherein the confidence level of the reference revised text is greater than the confidence level of the text to be processed;

[0006] Inputting the text to be processed and the reference revised text into a text processing model to obtain a target text corresponding to the text to be processed;

[0007] The text processing model is trained based on the label text, the label correction text associated with the label text, and the sample text, and the sample text is constructed based on the label text and the label correction text.

[0008] According to a second aspect of the embodiments of this specification, there is provided a text processing apparatus, comprising:

[0009] a determination module configured to determine a text to be processed and determine a reference revised text associated with the text to be processed, wherein the confidence level of the reference revised text is greater than the confidence level of the text to be processed;

[0010] An input module is configured to input the text to be processed and the reference revised text into a text processing model to obtain a target text corresponding to the text to be processed;

[0011] The text processing model is trained based on the label text, the label correction text associated with the label text, and the sample text, and the sample text is constructed based on the label text and the label correction text.

[0012] According to a third aspect of the embodiments of this specification, a text processing model training method is provided, which is applied to a cloud-side device, including:

[0013] Determining a label text and a label correction text associated with the label text, and constructing a sample text based on the label text and the label correction text;

[0014] The text processing model is trained according to the label text, the sample text and the label correction text until a text processing model that meets the training stop condition is obtained.

[0015] According to a fourth aspect of the embodiments of this specification, a text processing model training apparatus is provided, which is applied to a cloud-side device, including:

[0016] a determination module configured to determine a label text and a label correction text associated with the label text, and construct a sample text based on the label text and the label correction text;

[0017] The training module is configured to train the text processing model according to the label text, the sample text and the label correction text until a text processing model that meets the training stop condition is obtained.

[0018] According to a fifth aspect of the embodiments of this specification, a method for revising an article is provided, which is applied to a cloud-side device and includes:

[0019] receiving an article to be revised, and determining a reference revision text associated with the article to be revised, wherein the confidence level of the reference revision text is greater than the confidence level of the article to be revised;

[0020] Inputting the article to be revised and the reference revised text into a text processing model to obtain a target article corresponding to the article to be revised;

[0021] The text processing model is trained based on the label text, the label correction text associated with the label text, and the sample text, and the sample text is constructed based on the label text and the label correction text.

[0022] According to a sixth aspect of the embodiments of this specification, there is provided an article correction apparatus, applied to a cloud-side device, comprising:

[0023] a receiving module configured to receive an article to be revised and determine a reference revision text associated with the article to be revised, wherein the confidence level of the reference revision text is greater than the confidence level of the article to be revised;

[0024] An input module is configured to input the article to be revised and the reference revised text into a text processing model to obtain a target article corresponding to the article to be revised;

[0025] The text processing model is trained based on the label text, the label correction text associated with the label text, and the sample text, and the sample text is constructed based on the label text and the label correction text.

[0026] According to a seventh aspect of the embodiments of this specification, a computing device is provided, including:

[0027] memory and processor;

[0028] The memory is used to store computer programs / instructions, and the processor is used to execute the computer programs / instructions. When the computer programs / instructions are executed by the processor, the steps of the above method are implemented.

[0029] According to an eighth aspect of the embodiments of this specification, a computer-readable storage medium is provided, which stores a computer program / instruction, and the computer program / instruction implements the steps of the above method when executed by a processor.

[0030] According to a ninth aspect of the embodiments of this specification, a computer program product is provided, comprising a computer program / instruction, which implements the steps of the above method when executed by a processor.

[0031] One embodiment of the present specification provides a text processing method, comprising: determining a text to be processed, and determining a reference corrected text associated with the text to be processed, wherein the confidence of the reference corrected text is greater than the confidence of the text to be processed; inputting the text to be processed and the reference corrected text into a text processing model to obtain a target text corresponding to the text to be processed; wherein the text processing model is trained based on a label text, a label corrected text associated with the label text, and a sample text, and the sample text is constructed based on the label text and the label corrected text.

[0032] In the above method, the text to be processed is corrected using a text processing model and reference correction text with a greater confidence level than the text to be processed to obtain the target text. This allows for post-verification of the model-generated articles, further improving the accuracy of the model-generated articles. Furthermore, by constructing sample text based on labeled text and labeled correction text, and training the text processing model based on the sample text, labeled text, and labeled correction text, the text processing model can learn more about possible errors in the text, thereby improving the text processing model's error correction capabilities and ensuring its processing performance. This results in a more accurate target text obtained after processing by the text processing model, thereby ensuring a better user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 This is a schematic diagram of an application scenario of a text processing method provided by an embodiment of this specification;

[0034] Figure 2 is a flowchart of a text processing method provided by one embodiment of this specification;

[0035] Figure 3 is a schematic diagram of a text processing model in a text processing method provided in one embodiment of this specification;

[0036] Figure 4 This is a schematic diagram of constructing a sample text in a text processing method provided in one embodiment of this specification;

[0037] Figure 5 This is a structural diagram of a text processing device provided by an embodiment of this specification;

[0038] Figure 6 This is a flowchart of a text processing model training method provided by one embodiment of this specification;

[0039] Figure 7 This is a structural diagram of a text processing model training device provided by one embodiment of this specification;

[0040] Figure 8 This is a flow chart of a method for revising an article provided in one embodiment of this specification;

[0041] Figure 9 This is a schematic diagram of the structure of an article correction method provided by an embodiment of this specification;

[0042] Figure 10 This is a structural block diagram of a computing device provided by one embodiment of this specification. DETAILED DESCRIPTION

[0043] The following description sets forth many specific details to facilitate a thorough understanding of this specification. However, this specification can be implemented in many other ways than those described herein, and those skilled in the art can make similar generalizations without violating the scope of this specification. Therefore, this specification is not limited to the specific implementations disclosed below.

[0044] The terms used in one or more embodiments of this specification are for the purpose of describing specific embodiments only and are not intended to limit one or more embodiments of this specification. The singular forms "a," "the," and "the" used in one or more embodiments of this specification and the appended claims are also intended to include plural forms unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items.

[0045] It should be understood that although the terms first, second, etc. may be used to describe various information in one or more embodiments of this specification, such information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of one or more embodiments of this specification, the first may also be referred to as the second, and similarly, the second may also be referred to as the first. Depending on the context, the word "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determining".

[0046] In addition, it should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in one or more embodiments of this specification are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0047] In one or more embodiments of this specification, a large model refers to a deep learning model with large-scale model parameters, typically containing hundreds of millions, tens of billions, hundreds of billions, trillions, or even more than ten trillion model parameters. A large model can also be called a cornerstone model / foundation model. It is pre-trained on a large-scale unlabeled corpus to produce a pre-trained model with more than 100 million parameters. This model can adapt to a wide range of downstream tasks and has good generalization capabilities, such as a large language model (LLM) and a multi-modal pre-training model.

[0048] When large models are used in practice, only a small number of samples are needed to fine-tune the pre-trained model and it can be applied to different tasks. Large models can be widely used in natural language processing (NLP), computer vision and other fields. Specifically, they can be applied to computer vision tasks such as visual question answering (VQA), image caption (IC), and image generation, as well as natural language processing tasks such as text-based sentiment classification, text summary generation, and machine translation. The main application scenarios of large models include digital assistants, intelligent robots, search, online education, office software, e-commerce, and intelligent design.

[0049] First, the terms involved in one or more embodiments of this specification are explained.

[0050] Large model long article writing: refers to giving the large model article generation instructions, so that the large model generates longer articles, such as more than 1,000 words.

[0051] Large-model knowledge hallucination: A hallucination is a false perception that feels real. Large-model generation tasks sometimes generate text that is unfaithful to the input or factually incorrect. This undesirable phenomenon shares similar characteristics with this type of hallucination in psychology.

[0052] Illusion correction: also called revise, detects whether there is knowledge illusion in the content generated by the large model. If so, it modifies it and outputs the correct and complete content.

[0053] Fact conflict: When generating responses, large models may conflict with the input information. For example, in summary questions, the information in the summary and document may be inconsistent.

[0054] Creating something out of nothing: When generating responses, large models output additional information that is not reflected in the input, such as email addresses, phone numbers, and addresses, and it is difficult to verify their authenticity.

[0055] In this specification, a text processing method is provided. This specification also relates to a text processing device, a text processing model training method, a text processing model training device, an article correction method, an article correction device, a computing device, a computer-readable storage medium and a computer program product, which are described in detail one by one in the following embodiments.

[0056] See also Figure 1 , Figure 1 A schematic diagram of an application scenario of a text processing method provided according to an embodiment of this specification is shown.

[0057] like Figure 1 As shown, Figure 1 The system includes a terminal-side device 102 and a cloud-side device 104, wherein the cloud-side device 104 is deployed with a text generation model and a text processing model.

[0058] In specific implementation, the text processing method provided by the embodiments of this specification can be applied to the field of article creation. In the field of article creation, a user can input prompt information through the display interface of the terminal device 102. The prompt information can be the summary of the article the user wants to create. The terminal device 102 can generate a text processing request based on the input operation. The text processing request carries the prompt information. The terminal device 102 sends the text processing request to the cloud device 104. The cloud device 104 can call the text generation model to generate a text to be processed based on the prompt information. The text to be processed can be an article generated by the text generation model. Afterwards, the cloud device 102 can call the text processing model to determine the reference correction text corresponding to the text to be processed. The reference correction text can be, for example, the data referenced by the text generation model when generating the text to be processed. The text processing model can be used to correct the text to be processed based on the reference correction text to obtain a corrected target text. The target text can be, for example, an article generated by the text generation model with hallucination correction. The cloud device 104 can send the target text to the terminal device 102, and the terminal device 102 can display it to the user through the display interface.

[0059] Among them, when the text processing model is being trained, in order to ensure the error correction capability of the text processing model, sample text can be constructed based on the label text and the label correction text associated with the label text, and the text processing model can be trained based on the sample text, label text and label correction text to ensure the processing performance and stability of the text processing model during text correction.

[0060] The end-side device 102 may include a browser, an APP (Application), or a web application such as an H5 (Hyper Text Markup Language 5, version 5 of Hypertext Markup Language) application, or a light application (also known as a mini-program, a lightweight application) or a cloud application. The end-side device may be based on the software development kit (SDK) of the corresponding service provided by the server, such as developed based on the real-time communication (RTC) SDK. The end-side device can be deployed in an electronic device and needs to rely on the device to run or certain APPs in the device to run. The electronic device may have a display screen and support information browsing, such as a personal mobile terminal such as a mobile phone, tablet computer, personal computer, etc. Various other types of applications can usually be configured in the electronic device, such as human-computer dialogue applications, model training applications, text processing applications, web browser applications, shopping applications, search applications, instant messaging tools, email clients, social platform software, etc.

[0061] The cloud-side device 104 can be understood as a server that provides various services, including physical servers and cloud servers. For example, a server that provides communication services to multiple clients, a server that supports background training for models used on clients, and a server that processes data sent by clients. It should be noted that the cloud-side device 104 can be implemented as a distributed server cluster consisting of multiple servers, or as a single server. The cloud-side device 104 can also be a server in a distributed system, or a server integrated with a blockchain. The cloud-side device 104 can also be a cloud server for basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), and big data and artificial intelligence platforms, or an intelligent cloud computing server or intelligent cloud host with artificial intelligence technology.

[0062] It is worth noting that the text processing method provided in the embodiments of this specification can be executed by the cloud-side device 104. In other embodiments of this specification, the text processing model can be deployed in the terminal-side device 102, so that the terminal-side device 102 can also have similar functions as the cloud-side device 104, thereby executing the text processing method provided in the embodiments of this specification; in other embodiments, the text processing method provided in the embodiments of this specification can also be jointly executed by the terminal-side device 102 and the cloud-side device 104.

[0063] See also Figure 2 , Figure 2 A flowchart of a text processing method provided according to an embodiment of the present specification is shown, which specifically includes the following steps.

[0064] Step 202: Determine a text to be processed, and determine a reference correction text associated with the text to be processed, wherein the confidence level of the reference correction text is greater than the confidence level of the text to be processed.

[0065] Specifically, the text processing method provided in the embodiments of this specification can be used in the field of text correction. For example, for an article created by a large model based on user input, the text processing method can correct the article and correct any factual errors that may exist in the article to ensure the accuracy of the article creation.

[0066] Among them, the text to be processed can be understood as a text that needs to be revised, such as an article generated by a large model based on user input, or other text that needs to be revised. The reference revised text can be understood as real and correct text content. For example, the text to be processed includes "the distance between city A and city B is 100 kilometers", then the reference revised text associated with the text to be processed can include "the distance between city A and city B is 120 kilometers", and the 120 kilometers is real data. Since the content included in the text to be processed and the reference revised text are inconsistent, it means that the text content included in the text to be processed is wrong and needs to be revised. Or, the text to be processed includes "user C took the train to city A on October 20", then the reference revised text associated with the text to be processed can include "user C took the train to city A on October 20", and since the content included in the text to be processed and the reference revised text are consistent, it means that the text content included in the text to be processed is correct. Confidence can be used to indicate the degree of accuracy of a text. If the confidence of the reference corrected text is greater than that of the text to be processed, it can be understood that the degree of accuracy of the reference corrected text is greater than that of the text to be processed. In other words, the content in the reference corrected text can be understood as correct content, while the content in the text to be processed can be understood as content that is uncertain whether it is correct and needs to be tested and corrected.

[0067] Based on this, the text that needs to be corrected can be determined, and the correct text content associated with the text that needs to be corrected can be determined.

[0068] For example, an article that needs to be corrected can be identified, along with the correct textual content (ie, facts) associated with the article.

[0069] In one embodiment of this specification, the text quantity of the text to be processed is greater than a preset quantity threshold. The text quantity of the text to be processed can be understood as the number of words in the text to be processed. For example, the text to be processed can be a paragraph of 100 words. It is understandable that as the text quantity of the text to be processed increases, the possibility of errors also increases. Based on this, the text processing method provided in the embodiment of this specification can correct the text to be processed whose text quantity is greater than a preset quantity threshold. The preset quantity threshold can be, for example, 1,000 words, 5,000 words, etc., which is not limited in the embodiment of this specification.

[0070] In practical applications, the reference correction text associated with the text to be processed can be the text content used as the basis for generating the text to be processed. For example, when an article is generated by an article generation model, the article generation model can complete the creation of the article based on the text content input by the user by retrieving relevant information about the text content. In this case, the information referenced by the article generation model when creating the article can be used as the reference correction text to determine whether the article output by the article generation model is consistent with the information referenced during creation. The specific implementation method is as follows:

[0071] Determining the text to be processed includes:

[0072] Generate text using a text generation model based on prompt information and references to generate text to be processed corresponding to the prompt information, wherein the text to be processed has a larger number of texts than the text of the prompt information;

[0073] Accordingly, determining the reference revised text associated with the text to be processed includes:

[0074] The reference generated text is used as the reference revised text associated with the text to be processed.

[0075] The text generation model can be understood as a natural language model or a large model used for article creation. The prompt information can be understood as the text content input by the user into the text generation model, and this prompt information can serve as the basis for the text generation model to create the article. The reference generated text can be understood as the information referenced by the text generation model when creating the article, and this reference generated text can be retrieved from the knowledge base based on the prompt information. The to-be-processed text can be understood as the article created by the text generation model based on the prompt information. For example, the prompt information can be "Introduce City A", then the reference generated text can be relevant information about City A retrieved from the knowledge base, such as City A's history, attractions, etc., and the to-be-processed text corresponding to the prompt information can be an article introducing City A generated by the text generation model. The text length of the to-be-processed text can be understood as the number of words in the text to be processed. The text length of the prompt information can be understood as the number of words in the prompt information. Therefore, the number of words in the to-be-processed text can be greater than the number of words in the prompt information.

[0076] Based on this, the text generation model can be used to retrieve the reference generated text associated with the prompt information input by the user, and generate the to-be-processed text corresponding to the prompt information based on the reference generated text, and use the reference generated text as the reference corrected text associated with the to-be-processed text.

[0077] In addition, the prompt information can also be used as a reference correction text associated with the text to be processed, which can be used to subsequently detect whether the text to be processed generated by the text generation model is consistent with the content of the prompt information input by the user, further ensuring the accuracy of the text to be processed.

[0078] In summary, by using the data referenced by the text generation model when generating the text to be processed (i.e., the reference generated text) as the reference correction text associated with the text to be processed, it is convenient to subsequently detect whether the text to be processed generated by the text generation model is consistent with the relevant data referenced when generating the text to be processed, and further correct the text to be processed when inconsistency is determined to achieve the accuracy of the target text obtained after correction.

[0079] In another embodiment of the present specification, for the reference correction text, the retrieved facts obtained through network search based on the sentences describing objective facts in the text to be processed can be used as the reference correction text. The specific implementation method is as follows:

[0080] The step of determining a reference revised text associated with the text to be processed includes:

[0081] Determining a text segment to be processed contained in the text to be processed, wherein the text segment to be processed is text content describing objective facts;

[0082] Searching a knowledge base to obtain a first reference revised text corresponding to the text segment to be processed;

[0083] The first reference revised text corresponding to the text segment to be processed is determined as the reference revised text associated with the text to be processed.

[0084] Among them, the text segment to be processed can be understood as a text sentence in the text to be processed that describes an objective fact. For example, "The distance between city A and city B is 100 kilometers" is a text sentence that describes the objective fact of the distance between city A and city B. It can be understood that the objective fact described in the text segment to be processed may be true or false. The text processing method provided in the embodiment of this specification is to detect these text sentences describing objective facts in the text to be processed, detect whether the objective facts described are correct, and modify incorrect objective facts. Then, the first reference correction text corresponding to the text segment to be processed can be understood as the true objective fact described by the text to be processed. For example, the first reference correction text corresponding to the text segment to be processed "The distance between city A and city B is 100 kilometers" can be "The distance between city A and city B is 50 kilometers". This first reference correction text is a true objective fact.

[0085] In actual applications, the text to be processed may include multiple text segments to be processed. Correspondingly, the reference revised text associated with the text to be processed also includes multiple first reference revised texts.

[0086] Based on this, multiple text segments to be processed that describe objective facts contained in the text to be processed can be determined, and a search can be performed from the knowledge base to obtain the first reference correction text corresponding to each text segment to be processed, and the first reference correction text corresponding to each text segment to be processed can be determined as the reference correction text associated with the text to be processed.

[0087] In specific implementations, a natural language model can be used to extract text segments describing objective facts from the text to be processed. Alternatively, fact extraction rules can be used to extract text segments describing objective facts from the text to be processed. Fact extraction rules can be understood as rules abstracted from the factual elements of objective facts. Factual elements can be understood as the necessary conditions for the establishment of facts, such as subject, time, and space.

[0088] Continuing with the above example, we can determine sentences 1, 2, and 3 that describe objective facts contained in the article that needs to be revised, and through online search, determine the correct text content 1 corresponding to sentence 1, the correct text content 2 corresponding to sentence 2, and the correct text content 3 corresponding to sentence 3, and determine the correct text contents 1, 2, and 3 as the correct text contents associated with the article that needs to be revised.

[0089] In summary, by searching from the knowledge base, the retrieved facts are used as the correct text content associated with the article that needs to be revised, which facilitates subsequent detection of whether the objective facts described in the article that needs to be revised are accurate, and further ensures the accuracy of the article revision.

[0090] Step 204: Inputting the text to be processed and the reference revised text into a text processing model to obtain a target text corresponding to the text to be processed;

[0091] The text processing model is trained based on the label text, the label correction text associated with the label text, and the sample text, and the sample text is constructed based on the label text and the label correction text.

[0092] Specifically, after determining the text to be processed and the reference correction text associated with the text to be processed, the text to be processed and the reference correction text can be input into the text processing model, and the text processing model can correct the text to be processed according to the reference correction text to obtain the target text corresponding to the text to be processed.

[0093] Among them, the text processing model can be understood as a model for article correction. The model can be, for example, a neural network model, a large model, etc., and this specification embodiment does not limit this. The target text can be understood as the text obtained after the text to be processed is corrected. The label text, the label correction text and the sample text can be used as a set of training data to train the text processing model. The label text can be understood as a real and correct text, the label correction text can be understood as the text in the label text that correctly describes the objective facts, and the sample text can be understood as the text obtained by constructing the label text and containing incorrect text content (i.e., containing knowledge illusions). For example, the label text can be an article, and the objective facts described in the article are accurate. The label correction text can be the text in the article that correctly describes the objective facts, and the sample text can be the article obtained after the objective facts described in the article are modified, so as to train the knowledge illusion correction ability of the text processing model.

[0094] In a specific implementation, the text processing model includes a detection unit, a correction unit and a replacement unit;

[0095] Accordingly, the step of inputting the text to be processed and the reference revised text into a text processing model to obtain a target text corresponding to the text to be processed includes:

[0096] Inputting the text to be processed and the reference revised text into the text processing model, and using the detection unit to detect the text to be processed based on the reference revised text to obtain a target text segment to be processed;

[0097] Using the correction unit, the target text segment to be processed is corrected according to a first reference correction text corresponding to the target text segment to be processed to obtain a target text segment, wherein the first reference correction text is determined based on the reference correction text;

[0098] The replacement unit is used to replace the target text segment in the text to be processed according to the target text segment, so as to obtain the target text corresponding to the text to be processed.

[0099] Among them, the target text segment to be processed can be understood as the text segment to be processed included in the text to be processed that is inconsistent with the first reference correction text, that is, the target text segment to be processed is the text segment to be processed that has errors and needs to be corrected. For example, the text to be processed includes text segments 1, 2 and 3 to be processed, which correspond to the first reference correction texts 4, 5 and 6 respectively. Among them, the text segment to be processed 1 is "Bridge A is 2 kilometers long", and the first reference correction text 4 is "Bridge A is 3.5 kilometers long". The two are inconsistent, then the text segment to be processed 1 is the target text segment to be processed. The text segment to be processed can be understood as a sentence in the text to be processed. The target text segment can be understood as the correct text segment obtained after correcting the text segment to be processed.

[0100] Specifically, the text to be processed and the reference corrected text can be input into the detection unit of the text processing model, and the detection unit can be used to detect the text to be processed based on the reference corrected text. The text segment to be processed with an inconsistent detection result can be determined as a target text segment to be processed that has errors and needs to be corrected. The target text segment to be processed can be input into the correction unit, and the correction unit can be used to correct the target text segment to be processed to obtain a target text segment. The target text segment can be input into the replacement unit, and the replacement unit can be used to replace the target text segment to be processed in the text to be processed based on the target text segment to achieve original text replacement, thereby obtaining the target text corresponding to the text to be processed.

[0101] In practical applications, see Figure 3 , Figure 3 FIG1 shows a schematic diagram of a text processing model in a text processing method provided according to an embodiment of this specification. Figure 3As shown, the text processing model includes a hallucination detection unit (i.e., a detection unit), a hallucination rewriting unit (i.e., a correction unit) and an original text replacement unit (i.e., a replacement unit). The article with hallucinations (i.e., the text to be processed) and the retrieval facts (i.e., the reference correction text) can be input into the hallucination detection unit of the text processing model to detect the hallucination sentences that are inconsistent in the article and the retrieval facts, and obtain the hallucination sentences 1, hallucination sentences 2 and hallucination sentences 3 output by the hallucination detection unit. The hallucination sentences are the target text segments to be processed, and the hallucination sentences 1, hallucination sentences 2 and hallucination sentences 3 are input into the hallucination rewriting unit, and all the hallucination sentences are modified in turn to obtain the correct sentences 1, correct sentences 2 and correct sentences 3 output by the hallucination rewriting unit. The correct sentences are the target text segments, and the correct sentences 1, correct sentences 2 and correct sentences 3 are input into the original text replacement unit, and the hallucination sentences in the original text are replaced one by one with the correct sentences, and the corrected article (i.e., the target text) output by the original text replacement unit is obtained.

[0102] In summary, after the correction unit is used to replace each incorrect sentence in the article that needs to be corrected, the error detection granularity and rewriting granularity are limited to the sentence level, ensuring that each error is independent of each other. Only the incorrect sentence is rewritten each time instead of the entire article, improving the stability of the rewriting.

[0103] Furthermore, the detecting unit is used to detect the text to be processed according to the reference revised text to obtain a target text segment to be processed, including:

[0104] Using the detection unit, matching the reference revised text and the text to be processed to obtain a matching result;

[0105] A target text segment to be processed in the text to be processed is determined according to the matching result.

[0106] The matching result can be used to illustrate the consistency between the reference revised text and the text to be processed. For example, the matching result can be comparison information between the reference revised text and the text to be processed. Based on this comparison information, the different text content between the reference revised text and the text to be processed can be determined and used as the target text segment to be processed.

[0107] Specifically, the detection unit can be used to match the reference corrected text and the text to be processed. Specifically, the text segment to be processed included in the text to be processed and the first reference corrected text included in the reference corrected text can be matched, and the text segment to be processed with inconsistent matching results can be determined as the target text segment to be processed.

[0108] In summary, by directly introducing retrieval information (i.e., reference correction text) during detection, possible errors can be identified and the accuracy of monitoring can be improved.

[0109] In practical applications, the training steps of the text processing model include:

[0110] Determining a label text and a label correction text associated with the label text, and constructing a sample text based on the label text and the label correction text;

[0111] The text processing model is trained according to the label text, the sample text and the label correction text until a text processing model that meets the training stop condition is obtained.

[0112] The training stop condition can be understood as the model loss value reaching a preset loss value threshold or the number of training times reaching a preset number threshold.

[0113] Based on this, the label text and the label correction text associated with the label text can be determined, and sample text can be constructed based on the label text and the label correction text. The text processing model can be trained based on the label text, sample text and label correction text until a text processing model that meets the training stop conditions is obtained.

[0114] In specific implementation, the sample text and label-corrected text can be input into the text processing model to obtain the predicted text output by the text processing model, the model loss value is calculated based on the predicted text and the label text, and the text processing model is trained based on the model loss value until a text processing model that meets the training stop conditions is obtained.

[0115] In summary, by constructing sample text, it is easier to train the text processing model later and further improve the correction performance of the text processing model.

[0116] In a specific implementation, constructing a sample text based on the label text and the label correction text includes:

[0117] Determining a label text segment in the label text;

[0118] Aligning the label text segment and the label correction text, and determining a first label correction text corresponding to the label text segment according to the alignment result;

[0119] Processing the label text segment according to a preset construction rule associated with the first label-corrected text to construct a sample text segment, wherein the preset construction rule is used to process the label text segment;

[0120] The label text segment in the label text is replaced according to the sample text segment to obtain the sample text.

[0121] The label text can be understood as an article that accurately describes objective facts. The label text segment can be understood as any text segment in the label text that describes objective facts, such as a sentence in the label text segment that describes objective facts. The preset construction rule associated with the first label correction text can be understood as, when the label text segment is processed according to the preset construction rule to construct a sample text segment, making the constructed sample text segment inconsistent with the first label correction text, thereby enabling the text processing model to learn error correction capabilities.

[0122] It can be understood that the method for determining the label correction text is similar to the method for determining the aforementioned reference correction text, and will not be repeated here.

[0123] Based on this, the label text segment that describes the objective facts in the label text can be extracted, the label text segment and the label correction text can be aligned, the first label correction text corresponding to the label text segment can be determined according to the alignment result, and the label text segment can be processed according to the preset construction rules associated with the first label correction text to construct a sample text segment, so that the constructed sample text segment is inconsistent with the first label correction text, and the label text segment in the label text is replaced according to the constructed sample text segment to obtain the sample text.

[0124] In summary, by constructing sample text according to preset construction rules, it is convenient to subsequently train the text processing model based on the sample text, so that the text processing model can learn error correction capabilities.

[0125] In actual applications, the big model can be called through corresponding instructions to construct sample text segments through the big model. The specific implementation method is as follows:

[0126] The step of processing the label text segment according to the preset construction rule associated with the first label-corrected text to construct a sample text segment includes:

[0127] Responding to a construction instruction sent by the client, wherein the construction instruction is generated according to construction information input by a model trainer based on a display interface of the client, the construction information including the preset construction rule and the label text segment;

[0128] According to the construction instruction, a sample construction model is called, and the label text segment is processed according to the preset construction rule associated with the first label correction text to generate a sample text segment.

[0129] Among them, the sample construction model can be understood as a model used for sample construction, such as a large model, a natural language model, a neural network model, etc., and the embodiments of this specification do not limit this.

[0130] Specifically, the model trainer can input the construction information on the client's display interface. The client can generate construction instructions based on the construction information and send the construction instructions to the server. The server calls the sample construction model based on the construction instructions, processes the label text segment through preset construction rules, and generates a sample text segment.

[0131] For example, construction instructions include direct semantic error construction instructions and summary semantic error construction instructions. A direct semantic error construction instruction, for example, could be "Rewrite the following content, replacing nouns, verbs, adjectives, numerical terms, etc., so that it contains 1-2 complex factual errors: {sentence content}." The sentence content can be understood as the content of the labeled text segment. A direct semantic error construction instruction could also be "Rewrite the following content, replacing nouns, so that it contains 1-2 complex factual errors: {sentence content}." A direct semantic error construction instruction could also be "Rewrite the following content, replacing verbs, so that it contains 1-2 complex factual errors: {sentence content}." Specifically, the above three direct semantic error construction instructions can be randomly called for error construction. For example, a summary semantic error construction instruction could be, "Consider the following content succinctly. {Sentence content}\nPlease write a correct summary first. Then, based on the correct summary, modify the nouns, verbs, adjectives, and numerical terms to make it a summary with 1-2 factual errors. \nPlease output the correct summary in the following format: xxx\nIncorrect summary: xxx." Another example of a summary semantic error construction instruction could be, "Consider the following content succinctly. {Sentence content}\nPlease write a correct summary first. Then, based on the correct summary, modify the nouns to make it a summary with 1-2 factual errors. \nPlease output the correct summary in the following format: xxx\nIncorrect summary: xxx." Another example of a summary semantic error construction instruction could be, "Consider the following content succinctly. {Sentence content}\nPlease write a correct summary first. Then, based on the correct summary, modify the verbs to make it a summary with 1-2 factual errors. \nPlease output the correct summary in the following format: xxx\nIncorrect summary: xxx." The above three summary semantic error construction instructions can be called randomly for error construction.

[0132] In one embodiment of the present specification, the preset construction rule includes a first preset construction rule, wherein the first preset construction rule is used to process numbers in the tag text segment;

[0133] The step of processing the label text segment according to the preset construction rule associated with the first label-corrected text to construct a sample text segment includes:

[0134] In the case where it is determined that the label text segment includes numbers, the numbers included in the label text segment are modified according to the first preset construction rule, and the modified label text segment is determined as the sample text segment.

[0135] The numbers included in the label text segment may include numerical values, dates and other numbers. For example, a distance of 100 kilometers, a length of 20 meters, a duration of 10 days, and completion on October 2 are all numbers.

[0136] Specifically, when constructing a sample text segment, if numbers are included in the label text segment, the numbers included in the label text segment can be modified according to the first preset construction rule, and the modified label text segment can be determined as the sample text segment. It can be understood that the modified numbers included in the sample text segment are erroneous content.

[0137] In summary, when constructing sample text segments, modifying numbers can train the numerical error correction capability of the text processing model.

[0138] In one embodiment of the present specification, the preset construction rule includes a second preset construction rule, wherein the second preset construction rule is used to process the fact element in the label text segment;

[0139] The step of processing the label text segment according to the preset construction rule associated with the first label-corrected text to construct a sample text segment includes:

[0140] According to the second preset construction rule, the factual elements in the label text segment are modified, and the modified label text segment is determined as the sample text segment.

[0141] Among them, factual elements may include but are not limited to events, the location of events, the time of events, people, and other elements.

[0142] Specifically, when constructing a sample text segment, the factual elements contained in the label text segment can also be modified according to the second preset construction rule, and the modified label text segment can be determined as the sample text segment. It can be understood that the modified factual elements included in the sample text segment are erroneous content.

[0143] For example, for the labeled text segment "Character A and Character B met for the first time on Bridge C," the factual elements included in the labeled text segment are Character A, Character B, and Bridge C. According to the second preset construction rule, Character A can be modified to Character D, then the constructed sample text segment is "Character D and Character B met for the first time on Bridge C." Alternatively, Bridge C can be modified to Store E, then the constructed sample text segment is "Character A and Character B met for the first time in Store E."

[0144] It is understandable that one or more factual elements included in the label text segment may be modified according to the second preset construction rule, and the modified label text segment may be determined as the sample text segment.

[0145] In summary, when constructing sample text segments, the direct semantic error correction capability of the text processing model can be trained by modifying factual elements.

[0146] In one embodiment of the present specification, the preset construction rule includes a third preset construction rule, wherein the third preset construction rule is used to process the semantic text of the label text segment;

[0147] The step of processing the label text segment according to the preset construction rule associated with the first label-corrected text to construct a sample text segment includes:

[0148] According to the third preset construction rule, the semantic text of the label text segment is determined, the semantic text is modified, and the modified semantic text is determined as the sample text segment.

[0149] Specifically, the label text segment may be summarized according to the third preset construction rule to obtain the semantic text of the label text segment, and the semantic text may be modified, and the modified semantic text may be determined as the sample text segment.

[0150] In summary, when constructing sample text segments, modifying semantics can train the text processing model's ability to correct semantic errors.

[0151] In addition, when constructing a sample text, multiple types of errors can be covered in the sample text, that is, a sample text can be constructed according to the first preset construction rule, the second preset construction rule, and the third preset construction rule. Moreover, for the same label text and label correction text, multiple hallucination combinations can be randomly constructed. For example, a sample text can be constructed according to the first preset construction rule and the second preset construction rule, a sample text can be constructed according to the second preset construction rule and the third preset construction rule, and a sample text can also be constructed according to the first preset construction rule and the third preset construction rule. This embodiment of the present specification does not limit this.

[0152] In summary, by covering various factual errors such as numerical dates (i.e., sample text segments constructed according to the first preset construction rule), direct semantics (i.e., sample text segments constructed according to the second preset construction rule), and summary semantics (i.e., sample text segments constructed according to the third preset construction rule) when constructing training data, the diverse construction of training data is achieved, which can improve the recall rate and stability of monitoring.

[0153] In one embodiment of the present specification, the label correction text may be incorrectly constructed, that is, constructed out of nothing, so that the constructed sample text contains errors that are created out of nothing. The specific implementation method is as follows:

[0154] After constructing the sample text according to the label text and the label correction text, the method further includes:

[0155] Determine a plurality of label text segments in the label text, and determine a first label correction text corresponding to each label text segment in the plurality of label text segments;

[0156] According to a preset deletion rule, the first label correction texts corresponding to a preset number of label text segments are deleted from the multiple label text segments to obtain remaining label correction texts, wherein the preset deletion rule is used to delete the first label correction texts corresponding to the label text segments;

[0157] The step of training the text processing model according to the label text, the sample text, and the label-corrected text until a text processing model that satisfies a training stop condition is obtained includes:

[0158] The text processing model is trained according to the label text, the sample text and the remaining label correction text until a text processing model that meets the training stop condition is obtained.

[0159] The preset deletion rule may be, for example, deleting a preset number of first label correction texts.

[0160] Specifically, multiple label text segments in the label text can be determined, and the first label correction text corresponding to each label text segment can be determined. According to the preset deletion rules, a preset number of first label correction texts can be deleted from the first label correction texts corresponding to the multiple label text segments to obtain the remaining label correction texts. The text processing model can be trained based on the label text, sample text and the remaining label correction text until a text processing model that meets the training stop conditions is obtained.

[0161] For example, you can determine sentence 1, sentence 2, and sentence 3 in the label text, and determine the correct sentence 1 corresponding to sentence 1, the correct sentence 2 corresponding to sentence 2, and the correct sentence 3 corresponding to sentence 3. According to the preset deletion rule, delete one sentence from the three correct sentences. For example, delete sentence 1, then correct sentences 2 and 3 are the remaining label correction texts.

[0162] In summary, by deleting correct sentences, the text processing model can be trained to correct errors from scratch. That is, in subsequent applications, if the text processing model detects text content included in the processed text that does not exist in the reference correction text, it may indicate that the text generation model has randomly generated this text content without any reference to the search results. The text processing model can then be equipped to correct this randomly generated text content.

[0163] In practical applications, see Figure 4 , Figure 4 FIG1 shows a schematic diagram of constructing a sample text in a text processing method provided according to an embodiment of this specification. Figure 4 As shown, the sentences in the article (i.e., the labeled text segments in the labeled text) can be aligned with the search facts (i.e., the labeled correction text). Based on the alignment results, aligned sentences containing date and numerical values ​​and randomly aligned sentences can be found. The corresponding content in the search facts for each sentence in the article describing objective facts is found. If a date or numerical value exists in the aligned sentence, the date or numerical value is replaced with an error (i.e., a date numerical error), thereby constructing an incorrect sentence with a date numerical error. Factual elements such as places, people, and events in the sentence can also be replaced with an error, thereby constructing an incorrect sentence with a direct semantic error. Sentences can also be summarized first, and then the summarized sentence is modified with a direct semantic error, thereby constructing an incorrect sentence with a summary semantic error. Furthermore, by covering as many types of hallucinations (i.e., errors) as possible within the same article, such as constructing multiple types of hallucinations such as date numerical errors, direct semantic errors, and summary semantic errors within the same article, internal diversity in sample construction can be achieved. For the same article and retrieval facts, a variety of different hallucination combinations (i.e., error combinations) are randomly constructed according to the above different construction methods. For example, 2-3 different hallucination combinations are constructed for the same article and retrieval facts. For example, for the same article and retrieval facts, three hallucination combinations are constructed: date numerical error and direct semantic error, date numerical error and summary semantic error, and direct semantic error and summary semantic error, respectively, to achieve external diversity in sample construction. In addition, the constructed error sentence can be directly replaced with the correct sentence in the original text, thereby creating a factual conflict error in the article. It is also possible to directly delete the corresponding content of a sentence in the retrieval fact, thereby creating an error in the article that is not true. Based on the above error construction, training data (i.e., sample text) is constructed.

[0164] In addition, the method of using a text generation model to generate text according to prompt information and a reference, before generating the to-be-processed text corresponding to the prompt information, further includes:

[0165] receiving a text processing request sent by a client, wherein the text processing request carries prompt information and is generated based on an input operation of a user through a display interface of the client;

[0166] After obtaining the target text corresponding to the to-be-processed text, the method further includes:

[0167] The target text is sent to the client and displayed to the user through the display interface.

[0168] Specifically, in the process of making corrections using the text processing model, the user can input prompt information through the client's display interface. The client can generate a text processing request carrying the prompt information based on the user's input operation, and send the text processing request to the server. After the server receives the text processing request, it can use the text generation model to generate a to-be-processed text corresponding to the prompt information, and correct the to-be-processed text through the text processing model to obtain the target text. After obtaining the target text, the server can send the target text to the client and display it to the user through the client's display interface to meet the user's text generation needs.

[0169] In summary, the above method uses a text processing model and reference correction text with a greater confidence level than the text to be processed to correct the text to obtain the target text, thereby achieving post-verification of the model-generated articles and further improving the accuracy of the model-generated articles. Furthermore, by constructing sample text based on labeled text and labeled correction text, and training the text processing model based on the sample text, labeled text, and labeled correction text, the text processing model can learn more about possible errors in the text, thereby improving the text processing model's error correction capabilities, ensuring the text processing model's processing performance, and making the target text obtained after processing by the text processing model more accurate, thereby ensuring a better user experience.

[0170] Corresponding to the above method embodiment, this specification also provides a text processing device embodiment, Figure 5 FIG1 shows a schematic diagram of the structure of a text processing device provided by an embodiment of this specification. Figure 5 As shown, the device includes:

[0171] The determination module 502 is configured to determine a text to be processed and determine a reference correction text associated with the text to be processed, wherein the confidence level of the reference correction text is greater than the confidence level of the text to be processed;

[0172] An input module 504 is configured to input the text to be processed and the reference revised text into a text processing model to obtain a target text corresponding to the text to be processed;

[0173] The text processing model is trained based on the label text, the label correction text associated with the label text, and the sample text, and the sample text is constructed based on the label text and the label correction text.

[0174] In an optional embodiment, the text processing model includes a detection unit, a correction unit, and a replacement unit;

[0175] Accordingly, the input module 504 is further configured to:

[0176] Inputting the text to be processed and the reference revised text into the text processing model, and using the detection unit to detect the text to be processed based on the reference revised text to obtain a target text segment to be processed;

[0177] Using the correction unit, the target text segment to be processed is corrected according to a first reference correction text corresponding to the target text segment to be processed to obtain a target text segment, wherein the first reference correction text is determined based on the reference correction text;

[0178] The replacement unit is used to replace the target text segment in the text to be processed according to the target text segment, so as to obtain the target text corresponding to the text to be processed.

[0179] In an optional embodiment, the input module 504 is further configured to:

[0180] Using the detection unit, matching the reference revised text and the text to be processed to obtain a matching result;

[0181] A target text segment to be processed in the text to be processed is determined according to the matching result.

[0182] In an optional embodiment, the determining module 502 is further configured to:

[0183] Generate text using a text generation model based on prompt information and references to generate text to be processed corresponding to the prompt information, wherein the text to be processed has a larger number of texts than the text of the prompt information;

[0184] Accordingly, determining the reference revised text associated with the text to be processed includes:

[0185] The reference generated text is used as the reference revised text associated with the text to be processed.

[0186] In an optional embodiment, the determining module 502 is further configured to:

[0187] Determining a text segment to be processed contained in the text to be processed, wherein the text segment to be processed is text content describing objective facts;

[0188] Searching a knowledge base to obtain a first reference revised text corresponding to the text segment to be processed;

[0189] The first reference revised text corresponding to the text segment to be processed is determined as the reference revised text associated with the text to be processed.

[0190] In an optional embodiment, the apparatus further includes a training module configured to:

[0191] Determining a label text and a label correction text associated with the label text, and constructing a sample text based on the label text and the label correction text;

[0192] The text processing model is trained according to the label text, the sample text and the label correction text until a text processing model that meets the training stop condition is obtained.

[0193] In an optional embodiment, the training module is further configured to:

[0194] Determining a label text segment in the label text;

[0195] Aligning the label text segment and the label correction text, and determining a first label correction text corresponding to the label text segment according to the alignment result;

[0196] Processing the label text segment according to a preset construction rule associated with the first label-corrected text to construct a sample text segment, wherein the preset construction rule is used to process the label text segment;

[0197] The label text segment in the label text is replaced according to the sample text segment to obtain the sample text.

[0198] In an optional embodiment, the training module is further configured to:

[0199] Responding to a construction instruction sent by the client, wherein the construction instruction is generated according to construction information input by a model trainer based on a display interface of the client, the construction information including the preset construction rule and the label text segment;

[0200] According to the construction instruction, a sample construction model is called, and the label text segment is processed according to the preset construction rule associated with the first label correction text to generate a sample text segment.

[0201] In an optional embodiment, the preset construction rule includes a first preset construction rule, wherein the first preset construction rule is used to process numbers in the label text segment;

[0202] The training module is further configured to:

[0203] In the case where it is determined that the label text segment includes numbers, the numbers included in the label text segment are modified according to the first preset construction rule, and the modified label text segment is determined as the sample text segment.

[0204] In an optional embodiment, the preset construction rule includes a second preset construction rule, wherein the second preset construction rule is used to process the fact element in the label text segment;

[0205] The training module is further configured to:

[0206] According to the second preset construction rule, the factual elements in the label text segment are modified, and the modified label text segment is determined as the sample text segment.

[0207] In an optional embodiment, the preset construction rule includes a third preset construction rule, wherein the third preset construction rule is used to process the semantic text of the label text segment;

[0208] The training module is further configured to:

[0209] According to the third preset construction rule, the semantic text of the label text segment is determined, the semantic text is modified, and the modified semantic text is determined as the sample text segment.

[0210] In an optional embodiment, the training module is further configured to:

[0211] Determine a plurality of label text segments in the label text, and determine a first label correction text corresponding to each label text segment in the plurality of label text segments;

[0212] According to a preset deletion rule, the first label correction texts corresponding to a preset number of label text segments are deleted from the multiple label text segments to obtain remaining label correction texts, wherein the preset deletion rule is used to delete the first label correction texts corresponding to the label text segments;

[0213] The step of training the text processing model according to the label text, the sample text, and the label-corrected text until a text processing model that satisfies a training stop condition is obtained includes:

[0214] The text processing model is trained according to the label text, the sample text and the remaining label correction text until a text processing model that meets the training stop condition is obtained.

[0215] In an optional embodiment, the apparatus further includes a communication module configured to:

[0216] receiving a text processing request sent by a client, wherein the text processing request carries prompt information and is generated based on an input operation of a user through a display interface of the client;

[0217] The target text is sent to the client and displayed to the user through the display interface.

[0218] In the above-mentioned device, a text processing model is used to correct the text to be processed using a reference correction text with a greater confidence level than the text to be processed to obtain a target text, thereby achieving post-verification of the article generated by the model and further improving the accuracy of the article generated by the model. In addition, by constructing sample text based on the labeled text and the labeled correction text, and training the text processing model based on the sample text, labeled text, and the labeled correction text, the text processing model can learn more errors that may occur in the text, thereby improving the text processing model's error correction capabilities, ensuring the processing performance of the text processing model, and making the target text obtained after processing by the text processing model more accurate, thereby ensuring the user experience.

[0219] The above is a schematic diagram of a text processing device according to this embodiment. It should be noted that the technical solution of the text processing device and the technical solution of the above-mentioned text processing method are based on the same concept. For details not described in detail in the technical solution of the text processing device, please refer to the description of the technical solution of the above-mentioned text processing method.

[0220] See also Figure 6 , Figure 6 A flowchart of a text processing model training method provided according to an embodiment of this specification is shown, which is applied to a cloud-side device and specifically includes the following steps.

[0221] Step 602: determining a label text and a label correction text associated with the label text, and constructing a sample text based on the label text and the label correction text;

[0222] Step 604: Train the text processing model based on the label text, the sample text, and the label correction text until a text processing model that meets the training stop condition is obtained.

[0223] Among them, the labeled text, the labeled corrected text, and the sample text can be used as a set of training data to train the text processing model. The labeled text can be understood as real and correct text, the labeled corrected text can be understood as the text in the labeled text that correctly describes the objective facts, and the sample text can be understood as the text obtained by constructing the labeled text and containing incorrect text content (i.e., containing knowledge illusions). For example, the labeled text can be an article in which the objective facts described in the article are accurate, the labeled corrected text can be the text in the article that correctly describes the objective facts, and the sample text can be the article obtained after the objective facts described in the article are modified. This is used to train the knowledge illusion correction ability of the text processing model.

[0224] In the above method, the text to be processed is corrected using a text processing model and reference correction text with a greater confidence level than the text to be processed to obtain the target text. This allows for post-verification of the model-generated articles, further improving the accuracy of the model-generated articles. Furthermore, by constructing sample text based on labeled text and labeled correction text, and training the text processing model based on the sample text, labeled text, and labeled correction text, the text processing model can learn more about possible errors in the text, thereby improving the text processing model's error correction capabilities and ensuring its processing performance. This results in a more accurate target text obtained after processing by the text processing model, thereby ensuring a better user experience.

[0225] The above is a schematic diagram of a text processing model training method according to this embodiment. It should be noted that the technical solution of this text processing model training method is based on the same concept as the technical solution of the aforementioned text processing method. For details not described in detail in the technical solution of the text processing model training method, please refer to the description of the technical solution of the aforementioned text processing method.

[0226] Corresponding to the above method embodiment, this specification also provides an embodiment of a text processing model training device, which is applied to a cloud-side device. Figure 7 A structural diagram of a text processing model training device provided by an embodiment of this specification is shown.

[0227] like Figure 7 As shown, the device includes:

[0228] The determining module 702 is configured to determine a label text and a label correction text associated with the label text, and construct a sample text based on the label text and the label correction text;

[0229] The training module 704 is configured to train the text processing model according to the label text, the sample text and the label correction text until a text processing model that meets the training stop condition is obtained.

[0230] In the above-mentioned device, a text processing model is used to correct the text to be processed using a reference correction text with a greater confidence level than the text to be processed to obtain a target text, thereby achieving post-verification of the article generated by the model and further improving the accuracy of the article generated by the model. In addition, by constructing sample text based on the labeled text and the labeled correction text, and training the text processing model based on the sample text, labeled text, and the labeled correction text, the text processing model can learn more errors that may occur in the text, thereby improving the text processing model's error correction capabilities, ensuring the processing performance of the text processing model, and making the target text obtained after processing by the text processing model more accurate, thereby ensuring the user experience.

[0231] The above is a schematic diagram of a text processing model training device according to this embodiment. It should be noted that the technical solution of this text processing model training device and the technical solution of the aforementioned text processing method are based on the same concept. For details not described in detail in the technical solution of the text processing model training device, please refer to the description of the technical solution of the aforementioned text processing method.

[0232] See also Figure 8 , Figure 8 A flowchart of an article correction method provided according to an embodiment of this specification is shown, which is applied to a cloud-side device and specifically includes the following steps.

[0233] Step 802: Receive an article to be revised, and determine a reference revision text associated with the article to be revised, wherein the confidence level of the reference revision text is greater than the confidence level of the article to be revised;

[0234] Step 804: Input the article to be revised and the reference revised text into a text processing model to obtain a target article corresponding to the article to be revised;

[0235] The text processing model is trained based on the label text, the label correction text associated with the label text, and the sample text, and the sample text is constructed based on the label text and the label correction text.

[0236] In an optional embodiment, the receiving of the article to be revised includes:

[0237] Receiving an article generation request sent by a client, wherein the article generation request carries prompt information;

[0238] The prompt information is input into an article generation model to obtain an article to be revised corresponding to the prompt information.

[0239] In the above method, the text to be processed is corrected using a text processing model and reference correction text with a greater confidence level than the text to be processed to obtain the target text. This allows for post-verification of the model-generated articles, further improving the accuracy of the model-generated articles. Furthermore, by constructing sample text based on labeled text and labeled correction text, and training the text processing model based on the sample text, labeled text, and labeled correction text, the text processing model can learn more about possible errors in the text, thereby improving the text processing model's error correction capabilities and ensuring its processing performance. This results in a more accurate target text obtained after processing by the text processing model, thereby ensuring a better user experience.

[0240] The above is a schematic diagram of an article correction method according to this embodiment. It should be noted that the technical solution of this article correction method and the technical solution of the above-mentioned text processing method are based on the same concept. For details not described in detail in the technical solution of the article correction method, please refer to the description of the technical solution of the above-mentioned text processing method.

[0241] Corresponding to the above method embodiment, this specification also provides an article correction method embodiment, which is applied to cloud-side devices. Figure 9 FIG. 1 shows a schematic diagram of a method for revising an article provided in one embodiment of the present specification. Figure 9 As shown, the device includes:

[0242] The receiving module 902 is configured to receive an article to be revised and determine a reference revision text associated with the article to be revised, wherein the confidence level of the reference revision text is greater than the confidence level of the article to be revised;

[0243] An input module 904 is configured to input the article to be revised and the reference revised text into a text processing model to obtain a target article corresponding to the article to be revised;

[0244] The text processing model is trained based on the label text, the label correction text associated with the label text, and the sample text, and the sample text is constructed based on the label text and the label correction text.

[0245] In an optional embodiment, the receiving module 902 is further configured to:

[0246] Receiving an article generation request sent by a client, wherein the article generation request carries prompt information;

[0247] The prompt information is input into an article generation model to obtain an article to be revised corresponding to the prompt information.

[0248] In the above-mentioned device, a text processing model is used to correct the text to be processed using a reference correction text with a greater confidence level than the text to be processed to obtain a target text, thereby achieving post-verification of the article generated by the model and further improving the accuracy of the article generated by the model. In addition, by constructing sample text based on the labeled text and the labeled correction text, and training the text processing model based on the sample text, labeled text, and the labeled correction text, the text processing model can learn more errors that may occur in the text, thereby improving the text processing model's error correction capabilities, ensuring the processing performance of the text processing model, and making the target text obtained after processing by the text processing model more accurate, thereby ensuring the user experience.

[0249] The above is a schematic diagram of an article correction device according to this embodiment. It should be noted that the technical solution of the article correction device and the technical solution of the aforementioned text processing method are based on the same concept. For details not described in detail in the technical solution of the article correction device, please refer to the description of the technical solution of the aforementioned text processing method.

[0250] Figure 10 1000 according to an embodiment of the present disclosure. Components of the computing device 1000 include, but are not limited to, a memory 1010 and a processor 1020. The processor 1020 is connected to the memory 1010 via a bus 1030, and a database 1050 is used to store data.

[0251] The computing device 1000 also includes an access device 1040 that enables the computing device 1000 to communicate via one or more networks 1060. Examples of these networks include a public switched telephone network (PSTN), a local area network (LAN), a wide area network (WAN), a personal area network (PAN), or a combination of communication networks such as the Internet. The access device 1040 may include one or more of any type of network interface, wired or wireless (e.g., a network interface card (NIC)), such as an IEEE 802.11 wireless local area network (WLAN) wireless interface, a Worldwide Interoperability for Microwave Access (Wi-MAX) interface, an Ethernet interface, a Universal Serial Bus (USB) interface, a cellular network interface, a Bluetooth interface, a Near Field Communication (NFC) interface, and the like.

[0252] In one embodiment of the present application, the above components of the computing device 1000 and Figure 10 Other components not shown in the figure may also be connected to each other, for example, via a bus. Figure 10 The computing device structure block diagram shown is for illustrative purposes only and is not intended to limit the scope of the present application. Those skilled in the art may add or replace other components as needed.

[0253] Computing device 1000 may be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (e.g., a tablet computer, a personal digital assistant, a laptop computer, a notebook computer, a netbook computer, etc.), a mobile phone (e.g., a smartphone), a wearable computing device (e.g., a smartwatch, smart glasses, etc.), or other types of mobile devices, or a stationary computing device such as a desktop computer or a personal computer (PC). Computing device 1000 may also be a mobile or stationary server.

[0254] The processor 1020 is configured to execute the following computer-executable instructions, which implement the steps of the above method when executed by the processor.

[0255] The various embodiments in this specification are described in a progressive manner. Similar parts between the various embodiments can be referred to in conjunction with each other. Each embodiment focuses on the differences between the other embodiments. In particular, the computing device embodiment is generally similar to the method embodiment, so the description is relatively simple. For relevant parts, refer to the description of the method embodiment.

[0256] An embodiment of the present specification further provides a computer-readable storage medium storing a computer program / instruction, which implements the steps of the above method when executed by a processor.

[0257] The various embodiments in this specification are described in a progressive manner. Similar portions between the various embodiments can be referenced to each other. Each embodiment focuses on the differences between the other embodiments. In particular, the computer-readable storage medium embodiment is generally similar to the method embodiment, so its description is relatively simple. For relevant portions, refer to the description of the method embodiment.

[0258] An embodiment of the present specification further provides a computer program product, comprising a computer program / instruction, which implements the steps of the above method when executed by a processor.

[0259] The above is an illustrative solution of a computer program product of this embodiment. It should be noted that the technical solution of the computer program product and the technical solution of the above method belong to the same concept. For details not described in detail in the technical solution of the computer program product, please refer to the description of the technical solution of the above method.

[0260] The foregoing description of this specification describes specific embodiments. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in an order different from that described in the embodiments and still achieve the desired results. Furthermore, the processes depicted in the accompanying drawings do not necessarily require the specific order shown or the sequential order to achieve the desired results. In certain embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0261] The computer instructions include computer program codes, which may be in source code form, object code form, executable files, or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal, and software distribution medium. It should be noted that the content contained in the computer-readable medium may be appropriately increased or decreased according to the requirements of patent practice. For example, in some regions, according to patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0262] It should be noted that for the aforementioned method embodiments, for the sake of simplicity of description, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the embodiments of this specification are not limited by the order of the actions described, because according to the embodiments of this specification, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in this specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the embodiments of this specification.

[0263] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0264] The preferred embodiments disclosed above are intended only to help illustrate this specification. The optional embodiments do not exhaustively describe all details, nor do they limit the invention to the specific embodiments described. Obviously, many modifications and variations can be made based on the content of the embodiments of this specification. This specification selects and specifically describes these embodiments in order to better explain the principles and practical applications of the embodiments of this specification, so that those skilled in the art can better understand and utilize this specification. This specification is limited only by the claims and their full scope and equivalents.

Claims

1. A text processing method, comprising: Determining a text to be processed, and determining a reference revised text associated with the text to be processed, wherein the confidence level of the reference revised text is greater than the confidence level of the text to be processed; Inputting the text to be processed and the reference revised text into a text processing model to obtain a target text corresponding to the text to be processed; The text processing model is trained based on the label text, the label correction text associated with the label text, and the sample text, and the sample text is constructed based on the label text and the label correction text.

2. The text processing method according to claim 1, wherein the text processing model comprises a detection unit, a correction unit, and a replacement unit; Accordingly, the step of inputting the text to be processed and the reference revised text into a text processing model to obtain a target text corresponding to the text to be processed includes: Inputting the text to be processed and the reference revised text into the text processing model, and using the detection unit to detect the text to be processed based on the reference revised text to obtain a target text segment to be processed; Using the correction unit, the target text segment to be processed is corrected according to a first reference correction text corresponding to the target text segment to be processed to obtain a target text segment, wherein the first reference correction text is determined based on the reference correction text; The replacement unit is used to replace the target text segment in the text to be processed according to the target text segment, so as to obtain the target text corresponding to the text to be processed.

3. The text processing method according to claim 2, wherein the detecting unit detects the text to be processed based on the reference revised text to obtain a target text segment to be processed, comprising: Using the detection unit, matching the reference revised text and the text to be processed to obtain a matching result; A target text segment to be processed in the text to be processed is determined according to the matching result.

4. The text processing method according to claim 1, wherein determining the text to be processed comprises: Generate text using a text generation model based on prompt information and references to generate text to be processed corresponding to the prompt information, wherein the text to be processed has a larger number of texts than the text of the prompt information; Accordingly, determining the reference revised text associated with the text to be processed includes: The reference generated text is used as the reference revised text associated with the text to be processed.

5. The text processing method according to claim 1, wherein determining a reference correction text associated with the text to be processed comprises: Determining a text segment to be processed contained in the text to be processed, wherein the text segment to be processed is text content describing objective facts; Searching a knowledge base to obtain a first reference revised text corresponding to the text segment to be processed; The first reference revised text corresponding to the text segment to be processed is determined as the reference revised text associated with the text to be processed.

6. The text processing method according to claim 1, wherein the step of training the text processing model comprises: Determining a label text and a label correction text associated with the label text, and constructing a sample text based on the label text and the label correction text; The text processing model is trained according to the label text, the sample text and the label correction text until a text processing model that meets the training stop condition is obtained.

7. The text processing method according to claim 6, wherein constructing a sample text based on the label text and the label-corrected text comprises: Determining a label text segment in the label text; Aligning the label text segment and the label correction text, and determining a first label correction text corresponding to the label text segment according to the alignment result; Processing the labeled text segment according to a preset construction rule associated with the first labeled revised text to construct a sample text segment; The label text segment in the label text is replaced according to the sample text segment to obtain the sample text.

8. The text processing method according to claim 7, wherein the processing of the labeled text segment to construct a sample text segment based on the preset construction rule associated with the first label-corrected text comprises: Responding to a construction instruction sent by the client, wherein the construction instruction is generated according to construction information input by a model trainer based on a display interface of the client, the construction information including the preset construction rule and the label text segment; According to the construction instruction, a sample construction model is called, and the label text segment is processed according to the preset construction rule associated with the first label correction text to generate a sample text segment.

9. The text processing method according to claim 7, wherein the preset construction rule comprises a first preset construction rule, wherein: The first preset construction rule is used to process the numbers in the label text segment; The step of processing the label text segment according to the preset construction rule associated with the first label-corrected text to construct a sample text segment includes: In the case where it is determined that the label text segment includes numbers, the numbers included in the label text segment are modified according to the first preset construction rule, and the modified label text segment is determined as the sample text segment.

10. The text processing method according to claim 7, wherein the preset construction rule comprises a second preset construction rule, wherein: The second preset construction rule is used to process the fact elements in the label text segment; The step of processing the label text segment according to the preset construction rule associated with the first label-corrected text to construct a sample text segment includes: According to the second preset construction rule, the factual elements in the label text segment are modified, and the modified label text segment is determined as the sample text segment.

11. The text processing method according to claim 7, wherein the preset construction rule comprises a third preset construction rule, wherein: The third preset construction rule is used to process the semantic text of the label text segment; The step of processing the label text segment according to the preset construction rule associated with the first label-corrected text to construct a sample text segment includes: According to the third preset construction rule, the semantic text of the label text segment is determined, the semantic text is modified, and the modified semantic text is determined as the sample text segment.

12. The text processing method according to claim 6, after constructing the sample text based on the label text and the label-corrected text, further comprising: Determine a plurality of label text segments in the label text, and determine a first label correction text corresponding to each label text segment in the plurality of label text segments; According to a preset deletion rule, the first label correction texts corresponding to a preset number of label text segments in the plurality of label text segments are deleted to obtain remaining label correction texts; The step of training the text processing model according to the label text, the sample text, and the label-corrected text until a text processing model that satisfies a training stop condition is obtained includes: The text processing model is trained according to the label text, the sample text and the remaining label correction text until a text processing model that meets the training stop condition is obtained.

13. The text processing method according to claim 4, wherein the method further comprises: generating text based on prompt information and references using a text generation model, and generating the text to be processed corresponding to the prompt information before generating the text to be processed corresponding to the prompt information; receiving a text processing request sent by a client, wherein the text processing request carries prompt information and is generated based on an input operation of a user through a display interface of the client; After obtaining the target text corresponding to the to-be-processed text, the method further includes: The target text is sent to the client and displayed to the user through the display interface.

14. A text processing model training method, applied to a cloud-side device, comprising: Determining a label text and a label correction text associated with the label text, and constructing a sample text based on the label text and the label correction text; The text processing model is trained according to the label text, the sample text and the label correction text until a text processing model that meets the training stop condition is obtained.

15. A method for revising an article, applied to a cloud-side device, comprising: receiving an article to be revised, and determining a reference revision text associated with the article to be revised, wherein the confidence level of the reference revision text is greater than the confidence level of the article to be revised; Inputting the article to be revised and the reference revised text into a text processing model to obtain a target article corresponding to the article to be revised; The text processing model is trained based on the label text, the label correction text associated with the label text, and the sample text, and the sample text is constructed based on the label text and the label correction text.

16. The article correction method according to claim 15, wherein receiving the article to be corrected comprises: Receiving an article generation request sent by a client, wherein the article generation request carries prompt information; The prompt information is input into an article generation model to obtain an article to be revised corresponding to the prompt information.

17. A computing device comprising: memory and processor; The memory is used to store computer programs / instructions, and the processor is used to execute the computer programs / instructions. When the computer program / instructions are executed by the processor, the steps of the method according to any one of claims 1 to 16 are implemented.

18. A computer-readable storage medium storing a computer program / instruction, wherein the computer program / instruction, when executed by a processor, implements the steps of the method according to any one of claims 1 to 16.

19. A computer program product comprising a computer program / instruction, which, when executed by a processor, implements the steps of the method according to any one of claims 1 to 16.

Images