Task processing method, text processing method, automatic question-answering method, task processing model training method, information processing method based on task processing model, and cloud training platform

Abstract

Provided in the embodiments of the present disclosure are a task processing method, a text processing method, an automatic question-answering method, a task processing model training method, an information processing method based on a task processing model, and a cloud training platform, which are applied to the technical field of computers. The task processing method comprises: acquiring task data of a target task; and inputting the task data into a task processing model, so as to obtain a task processing result of the target task, wherein the task processing model is obtained by means of performing training on the basis of a plurality of pieces of target sample data and target sample results of the plurality of pieces of target sample data, the target sample data is obtained by means of performing screening on the basis of detection results of repeated content of a plurality of pieces of sample data, and the content of the target sample results is not repeated. Target sample data is obtained by means of performing screening on the basis of detection results of repeated content, and the content of target sample results is not repeated, such that the repetition illusion of a model is reduced without damaging the processing capability of the model itself and relying on external knowledge, thereby improving the accuracy and integrity of a task processing result.

Description

[0001] This disclosure claims priority to Chinese patent application number 202410318207.7, filed with the China Patent Office on March 19, 2024, and entitled "Task Processing Method, Text Processing Method, Automatic Question Answering Method, Task Processing Model Training Method, Information Processing Method Based on Task Processing Model, and Cloud Training Platform," the entire contents of which are incorporated herein by reference. Technical Field: The embodiments of the present disclosure relate to the field of computer technology, and more particularly to task processing methods, text processing methods, automatic question answering methods, task processing model training methods, information processing methods based on task processing models, and cloud training platforms. Background: With the development of computer technology, large models have begun to shine. Their remarkable capabilities in language understanding, generation, interaction, and reasoning have led to their widespread application in natural language processing fields such as dialogue, translation, and code generation. At the same time, large models also face the challenge of hallucinations. Hallucination refers to a phenomenon that occurs in large models, particularly in natural language processing tasks such as text generation and question answering. While the model's output may appear reasonable and fluent, it may actually contain erroneous information, fabricated facts, or be seriously inconsistent with real-world conditions. This inconsistency between model output and real-world knowledge or user input is called hallucination. Currently, to address the poor accuracy of task processing results caused by hallucination in large models, retrieval enhancement methods are often used to leverage external knowledge sources, allowing large models to rely on reliable knowledge sources for task processing. However, retrieval enhancement methods rely too heavily on external knowledge sources, and the hallucination problem inherent in large models is not alleviated. The accuracy of large models remains low, resulting in poor accuracy when directly using large models for task processing. Therefore, a highly accurate task processing solution is urgently needed. SUMMARY OF THE INVENTION In view of this, embodiments of the present disclosure provide a task processing method. One or more embodiments of the present disclosure simultaneously relate to a text processing method, an automatic question-answering method, a task processing model training method, an information processing method based on a task processing model, a cloud training platform, a task processing device, a text processing device, an automatic question-answering device, a task processing model training device, an information processing device based on a task processing model, a computing device, a computer-readable storage medium, and a computer program product, to address technical deficiencies in the prior art.According to a first aspect of an embodiment of the present disclosure, a task processing method is provided, comprising: obtaining task data for a target task; inputting the task data into a task processing model to obtain a task processing result for the target task, wherein the task processing model is trained based on multiple target sample data and target sample results for the multiple target sample data, the target sample data is obtained by screening based on duplicate content detection results for the multiple sample data, and the target sample results contain no duplicate content. According to a second aspect of an embodiment of the present disclosure, a text processing method is provided, comprising: obtaining text to be processed for a target text task; inputting the text to be processed into the task processing model to obtain a text processing result for the target text task, wherein the task processing model is trained based on multiple target sample data and target sample results for the multiple target sample data, the target sample data is obtained by screening based on duplicate content detection results for the multiple sample data, and the target sample results contain no duplicate content. According to a third aspect of an embodiment of the present disclosure, an automatic question-answering method is provided, comprising: obtaining a pending question for a target question-answering task; inputting the pending question into a task processing model to obtain an answer result for the target question-answering task, wherein the task processing model is trained based on multiple target sample data and target sample results for the multiple target sample data, wherein the target sample data is screened based on duplicate content detection results for the multiple sample data, and wherein the target sample results have non-duplicate content. According to a fourth aspect of an embodiment of the present disclosure, a task processing model training method is provided, comprising: obtaining multiple target sample data, wherein the target sample data is screened based on duplicate content detection results for the multiple sample data; inputting the multiple target sample data into an initial processing model to obtain target sample prediction results for the multiple target sample data; and training the initial processing model based on the target sample prediction results and the target sample results for the multiple target sample data to obtain a trained task processing model, wherein the target sample results have non-duplicate content. According to a fifth aspect of an embodiment of the present disclosure, an information processing method based on a task processing model is provided, which is applied to a cloud training platform, including: receiving a task generation request sent by a terminal device, wherein the task generation request includes request information; obtaining a task processing model based on the request information, wherein the task processing model is trained based on multiple target sample data and target sample results of the multiple target sample data, the target sample data is obtained by screening based on duplicate content detection results of the multiple sample data, and the target sample result content is non-duplicate; generating task information based on the task processing model, wherein the task information is used for the terminal device to execute the target task.According to a sixth aspect of an embodiment of the present disclosure, a task processing device is provided, comprising: a first acquisition module configured to acquire task data for a target task; a first input module configured to input the task data into a task processing model to obtain a task processing result for the target task, wherein the task processing model is trained based on multiple target sample data and target sample results for the multiple target sample data, the target sample data is screened based on duplicate content detection results for the multiple sample data, and the target sample results contain no duplicate content. According to a seventh aspect of an embodiment of the present disclosure, a text processing device is provided, comprising: a second acquisition module configured to acquire text to be processed for a target text task; a second input module configured to input the text to be processed into the task processing model to obtain a text processing result for the target text task, wherein the task processing model is trained based on multiple target sample data and target sample results for the multiple target sample data, the target sample data is screened based on duplicate content detection results for the multiple sample data, and the target sample results contain no duplicate content. According to an eighth aspect of an embodiment of the present disclosure, an automatic question-answering device is provided, comprising: a third acquisition module configured to acquire a pending question for a target question-answering task; a third input module configured to input the pending question into a task processing model to obtain an answer result for the target question-answering task, wherein the task processing model is trained based on a plurality of target sample data and target sample results for the plurality of target sample data, wherein the target sample data is obtained by screening based on duplicate content detection results for the plurality of sample data, and wherein the target sample results have non-duplicate content. According to a ninth aspect of an embodiment of the present disclosure, a task processing model training device is provided, comprising: a fourth acquisition module configured to acquire a plurality of target sample data, wherein the target sample data is obtained by screening based on duplicate content detection results for the plurality of sample data; a fourth input module configured to input the plurality of target sample data into an initial processing model to obtain target sample prediction results for the plurality of target sample data; and a first training module configured to train the initial processing model based on the target sample prediction results and the target sample results for the plurality of target sample data, to obtain a trained task processing model, wherein the target sample results have non-duplicate content.According to a tenth aspect of an embodiment of the present disclosure, there is provided an information processing device based on a task processing model, which is applied to a cloud training platform and includes: a first receiving module, configured to receive a task generation request sent by a terminal device, wherein the task generation request includes request information; a fifth acquisition module, configured to acquire a task processing model based on the request information, wherein the task processing model is trained based on multiple target sample data and target sample results of multiple target sample data, the target sample data is obtained by screening based on duplicate content detection results of multiple sample data, and the target sample result content is non-duplicate; a first generation module, configured to generate task information based on the task processing model, wherein the task information is used for the terminal device to perform the target task. According to an eleventh aspect of the embodiments of the present disclosure, a cloud training platform is provided, comprising: a request interface for receiving a task generation request sent by a terminal device, wherein the task generation request includes request information; a response unit for obtaining a task processing model based on the request information, wherein the task processing model is trained based on multiple target sample data and target sample results of the multiple target sample data, wherein the target sample data is obtained by screening duplicate content detection results of the multiple sample data, and the target sample results have non-duplicate content; and task information is generated based on the task processing model, wherein the task information is used for the terminal device to execute the target task. According to a twelfth aspect of the embodiments of the present disclosure, a computing device is provided, comprising: a memory and a processor; the memory is configured to store a computer program / instructions; the processor is configured to execute the computer program / instructions, wherein when executed by the processor, the computer program / instructions implement the steps of the method provided in the first, second, third, fourth, or fifth aspects above. According to a thirteenth aspect of the embodiments of the present disclosure, a computer-readable storage medium is provided, wherein the computer program / instructions are stored; wherein when executed by the processor, the computer program / instructions implement the steps of the method provided in the first, second, third, fourth, or fifth aspects above. According to a fourteenth aspect of an embodiment of the present disclosure, a computer program product is provided, comprising a computer program / instruction, which, when executed by a processor, implements the steps of the method provided in the first aspect, second aspect, third aspect, fourth aspect, or fifth aspect.An embodiment of the present disclosure provides a task processing method, comprising: obtaining task data for a target task; and inputting the task data into a task processing model to obtain a task processing result for the target task. The task processing model is trained based on multiple target sample data and target sample results for the multiple target sample data. The target sample data is obtained by screening the multiple sample data based on duplicate content detection results, and the target sample results contain no duplicate content. Because the target sample data used to train the task processing model is obtained by screening the multiple sample data based on duplicate content detection results, and the target sample results contain no duplicate content, the method effectively mitigates the duplication illusion problem of the task processing model, improves the accuracy of the task processing model, and further enhances the accuracy and completeness of the task processing results, without compromising the task processing model's inherent processing capabilities and without relying on external knowledge. BRIEF DESCRIPTION OF THE DRAWINGS FIG1 is an architecture diagram of a task processing system provided by an embodiment of the present disclosure; FIG2 is an architecture diagram of another task processing system provided by an embodiment of the present disclosure; FIG3 is a flow chart of a task processing method provided by an embodiment of the present disclosure; FIG4 is a flow chart of duplicate content detection in a task processing method provided by an embodiment of the present disclosure; FIG5 is a flow chart of the processing process of a task processing method provided by an embodiment of the present disclosure; FIG6 is a flow chart of a text processing method provided by an embodiment of the present disclosure; FIG7 is a flow chart of an automatic question-answering method provided by an embodiment of the present disclosure; FIG8 is a flow chart of a task processing model training method provided by an embodiment of the present disclosure; FIG9 is a flow chart of an information processing method based on a task processing model provided by an embodiment of the present disclosure; FIG10 is a structural diagram of a cloud training platform provided by an embodiment of the present disclosure; FIG11 is a structural diagram of a task processing device provided by an embodiment of the present disclosure; FIG12 is a structural diagram of a text processing device provided by an embodiment of the present disclosure; FIG13 is a structural diagram of an automatic question-answering device provided by an embodiment of the present disclosure; FIG14 is a structural diagram of a task processing model training device provided by an embodiment of the present disclosure; Figure 15 is a schematic diagram of the structure of an information processing device based on a task processing model provided by one embodiment of the present disclosure; Figure 16 is a block diagram of the structure of a computing device provided by one embodiment of the present disclosure. The following description of the specific embodiments sets forth numerous specific details to facilitate a thorough understanding of the present disclosure. However, the present disclosure can be implemented in many other ways than those described herein, and those skilled in the art may make similar generalizations without departing from the scope of the present disclosure. Therefore, the present disclosure is not limited to the specific implementations disclosed below.The terminology used in one or more embodiments of the present disclosure is for the purpose of describing specific embodiments only and is not intended to limit the present disclosure. As used in one or more embodiments of the present disclosure and the appended claims, the singular forms "a," "an," "the," and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It should also be understood that the term "and / or" used in one or more embodiments of the present disclosure refers to and encompasses any and all possible combinations of one or more of the associated listed items. It should be understood that while the terms "first," "second," and so on may be employed to describe various information in one or more embodiments of the present disclosure, such information should not be limited to these terms. These terms are merely used to distinguish information of the same type from one another. For example, first could be referred to as second, and similarly, second could be referred to as first, without departing from the scope of one or more embodiments of the present disclosure. Depending on the context, the term "if" as used herein could be interpreted as meaning "when," "when," or "in response to determining." Furthermore, it should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, storage, and display) involved in one or more embodiments of this disclosure are all authorized by the user or fully authorized by all parties. The collection, use, and processing of such data must comply with the relevant laws, regulations, and standards of the relevant region, and corresponding access points are provided for users to choose to authorize or deny such data. In one or more embodiments of this disclosure, a large model refers to a deep learning model with large-scale model parameters, typically containing hundreds of millions, tens of billions, hundreds of billions, trillions, or even more than ten trillion model parameters. Large models, also known as foundation models, are pre-trained on large-scale unlabeled corpora to produce pre-trained models with over 100 million parameters. These models are adaptable to a wide range of downstream tasks and have good generalization capabilities. Examples include large language models (LLM) and multimodal pre-training models.In practical applications, large models only require a small number of samples to fine-tune the pre-trained model before being applied to various tasks. Large models can be widely used in fields such as natural language processing (NLP) and computer vision. Specifically, they can be applied to computer vision tasks such as visual question answering (VQA), image captioning (IC), and image generation, as well as natural language processing tasks such as text-based sentiment classification, text summarization, and machine translation. Key application scenarios for large models include digital assistants, intelligent robots, search, online education, office software, e-commerce, and intelligent design. First, the terms used in one or more embodiments of this disclosure are explained. Knowledge edge compilation refers to updating the knowledge representation within the model through fine-tuning or specific training methods to reflect the latest facts or correct the model's erroneous cognition. Supervised fine-tuning (SFT): A large-model training technique that typically involves fine-tuning a pre-trained model using labeled data on a specific task or target dataset to adapt it to a new, specific learning task. Oracle model: In natural language processing, an oracle model typically refers to an idealized model or system that exhibits exceptional performance or capabilities in a particular area and can be used to evaluate the performance of other models or systems, such as the Chat Generative Pretrained Transformer (ChatGPT) model. Convolutional Neural Network (CNN): A multi-layer deep learning model with forward and backpropagation, featuring convolutional kernels that process feature data. Recurrent Neural Network (RNN) model: A recursive deep learning model that processes vector representations recursively and connects intermediate layers in a chain-like manner.Long Short-Term Memory (LSTM) model: A deep learning model capable of memorizing both long-term and short-term information, with convolutional kernels that process feature data. Deep Self-Attention Model (Transformer model): A deep learning architecture based on the attention mechanism, used to process sequential data, such as natural language. Bidirectional Encoder Representations from Transformers (BERT): A special Transformer model trained using a bidirectional Transformer encoder and large-scale unlabeled text data. Repetition Penalty: A constraint imposed on the model during the generation process. It modifies the model's generation probability distribution to reduce the probability of generating consecutive repetitive content. Specifically, if the model considers a word that has already appeared in a nearby position when generating the next word, the probability of generating that word is reduced by the penalty factor. For example, if the repetition penalty parameter is set to a value greater than 1, the model will suppress duplicate content; if it is set to less than 1, it may allow more repetition. Data cleaning refers to a series of preprocessing operations performed on raw data during data analysis or data mining. Its purpose is to improve data quality and ensure the accuracy and reliability of subsequent analysis results. This process involves identifying and correcting (or deleting) erroneous, incomplete, inconsistent, duplicate, or irrelevant data in the dataset. With the development of natural language processing technology, large models have achieved significant breakthroughs in text understanding and generation, but they also face challenges such as hallucinations. Large model hallucinations include the content generated by the model being inconsistent with the real world or user instructions, or the continuous repetition of previous output content. The danger of hallucinations is that seemingly authentic or reasonable answers generated by the model may actually be incorrect, confusing, or completely fabricated knowledge. These hallucinations make it difficult to guarantee the accuracy and completeness of the content generated by large models, causing significant difficulties in their practical application. Therefore, how to effectively detect and alleviate large model hallucinations has become one of the current hot topics in natural language research.Currently, retrieval-augmented generation and tuning inference hyperparameters are commonly used to mitigate the hallucination problem of large models. However, retrieval-augmented generation methods leverage external knowledge sources, allowing the model to generate answers based on reliable knowledge. This reliance on external capabilities leaves the hallucination problem inherent in the large model unresolved and fails to truly alleviate it. Adjusting inference hyperparameters, particularly increasing the repeat penalty parameter during inference to force the model to sample fewer or no previously output text tokens, can also mitigate repeat hallucination. However, this method relies on external user input to set the repeat penalty parameter value, similarly failing to truly mitigate the hallucination problem inherent in the large model and potentially damaging the model's performance. In practical applications, hallucinated outputs can be broadly categorized into three types: factual hallucinations, faithful hallucinations, and repetition hallucinations. Factual hallucinations occur when the model claims that a false fact is true, or generates statements that have no basis in reality. Faithful hallucinations occur when the model fails to follow the user's input instructions to complete the task, resulting in output that deviates from the user's requirements. Repetition hallucination occurs when a model repeatedly repeats itself after outputting a certain number of words when answering a question. See Table 1 below for an example of model hallucination: The following is a detailed analysis using the examples in Table 1. As can be seen from the example of factual hallucination, the model mistakenly believes that Edison invented the lightbulb. In fact, Edison improved the design of the lightbulb and was not the sole inventor. The model has learned common human errors in the world corpus. These collective misconceptions are reflected in online posts, forums, and articles. As the model learns from this vast amount of data, it inevitably inherits and imitates these "collective hallucinations," leading to cognitive errors. These hallucinations primarily manifest as "inconsistency with facts" or "fabrication," creating discrepancies between the generated content and the real world. The model's factual hallucinations are largely due to inherent errors in the training data. As shown in the example of the faithfulness illusion, the user explicitly requested that apples be excluded from the input, yet the model still output "red apple" in its response. This demonstrates the model's inability to follow instructions and its excessive memorization of repetitive information in the training data. This illusion primarily manifests as inconsistent generated content: that is, the generated content deviates from the user's input instructions and context. The model simply memorizes the data rather than understands it, thus impairing its ability to understand the semantics of the specific instructions. As shown in the example of the repetition illusion, the repetition illusion phenomenon is particularly severe when using model-generated datasets to train small models. This is because the generated conversation data often contains expressions that are inconsistent with human language habits. For example, the model tends to list multiple concepts or nouns in its responses, or repeatedly repeat certain words. This type of data is difficult to learn and can easily confuse small models, leading to the repetition illusion phenomenon. After outputting a certain number of words, the model will repeatedly repeat the content it has listed. The three types of model hallucinations reveal the following patterns: Model output largely reflects the distribution, patterns, and tendencies of the data; models are highly sensitive to errors and duplications in training data and are easily influenced by negative data. However, there's a fine line between model creativity and hallucinations. In the phenomenon of emergent model capabilities, creativity and hallucinations are essentially two-in-one concepts.Therefore, the disclosed embodiments focus on mitigating the problem of model hallucinations. A knowledge editing solution based on data cleaning is proposed. By constructing target sample data and target sample results for the target sample data, and then training a task processing model based on multiple target sample data and target sample results for the target sample data, this solution effectively mitigates the problem of hallucinations in the task processing model without compromising the task processing model's inherent processing capabilities and without relying on external knowledge. This improves the accuracy of the task processing model and the user experience in actual use, further enhancing the accuracy and completeness of the content generated by the task processing model. This solution is a more feasible and practical task processing solution. Specifically, when performing task processing using the above solution, task data for the target task can be obtained; the task data is input into the task processing model to obtain the task processing results for the target task. The task processing model is trained based on multiple target sample data and target sample results for the target sample data; the target sample data is screened based on duplicate content detection results from the multiple sample data; and the target sample results contain no duplicate content. The present disclosure provides a task processing method. The present disclosure also relates to a text processing method, an automatic question-answering method, a task processing model training method, an information processing method based on a task processing model, a cloud training platform, a task processing device, a text processing device, an automatic question-answering device, a task processing model training device, an information processing device based on a task processing model, a computing device, a computer-readable storage medium, and a computer program product, each of which is described in detail in the following embodiments. Referring to FIG. 1 , FIG. 1 is an architecture diagram of a task processing system provided by an embodiment of the present disclosure. The task processing system may include a client 100 and a server 200. The client 100 is configured to send task data of a target task to the server 200. The server 200 is configured to input the task data into a task processing model to obtain a task processing result of the target task, wherein the task processing model is trained based on multiple target sample data and target sample results of the multiple target sample data, the target sample data is obtained by screening based on duplicate content detection results of the multiple sample data, and the target sample results have non-duplicate content. The task processing result is sent to the client 100. The client 100 is also configured to receive the task processing result sent by the server 200.Applying the solution of the embodiments of the present disclosure, since the target sample data for training the task processing model is obtained based on the duplicate content detection results of multiple sample data, the target sample results do not have duplicate content. Therefore, without compromising the task processing model's own processing capabilities and without relying on external knowledge, the problem of duplicate hallucination in the task processing model is effectively alleviated, the precision of the task processing model is improved, and the accuracy and completeness of the task processing results are further improved. Referring to Figure 2, Figure 2 is an architecture diagram of another task processing system provided by one embodiment of the present disclosure. The task processing system may include multiple clients 100 and a server 200. The clients 100 may include terminal devices, and the server 200 may include cloud devices. Multiple clients 100 can establish communication connections through the server 200. In task processing scenarios, the server 200 is used to provide task processing services between the multiple clients 100. The multiple clients 100 can act as senders or receivers, respectively, and communicate through the server 200. Users can interact with the server 200 through the client 100 to receive data from other clients 100 or send data to other clients 100. In a task processing scenario, a user can publish a data stream to the server 200 through the client 100. The server 200 generates task processing results based on the data stream and pushes the task processing results to other clients with which communication has been established. The connection between the client 100 and the server 200 is established via a network. The network provides the medium for the communication link between the client 100 and the server 200. The network can include various connection types, such as wired or wireless communication links or fiber optic cables. Data transmitted by the client 100 may need to undergo encoding, transcoding, compression, and other processing before being published to the server 200. oThe client 100 can be a browser, an APP (Application Program), a web application such as an H5 (HyperText Markup Languages, version 5) application, a light application (also known as a mini-program, a lightweight application), or a cloud application. The client 100 can be developed based on a software development kit (SDK) of a corresponding service provided by the server 200, such as a real-time communication (RTC) SDK. The client 100 can be deployed in an electronic device and rely on the device or certain APPs in the device to run. The electronic device can, for example, have a display screen and support information browsing, and can be a personal mobile terminal such as a mobile phone, a tablet computer, or a personal computer. Electronic devices can also typically be configured with various other applications, such as human-computer interaction applications, model training applications, text processing applications, web browser applications, shopping applications, search applications, instant messaging tools, email clients, social platform software, and the like. The server 200 may include servers that provide various services, such as servers that provide communication services to multiple clients, servers that provide backend training support for models used on clients, and servers that process data sent by clients. It should be noted that the server 200 can be implemented as a distributed server cluster consisting of multiple servers or as a single server. The server can also be a server in a distributed system or a server integrated with blockchain. The server can also be a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, content delivery networks (CDNs), big data and artificial intelligence platforms, or intelligent cloud computing servers or intelligent cloud hosts with artificial intelligence technology. It is worth noting that the task processing result method provided in the embodiments of the present disclosure is generally executed by the server. However, in other embodiments of the present disclosure, the client may also have similar functions to the server and thus execute the task processing result method provided in the embodiments of the present disclosure. In other embodiments, the task processing result method provided in the embodiments of the present disclosure may also be executed jointly by the client and the server.Referring to Figure 3, which is a flowchart of a task processing method provided in one embodiment of the present disclosure, the method specifically includes the following steps: Step 302: Obtaining task data for a target task. In one or more embodiments of the present disclosure, during task processing, task data for the target task may be obtained, and then the task data may be processed using a task processing model to generate a task processing result for the target task. Specifically, the target task may be a task in various scenarios, such as an intelligent question-answering task, a summary extraction task, an optical character recognition task, an object counting task, and so on. Task data is the processing object of the task processing model. Task data may be data in various modalities, such as text data, image data, voice data, video data, and so on. In actual applications, there are various methods for obtaining task data for the target task, and the method selected depends on the specific situation. This embodiment of the present disclosure does not impose any limitation on this method. In one possible implementation of the present disclosure, task data for the target task may be received from a front-end user. In another possible implementation of the present disclosure, task data for the target task may be read from other data acquisition devices or databases. Step 304: Input the task data into the task processing model to obtain the task processing result of the target task. The task processing model is trained based on multiple target sample data and target sample results for the multiple target sample data. The target sample data is screened based on duplicate content detection results from the multiple sample data, and the target sample results contain no duplicate content. In one or more embodiments of the present disclosure, after obtaining the task data for the target task, the task data can be further input into the task processing model to obtain the task processing result for the target task. Specifically, the task processing model is a deep learning model obtained by performing a SFT on an initial processing model based on the multiple target sample data and the target sample results for the multiple target sample data. The task processing model can include a large number of model parameters and can therefore be a large model. Task processing models include, but are not limited to, CNN models, RNN models, LSTM models, Transformer models, and BERT models. The task processing result is related to the target task. For example, if the target task is a summary extraction task, the task processing result is a summary of a long text; for example, if the target task is a text translation task, the task processing result is the translated text. It should be noted that the task processing model includes an encoding unit and a decoding unit. When the task data is input into the task processing model, the encoding unit first captures the complex dependencies between the elements within the task data through the self-attention mechanism, and converts the task data layer by layer into a high-level semantic embedding representation to obtain an encoding vector.Next, the decoding unit progressively decodes the encoding vectors provided by the encoding unit using a self-attention mechanism and a masking mechanism to obtain the task processing result. Applying the solution of the embodiment of the present disclosure, since the target sample data for training the task processing model is obtained based on the duplicate content detection results of multiple sample data, the target sample results do not have duplicate content. Therefore, without compromising the task processing model's inherent processing capabilities and without relying on external knowledge, the problem of hallucination of duplicates in the task processing model is effectively mitigated, the precision of the task processing model is improved, and the accuracy and completeness of the task processing results are further enhanced. In an optional embodiment of the present disclosure, after inputting the task data into the task processing model and obtaining the task processing result of the target task, the following steps may also be included: annotating key information in the task processing result to obtain an updated task processing result; and sending the updated task processing result to the front-end user. Specifically, the key information is related to the modality of the task processing result. If the task processing result is in text modality, the key information includes key text information such as keywords, key sentences, and key words. If the task processing result is in an image modality, key information refers to key visual points (such as eyes, eyebrows, and mouth corners in an image), and other key visual information. For example, if key information is key text information, key text information refers to facts, data, concepts, sentences, or vocabulary that can highlight the text's theme, convey the core idea, support the author's point of view, or describe the essence of an event. Key text information helps quickly understand the text's main theme, grasp the article's structure, answer relevant questions, make effective decisions, or extract information. In practical applications, for example, if key information is key text information, when annotating key information in the task processing result, methods such as bolding, highlighting, and italicizing can be used. For example, if key information is key visual information, when annotating key information in the task processing result, methods such as adding a background color and borders to key visual points can be used. Using the solutions of the embodiments of the present disclosure, key information in the task processing result is annotated to obtain an updated task processing result. The updated task processing result is then sent to the front-end user for easy viewing, thereby improving the user experience. In an optional embodiment of the present disclosure, after inputting task data into a task processing model and obtaining a task processing result for a target task, the following steps may be further included: sending the task processing result to a front-end user; receiving editing information sent by the front-end user, wherein the editing information is used to edit the task processing result; and editing the task processing result according to the editing information to obtain an edited task processing result.Specifically, editing information is used to describe the front-end user's editing requirements for the task processing result and can also be used to adjust parameters of the task processing model. Editing requirements include, but are not limited to, translation requirements, synonym replacement requirements, and paraphrase requirements. Furthermore, editing information can also be editing information specific to key information in the task processing result. For example, assuming the task processing result is "You can buy a ticket on Saturday morning," and the editing information is "Please translate the result into English," the task processing result is translated based on the editing information to obtain the translated task processing result, "You can buy your ticket on Saturday morning." Furthermore, after obtaining the edited task processing result, the edited task processing result can be used to adjust parameters of the task processing model. Using the solution of the embodiments of the present disclosure, the task processing result is sent to the front-end user; editing information sent by the front-end user is received, wherein the editing information is used to edit the task processing result; and the task processing result is edited based on the editing information to obtain the edited task processing result. By editing the task processing results based on the editing information sent by the front-end user, human-computer interaction is enhanced, and the adaptability and flexibility of task processing are improved. In an optional embodiment of the present disclosure, after inputting task data into the task processing model and obtaining the task processing results for the target task, the following steps may be included: sending the task processing results to the front-end user; receiving result feedback information from the front-end user, wherein the result feedback information is information providing feedback on the task processing results based on the task information of the target task; constructing model optimization data based on the result feedback information; and using the model optimization data to adjust parameters of the task processing model. Specifically, the result feedback information may include feedback on the content, quality, and completeness of the task processing results, reflecting the front-end user's true feelings and expectations about the task processing results. The result feedback information includes, but is not limited to, result quality evaluation information, corrected and accurate task processing results, and model optimization areas. Model optimization data refers to accurately optimized sample data used to optimize the task processing model. It should be noted that if the result feedback information is an accurate and corrected task processing result, model optimization data can be constructed based on the task data of the target task and the corrected and accurate task processing results. If the result feedback information is the optimization field of the model, such as XXX field, sample data of the XXX field can be obtained and the sample data of the XXX field can be determined as the model optimization data.The process of adjusting the parameters of the task processing model using model optimization data is similar to the training process for the task processing model described above, and will not be further described in detail in this embodiment of the present disclosure. Using the solution of this embodiment of the present disclosure, the task processing results are sent to a front-end user; result feedback information is received from the front-end user, where the result feedback information is feedback on the task processing results based on the task information of the target task; model optimization data is constructed based on the result feedback information; and the parameters of the task processing model are adjusted using the model optimization data. By collecting and utilizing the result feedback information, the performance of the task processing model is continuously optimized to more accurately meet the actual needs of the front-end user and improve the quality and accuracy of the final task processing results. In an optional embodiment of the present disclosure, constructing the model optimization data based on the result feedback information may include the following steps: generating optimization prompt information based on the result feedback information, where the optimization prompt information is used to guide the front-end user to send model optimization data for optimizing the task processing model; sending the optimization prompt information to the front-end user, and receiving the model optimization data sent by the front-end user based on the optimization prompt information. It should be noted that there are multiple ways to generate optimization prompt information, and the specific method to be used depends on the actual situation. This disclosure does not impose any restrictions on this. In one possible implementation of this disclosure, a pre-set optimization prompt information can be directly obtained, such as "I apologize for providing you with inaccurate information. Please point out the specific inaccuracies or provide the correct answers to relevant questions. I will correct and optimize my answers as soon as possible to better serve you." In another possible implementation of this disclosure, the result feedback information can be type-identified to determine the information type of the result feedback information. The information type can then be matched with the prompt type of each prompt information in a prompt information library, and prompt information with the same prompt type as the information type can be determined as the optimization prompt information. Using the solution of this embodiment of the disclosure, optimization prompt information is generated based on the result feedback information; the optimization prompt information is sent to the front-end user; and model optimization data sent by the front-end user based on the optimization prompt information is received.Obtaining model optimization data through interactive guidance improves user interactivity and enhances user satisfaction. In an optional embodiment of the present disclosure, a task processing model training method is described. Specifically, before inputting task data into the task processing model and obtaining the task processing result for the target task, the following steps may be included: obtaining multiple target sample data; inputting the multiple target sample data into an initial processing model to obtain target sample prediction results for the multiple target sample data; and training the initial processing model based on the target sample prediction results and the target sample results for the multiple target sample data to obtain a trained task processing model. Specifically, the task processing model training method can be referred to as knowledge editing. The target sample data carries real data labels, which are the target sample results. Since the target sample results are non-repetitive, the target sample data can be considered pure data and can serve as the processing target of the task processing model to guide the task processing model training process. Ideally, the task processing model trained using the target sample data will not exhibit repetitive hallucinations. It should be noted that the method for obtaining multiple target sample data can be to read a large amount of target sample data carrying target sample results from other data acquisition devices or databases. Alternatively, the method can be to receive a large amount of target sample data carrying target sample results input by a user. The method for obtaining multiple target sample data is selected based on actual circumstances and is not limited in this embodiment. In practical applications, when training an initial processing model based on the target sample prediction results and the target sample results of the multiple target sample data, a loss value can be calculated based on the target sample prediction results and the multiple target sample data. The model parameters of the initial processing model can be adjusted based on the loss value until the training process meets a preset stopping condition, thereby obtaining a trained task processing model. Various functions can be used to calculate the loss value, such as the cross-beam loss function, the L1 norm loss function, the maximum loss function, the mean square error loss function, the logarithmic loss function, etc., and the selection can be made based on actual circumstances and is not limited in this embodiment. The preset stopping conditions include, but are not limited to, the loss value being less than or equal to a preset threshold and the number of iterations reaching a preset number of iterations. The preset threshold and the preset number of iterations are selected based on actual circumstances and are not limited in this embodiment. In a possible implementation of the present disclosure, after the loss value is calculated, the loss value is compared with a preset threshold.Specifically, if the loss value is greater than a preset threshold, it indicates that the difference between the target sample prediction result and the target sample result is significant, indicating that the initial processing model's predictive ability for the target sample data is poor. In this case, the model parameters of the initial processing model can be adjusted, and the process returns to the step of inputting multiple target sample data into the initial processing model to obtain target sample prediction results for the multiple target sample data. The initial processing model is then trained until the loss value is less than or equal to the preset threshold, indicating that the difference between the target sample prediction result and the target sample result is small. This meets the preset stopping condition, resulting in a trained task processing model. In another possible implementation of the present disclosure, in addition to comparing the loss value with the preset threshold, the number of iterations can also be used to determine whether the current initial processing model has been trained. Specifically, if the loss value is greater than the preset threshold, the model parameters of the initial processing model are adjusted, and the process returns to the step of inputting multiple target sample data into the initial processing model to obtain target sample prediction results for the multiple target sample data. Training of the initial processing model continues until the preset number of iterations is reached, at which point iterations are stopped, resulting in a trained task processing model. Using the solution of the embodiments of the present disclosure, an initial processing model is trained based on the target sample prediction results and the target sample results of multiple target sample data. If a preset stopping condition is not met, the initial processing model is trained until the preset stopping condition is met, completing the training and obtaining a task processing model. By continuously adjusting the model parameters of the initial processing model, the resulting task processing model can be made more accurate. In an optional embodiment of the present disclosure, since model parameters affect the frequency of repetition hallucination, repetition is more likely to occur when using low randomness parameters than when using high randomness parameters. Taking the sampling temperature as an example, when the sampling temperature is lowered, the model will be more inclined to generate tokens with the highest probability. This may cause the model to be overly conservative and prone to falling into loops or repeating previously generated content, because the most likely sequence may sometimes repeat an existing pattern. Therefore, the search results of the randomness parameters of the task processing model and the output stability of the model can be weighed to select better model parameters and further reduce the model's repetition hallucination problem. That is, after the initial processing model is trained based on the target sample prediction results and the target sample results of multiple target sample data to obtain the trained task processing model, the following steps may also be included: performing parameter search for the task processing model within a preset parameter range, and adjusting the parameters of the task processing model based on the parameter search results to obtain the adjusted task processing model.Specifically, parameter search refers to an optimization method in the fields of machine learning and deep learning. Its goal is to find a set of optimal model parameters or hyperparameters that enable the model to achieve better performance on specific evaluation metrics. Parameter search is used to determine the optimal combination of model algorithm internal parameters (such as weights and biases in a neural network) and / or hyperparameters (such as learning rate, regularization strength, number of hidden layers, activation function, etc., which are variables set before model training to control the model structure and training process). It should be noted that in the embodiments of the present disclosure, the model randomness parameters (Top_P, Top_K, and Temperature) can be adjusted and searched within a preset parameter range. The Temperature parameter is used to adjust the randomness of the output. Increasing the Temperature parameter setting can make the generated results more random and innovative, while lowering the Temperature parameter can lead to more stable and repeatable results. The Top_K parameter limits the range of choices when the model predicts the next word, restricting the model to predicting the word from the top K most likely words. As the K value increases, the range of possible words widens, increasing the diversity of the results. Conversely, decreasing the K value narrows the range of possible words, making the generated results more inclined towards words with higher probabilities. The Top_P parameter defines the next word selected from a set of words when the cumulative probability reaches a given P value. A lower Top_P value makes the generated results more predictable and relevant, while a higher Top_P value increases the diversity and creativity of the results. The number of possible words in the above sampling method is dynamic. The preset parameter range, K, and P are set according to actual circumstances and are not limited in this embodiment. In practical applications, parameter search methods for task processing models include, but are not limited to: Grid Search: exhaustively enumerates all possible parameter combinations within the preset parameter range, trains and verifies the model performance one by one, and finds the optimal combination. Random Search: randomly selects points within the preset parameter range for sampling, which is more efficient than grid search. Bayesian Optimization: A method based on probability statistics that uses a surrogate model (such as a Gaussian process) to fit the relationship between parameters and model performance, thereby intelligently selecting the next parameter combination to be tested within a preset parameter range, aiming to minimize the number of experiments to find a better solution.Gradient-based Optimization: Differentiable parameters can be updated using gradient descent or other optimization algorithms to achieve automatic parameter adjustment. Using the solution of the disclosed embodiments, a parameter search is performed on the task processing model within a preset parameter range. The parameters of the task processing model are then adjusted based on the parameter search results to obtain the adjusted task processing model. By balancing the randomness of the task processing model's parameter search results with the model's output stability, optimal model parameters are selected, further mitigating the model's repeatability problem. In an optional embodiment of the present disclosure, after training the initial processing model based on the target sample prediction results and the target sample results of multiple target sample data to obtain a trained task processing model, the following steps may also be included: obtaining multiple repeated evaluation sample data; inputting the multiple repeated evaluation sample data into the task processing model to obtain a first evaluation result, and generating a first evaluation metric for the task processing model based on the first evaluation result; if the first evaluation metric does not meet the model metric condition, returning to the step of obtaining multiple target sample data until the first evaluation metric meets the model metric condition, thereby obtaining an optimized task processing model. Specifically, the repeated evaluation sample data is used to quantify the repetition illusion of the task processing model. The repeated evaluation sample data contains duplicate content in the repeated evaluation sample results, and therefore, the repeated evaluation sample data can be considered toxic data. Because duplicate evaluation sample data is toxic, the model's processing of duplicate evaluation sample data is prone to causing hallucinations of repetition. Therefore, the duplicate evaluation sample data is input into the task processing model. After obtaining the first evaluation result, duplicate content detection is performed on the first evaluation result to determine whether the duplication hallucination problem of the current task processing model has been alleviated. The first evaluation result refers to the prediction result of the duplicate evaluation sample data. The first evaluation metric is used to describe the content duplication of the first evaluation result, such as the duplication quality score and the first evaluation word frequency. Word frequency can be the number of times a word appears in the result, or it can be relative frequency, that is, the value obtained by dividing the number of word appearances by the total number of words in the result. Model metric conditions include, but are not limited to, the number of word appearances being less than a preset value and the word frequency being less than or equal to a preset frequency. The specific selection depends on the actual situation and is not limited in this embodiment. In actual applications, there are various ways to obtain multiple duplicate evaluation sample data, and the selection depends on the actual situation and is not limited in this embodiment. In one possible implementation of this disclosure, multiple duplicate evaluation sample data can be received from a front-end user.In another possible implementation of the present disclosure, multiple repeated evaluation sample data can be read from other data acquisition devices or databases. Furthermore, after generating a first evaluation metric for the task processing model based on the first evaluation results, if the first evaluation metric meets the model metric condition, it indicates that the repetition hallucination problem of the current task processing model has been effectively alleviated. In this case, training can be terminated and task processing can be performed using the current task processing model. If the first evaluation metric does not meet the model metric condition, it indicates that the repetition hallucination problem of the current task processing model still has a significant impact on model accuracy. In this case, the process can return to the step of acquiring multiple target sample data and train the current task processing model until the first evaluation metric meets the model metric condition, thereby obtaining an optimized task processing model. Applying the solution of the embodiments of the present disclosure, multiple repeated evaluation sample data are obtained; the multiple repeated evaluation sample data are input into a task processing model to obtain a first evaluation result, and a first evaluation metric for the task processing model is generated based on the first evaluation result. If the first evaluation metric does not meet the model metric condition, the process returns to the step of obtaining multiple target sample data until the first evaluation metric meets the model metric condition, thereby obtaining an optimized task processing model. By performing a repetition hallucination evaluation on the trained task processing model, the severity of the model's repetition hallucination can be effectively quantified, further ensuring the accuracy of the task processing model. In an optional embodiment of the present disclosure, the aforementioned acquisition of multiple target sample data may include the following steps: obtaining multiple sample data; performing duplicate content detection on each sample result of the multiple sample data to obtain duplicate content detection results; and, based on the duplicate content detection results, filtering the multiple target sample data from the multiple sample data to select a plurality of target sample data whose sample results do not contain duplicate content. Specifically, duplicate content detection can be understood as repetition hallucination detection, which is used to determine whether the sample results contain duplicate content (e.g., words, characters, or phrases). The duplicate content detection result is used to describe whether the sample results contain duplicate content, such as duplicate content or non-duplicate content. In practical applications, there are various ways to obtain multiple sample data, and the method selected depends on the actual situation. This disclosure does not impose any restrictions on this method. In one possible implementation of this disclosure, multiple sample data can be received from a front-end user. In another possible implementation of this disclosure, multiple sample data can be read from other data acquisition devices or databases. For example, a race model can be used to collect a batch of model generation data, where the data includes instructions.

[0002] The model generates question-answer pairs that conform to human conversational habits, and the model-generated data is determined as sample data. The solution of the embodiments of the present disclosure is applied to obtain multiple sample data sets; perform duplicate content detection on the sample results of each of the multiple sample data sets to obtain duplicate content detection results; and, based on the duplicate content detection results, filter out multiple target sample data sets from the multiple sample data sets whose sample results contain non-duplicate content. By filtering out multiple target sample data sets from the sample data sets whose sample results contain non-duplicate content, the duplication illusion problem of the task processing model is alleviated. In an optional embodiment of the present disclosure, performing duplicate content detection on the sample results of the multiple sample data to obtain duplicate content detection results may include the following steps: performing duplicate content detection on the sample results of the multiple sample data according to a word frequency screening strategy to obtain duplicate content detection results, wherein the word frequency screening strategy is used to perform duplicate content detection on the sample results based on the frequency of occurrence of words in the sample results; and / or inputting detection prompt information and the sample results of the multiple sample data into a duplicate content detection model to obtain duplicate content detection results. Specifically, the duplicate content detection model may be a pre-trained large model or a deep learning model trained based on multiple training samples and the training results corresponding to the multiple training samples. Duplicate content detection models include, but are not limited to, CNN models, RNN models, LSTM models, Transformer models, and BERT models. It should be noted that during duplicate content detection, a word frequency screening strategy can be used to check the output formats of multiple sample data samples for errors that do not conform to human conventions. This strategy primarily uses rules based on common characteristics of duplicate data. For example, data with a high proportion of commas, a high total number of commas, or a high proportion of numbers in the data can be identified as duplicate content. This strategy can be used to screen out some severely repetitive sample data. Furthermore, because problems such as verbose language and missing information can lead to model hallucinations, quality scoring can be performed based on the duplicate content detection model to filter out duplicate sample data. Specifically, by specifying low-quality data types and descriptions in the detection prompt, such as verbose language, missing information, and irrelevant answers, which can lead to model hallucinations, this can prevent model hallucinations caused by overfitting of low-quality data during training. In practical applications, low-quality duplicate sample data often exhibits three characteristics.First, it contains a lot of offensive, negative, or sensitive content; second, it contains significant semantic repetition; and third, it is wordy, using numerous sentences to convey the same idea, resulting in very low information content. Therefore, based on these three characteristics, the present embodiment has designed a scoring detection prompt for each item, as shown below:

[0003] As a rigorous data screener, please follow the following guidelines when scoring the quality of model training data. When scoring, first determine if the following errors exist. If any of the following occur, assign a score of 0: 1. An offensive answer; 2. Unhealthy content; 3. Content related to sensitive topics. If none of the above errors exist, score each item according to the following criteria: 1. Repetition (0 or 1): The answer is considered to contain semantic repetitions. If not, assign 1 point; if so, assign 0. 2. Wordiness (0 or 1): The answer is considered to contain repeated sentences, meaning multiple sentences expressing the same meaning. If not, assign 1 point; if so, assign 0. Each item is scored according to the above distribution and scoring criteria, and the final score is summed. Please score carefully and ensure strict adherence to the scoring criteria to avoid incorrect answers being assigned a score other than 0. Question [Sample Data]; Answer [Sample Results]; Scoring [Duplicate Content Detection Results]. Please use JSON (JavaScript Object Notation) format to score answers. This format contains two fields: a rate value (rate) and a reason (reason). If the score is 0, provide the reason for the error; otherwise, provide the score and reason for each item. Scoring example: { "rate": score value, "reason": "If the score is 0, provide the reason for the error; otherwise, provide the score and reason for each item"}". JSON is a lightweight data exchange format that is easy for humans to read and write, as well as for machines to parse and generate. It uses a completely language-independent text format to store and represent data, enabling efficient data exchange between different programming languages. It's worth noting that the above detection prompt information has three major advantages: First, it prevents inappropriate data from contaminating the model: By directly scoring answers containing inappropriate content with a score of 0, this data is prevented from contaminating the training set, which is crucial for ensuring model health. Second, the scoring system is simplified: the scoring criteria are binary (0 or 1), which reduces random judgments in the model during annotation, improving the consistency of the scoring criteria and annotation efficiency. Third, the output is traceable and easy to parse: The scoring model needs to record scores and reasons in JSON format. This helps maintain consistency and facilitates subsequent review and analysis. Using JSON format also facilitates automated parsing.Applying the solution of an embodiment of the present disclosure, duplicate content detection is performed on sample results of multiple sample data using a word frequency screening strategy to obtain duplicate content detection results; and / or, detection prompt information and the sample results of multiple sample data are input into a duplicate content detection model to obtain duplicate content detection results. By detecting duplicate sample data that is likely to induce the illusion of duplicates, and using this method to selectively separate the sample data into "toxic" duplicate sample data and "non-toxic" target sample data, a task processing model is trained using the target sample data, alleviating the problem of the task processing model's illusion of duplicates. Referring to Figure 4, which is a flowchart of duplicate content detection in a task processing method provided by an embodiment of the present disclosure, the duplicate content detection process consists of two parts: screening based on a word frequency screening strategy and screening based on a model's quality score. The execution flow is as follows: Screening based on a word frequency screening strategy: The sample data is screened using a word frequency screening strategy. This strategy performs rule screening based on common characteristics of duplicate data. For example, data containing a high proportion of commas, a high total number of commas, or a high proportion of numbers in the data is identified as duplicate content. The word frequency filtering strategy can roughly screen out some severely duplicated sample data. Model-based quality scoring screening: Detection prompt information is obtained, and the detection prompt information and the sample results of the sample data remaining after filtering using the word frequency filtering strategy are input into the duplicate content detection model for scoring, thereby filtering out duplicate sample data with lower scores. It should be noted that by first filtering using the word frequency filtering strategy to obtain a rough screening result, and then filtering the remaining sample data based on the model quality score, the data filtered out by the word frequency filtering strategy and the model quality score is determined as duplicate sample data. The data that is not filtered out is the target sample data, which is then used in the training process of the task processing model. Furthermore, the target sample data can be deduplicated to reduce errors and duplicate information in the target sample data. In an optional embodiment of the present disclosure, after performing duplicate content detection on the sample results of the plurality of sample data and obtaining the duplicate content detection results, the following steps may be further included: screening, based on the duplicate content detection results, multiple duplicate sample data having duplicate sample result content from the plurality of sample data; and training an initial processing model based on the multiple duplicate sample data to obtain a trained duplicate processing model. It should be noted that duplicate sample data refers to data in the sample data having duplicate sample result content. Since the sample results of the duplicate sample data have duplicate content, the duplicate sample data can be considered toxic data.Because repeated evaluation sample data is toxic, the model's processing of repeated sample data is prone to hallucinations of repetition. This phenomenon is further exacerbated by training the initial processing model based on multiple repeated sample data to obtain a trained repeated processing model. In practical applications, the implementation of "training the initial processing model based on multiple repeated sample data to obtain a trained repeated processing model" is similar to the aforementioned implementation of "inputting multiple target sample data into the initial processing model to obtain target sample prediction results for the multiple target sample data; training the initial processing model based on the target sample prediction results and the target sample results for the multiple target sample data to obtain a trained task processing model." This embodiment of the present disclosure will not be further described. Using the solution of the present embodiment, multiple repeated sample data with duplicate sample result content are screened out from multiple sample data based on the repeated content detection results; and the initial processing model is trained based on the multiple repeated sample data to obtain a trained repeated processing model. By using toxic data for training, a repetition processing model with exacerbated repetition hallucination is generated. The performance of the repetition processing model can then be compared with that of the task processing model, visually demonstrating the severity of the repetition hallucination in the task processing model. In an optional embodiment of the present disclosure, after screening multiple duplicate sample data with duplicate sample result content from multiple sample data based on the duplicate content detection results, the following steps may also be included: dividing the multiple duplicate sample data to obtain multiple duplicate evaluation sample data and multiple duplicate training sample data; and training an initial processing model based on the multiple duplicate sample data to obtain a trained repetition processing model. This may include the following steps: training the initial processing model based on the multiple duplicate training sample data to obtain a trained repetition processing model. It should be noted that to further quantify the severity of the repetition hallucination in the repetition processing model, the duplicate sample data may be divided into multiple duplicate evaluation sample data and multiple duplicate training sample data, strictly ensuring that duplicate training sample data does not appear in the duplicate evaluation sample data, thereby making the hallucination quantification process more accurate. In practical applications, there are various ways to divide multiple repeated sample data to obtain multiple repeated evaluation sample data and multiple repeated training sample data. The specific method selected depends on the actual situation and is not limited in the present embodiment. In one possible implementation of the present disclosure, a preset number of repeated sample data can be randomly selected from the multiple repeated sample data as repeated evaluation sample data, and the unselected repeated sample data can be determined as repeated training sample data.In another possible implementation of the present disclosure, a preset number of duplicate sample data with significant content repetition can be used as duplicate evaluation sample data, and unselected duplicate sample data can be determined as duplicate training sample data. Applying the solution of an embodiment of the present disclosure, multiple duplicate sample data are divided to obtain multiple duplicate evaluation sample data and multiple duplicate training sample data. An initial processing model is trained based on the multiple duplicate training sample data to obtain a trained duplicate processing model. By strictly ensuring that the duplicate training sample data and the duplicate evaluation sample data do not overlap, the hallucination quantification process is more accurate. In an optional embodiment of the present disclosure, after training the initial processing model based on the multiple duplicate training sample data to obtain the trained duplicate processing model, the following steps may also be included: inputting the multiple duplicate evaluation sample data into the duplicate processing model to obtain a second evaluation result, and generating a second evaluation metric for the duplicate processing model based on the second evaluation result; and comparing the first evaluation metric and the second evaluation metric of the task processing model to obtain an indicator comparison result, wherein the indicator comparison result is used to describe the processing capability of the task processing model. It should be noted that the implementation of "inputting multiple repeated evaluation sample data into the repetition processing model to obtain a second evaluation result, and generating a second evaluation metric for the repetition processing model based on the second evaluation result" is similar to the implementation of "inputting multiple repeated evaluation sample data into the task processing model to obtain a first evaluation result, and generating a first evaluation metric for the task processing model based on the first evaluation result," and will not be further described in this embodiment of the disclosure. In actual applications, after determining the first evaluation metric for the task processing model and the second evaluation metric for the repetition processing model, the first evaluation metric and the second evaluation metric for the repetition processing model can be compared. Based on the metric comparison result, the severity of the repetition hallucination of the task processing model compared to the repetition processing model can be determined, thereby measuring the processing capability of the task processing model. Furthermore, when comparing the first evaluation indicator and the second evaluation indicator, multiple repeated evaluation sample data can be additionally input into the initial processing model to obtain a third evaluation result. Based on the third evaluation result, a third evaluation indicator of the initial processing model is generated. The first evaluation indicator, the second evaluation indicator, and the third evaluation indicator are compared to obtain an indicator comparison result. Based on the indicator comparison result, the severity of the repetition hallucination of the task processing model compared with the initial processing model and the repeated processing model is determined, thereby measuring and determining the processing capability of the task processing model.For example, assuming the first evaluation index of the task processing model is 68, the second evaluation index of the repetition processing model is 37.89, and the third evaluation index of the initial processing model is 60.86, this indicates that the task processing model's ability to address the repetition illusion problem reaches 68 points, the initial processing model's ability to address the repetition illusion problem reaches 60.86 points, and the repetition processing model's ability to address the repetition illusion problem reaches 37.89 points. This indicates that the repetition illusion problem is exacerbated in the repetition processing model compared to the initial processing model, while the repetition illusion problem is alleviated in the task processing model compared to the initial processing model. Therefore, the repetition illusion problem of the task processing model is effectively alleviated, and the accuracy of the task processing model is improved. Using the solution of the embodiments of the present disclosure, multiple repetition evaluation sample data are input into the repetition processing model to obtain a second evaluation result. Based on the second evaluation result, a second evaluation index for the repetition processing model is generated. The first evaluation index and the second evaluation index of the task processing model are compared to obtain an index comparison result. The index comparison result is used to describe the processing capability of the task processing model. By comparing the metrics and determining the severity of the repetition hallucination of the task processing model compared to the repetition processing model, the processing capabilities of the task processing model can be intuitively determined. Referring to Figure 5 , which is a flowchart of the processing process of a task processing method provided by one embodiment of the present disclosure, this method provides a detailed discussion of resolving the repetition hallucination problem in the model, focusing on aspects such as data sources, training strategies, and model hyperparameter settings. Specifically, the method includes: First, duplicate sample data is separated from sample data through duplicate content detection, and target sample data is deduplicated to reduce errors and duplicate information in the target sample data.To further quantify the severity of the model's repetition hallucination, the repeated sample data is divided into repeated evaluation sample data and repeated training sample data, strictly ensuring that the repeated evaluation sample data does not appear in the repeated training sample data. Secondly, the target sample data is used to edit the knowledge of the initial processing model, and the parameters in the initial processing model are fine-tuned through training to obtain a task processing model that effectively alleviates the repetition hallucination problem. At the same time, the repeated training sample data is used to edit the knowledge of the initial processing model, and the parameters in the initial processing model are fine-tuned through training to obtain a repetition processing model with an exacerbated repetition hallucination problem. Next, the repeated evaluation sample data is used to quantitatively calculate the repetition hallucination of the task processing model and the repetition processing model, respectively, to obtain a first evaluation index for the task processing model and a second evaluation index for the repetition processing model. The first and second evaluation indexes are compared to determine the severity of the repetition hallucination of the task processing model and the repetition processing model. Finally, to further reduce the repetition hallucination problem of the task processing model, a parameter search can be performed on the task processing model within a preset parameter range. The parameters of the task processing model are adjusted based on the parameter search results to obtain the adjusted task processing model. By applying the solution of an embodiment of the present disclosure, toxic data is separated from sample data to obtain pure target sample data. The target sample data is then used in a supervised fine-tuning approach to correct knowledge bias in the initial processing model, causing the model to output non-repetitive content, thereby alleviating the model's hallucination of repetition. This effectively reduces the model's hallucination of repetition without relying on external capabilities or compromising the model's inherent conversational capabilities. The following, combined with Figure 6, further illustrates the task processing method provided by the present disclosure using its application in a text processing scenario as an example. Figure 6 is a flowchart of a text processing method provided by an embodiment of the present disclosure, specifically comprising the following steps: Step 602: Obtaining a target text task's to-be-processed text. Step 604: Inputting the target text into the task processing model to obtain a text processing result for the target text task. The task processing model is trained based on multiple target sample data and target sample results for the multiple target sample data. The target sample data is screened based on duplicate content detection results from the multiple sample data, and the target sample results contain no duplicate content. It should be noted that the implementation of steps 602 to 604 is the same as the implementation of steps 302 to 304, and will not be described in detail in this embodiment of the present disclosure.Applying the solution of the embodiments of the present disclosure, since the target sample data for training the task processing model is obtained based on the duplicate content detection results of multiple sample data, the target sample results do not contain duplicate content. Therefore, without compromising the task processing model's own processing capabilities and without relying on external knowledge, the problem of duplicate hallucination in the task processing model is effectively alleviated, the task processing model's precision is improved, and the accuracy and completeness of the text processing results are further improved. In an optional embodiment of the present disclosure, after inputting the to-be-processed text into the task processing model and obtaining the text processing results for the target text task, the following steps may also be included: sending the text processing results to a front-end user; receiving result feedback information from the front-end user, wherein the result feedback information is information providing feedback on the text processing results based on the task information of the target text task; and sending the result feedback information to a cloud training platform, wherein the cloud training platform is configured to adjust the parameters of the task processing model using the result feedback information. In actual applications, after receiving the result feedback information from the front-end user, the result feedback information may be sent to the cloud training platform, and the cloud training platform may adjust the parameters of the task processing model based on the result feedback information. The cloud training platform's approach to adjusting the parameters of the task processing model based on result feedback information can be referenced in the aforementioned implementation of "building model optimization data based on result feedback information; and adjusting the parameters of the task processing model using the model optimization data." This embodiment of the present disclosure will not be further described. By utilizing the solution of the present embodiment, the cloud training platform adjusts the parameters of the task processing model based on result feedback information. This ensures task processing model accuracy and training efficiency while reducing system deployment and maintenance costs, providing users with convenient and efficient model training services. The following, with reference to FIG7 , further illustrates the task processing method provided by the present disclosure, using its application in an automatic question-answering scenario as an example. FIG7 is a flowchart of an automatic question-answering method provided by one embodiment of the present disclosure, specifically comprising the following steps: Step 702: Obtaining pending questions for the target question-answering task. Step 704: Input the question to be processed into the task processing model to obtain an answer result for the target question-answering task. The task processing model is trained based on multiple target sample data and target sample results for the multiple target sample data. The target sample data is obtained by screening the duplicate content detection results for the multiple sample data, and the target sample results contain no duplicate content. It should be noted that the implementation of steps 702 to 704 is the same as that of steps 302 to 304, and will not be further described in this embodiment.Applying the solution of the embodiments of the present disclosure, since the target sample data for training the task processing model is obtained based on the duplicate content detection results of multiple sample data, the target sample results do not have duplicate content. Therefore, without compromising the task processing model's own processing capabilities and without relying on external knowledge, the problem of duplicate hallucination in the task processing model is effectively alleviated, the task processing model's accuracy is improved, and the accuracy and completeness of the response results are further improved. In an optional embodiment of the present disclosure, before obtaining the pending questions for the target question-and-answer task, the following steps may also be included: sending a question-and-answer prompt message to the front-end user, wherein the question-and-answer prompt message is used to guide the front-end user to submit the pending questions for the target question-and-answer task; and receiving the pending questions for the target question-and-answer task sent by the front-end user based on the question-and-answer prompt message. It should be noted that the question-and-answer prompt message can be sent to the front-end user before the automatic question-and-answer process begins. The front-end user can use the question-and-answer prompt message to understand the automatic question-and-answer process or related question-and-answer products, thereby entering a pending question that is more relevant to their actual needs. For example, a Q&A prompt message could read, "Hello, I'm your automated Q&A assistant. I'm ready to answer all your questions. You can directly enter the question you want to answer." Furthermore, after sending the Q&A prompt message to the front-end user, the system can receive the pending question for the target Q&A task sent by the front-end user based on the Q&A prompt message. The pending question is then input into the task processing model to obtain a response for the target Q&A task. Using the solution of an embodiment of the present disclosure, a Q&A prompt message is sent to the front-end user; and the pending question for the target Q&A task sent by the front-end user based on the Q&A prompt message is received. The Q&A prompt message guides the front-end user to enter the pending question to be answered, thereby enhancing human-computer interaction and ensuring that the pending question accurately reflects the user's actual needs. Referring to FIG8 , FIG8 is a flowchart of a task processing model training method provided by one embodiment of the present disclosure, specifically comprising the following steps: Step 802: Obtain multiple target sample data, wherein the target sample data is obtained by screening based on duplicate content detection results from multiple sample data. Step 804: Input the multiple target sample data into the initial processing model to obtain target sample prediction results for the multiple target sample data. Step 806: Train the initial processing model based on the target sample prediction results and the target sample results for the multiple target sample data to obtain a trained task processing model. The target sample results are not repeated. It should be noted that the implementation of steps 802 through 806 is the same as the training method for the task processing model in the embodiment shown in FIG. 3 , and will not be further described in this embodiment.Applying the solution of the embodiments of the present disclosure, since the target sample data for training the task processing model is obtained by screening the duplicate content detection results of multiple sample data, the target sample results are non-duplicate. Therefore, without compromising the task processing model's own processing capabilities and without relying on external knowledge, the problem of duplicate hallucination in the task processing model is effectively alleviated, thereby improving the accuracy of the task processing model. Referring to Figure 9, Figure 9 is a flowchart of an information processing method based on a task processing model, provided by one embodiment of the present disclosure. This method is applied to a cloud training platform and specifically includes the following steps: Step 902: Receive a task generation request sent by a terminal device, where the task generation request includes request information. Specifically, the cloud training platform is a cloud computing-based infrastructure that provides large-scale data processing and high-performance computing resources for training, optimizing, and deploying various machine learning models, particularly deep learning models. On the cloud training platform, users can upload data, select or customize algorithm models, and efficiently perform model training and verification through distributed computing capabilities. The cloud training platform receives the task generation request from the terminal device, obtains the corresponding task processing model based on the request information, and generates task information based on the task processing model. The cloud training platform can quickly respond to the needs of different tasks, invoke appropriate processing models to perform task processing, and ultimately generate high-quality task processing results. A task generation request is a request instruction sent by a terminal device to the cloud training platform, requesting the cloud training platform to generate task information for the target task. A task generation request typically includes the data to be processed, the task type, the desired output format, and request information. For example, when a user selects the "Text Translation" function on the cloud training platform's front-end interface and uploads text, the cloud training platform constructs a task generation request containing information such as the text, the task type (i.e., text translation), and the language type to be translated. The request information refers to the parameters or descriptive information related to the target task carried in the task generation request. The request information guides the cloud training platform to correctly identify and execute the requested task information. The request information includes, but is not limited to, the target task scenario identifier, the task model identifier, or sample data for the target generation task. Step 904: Based on the request information, obtain a task processing model, wherein the task processing model is trained based on multiple target sample data and target sample results of the multiple target sample data, the target sample data is screened based on duplicate content detection results of the multiple sample data, and the target sample results have no duplicate content.In an optional embodiment of the present disclosure, obtaining a task processing model based on the request information may include the following steps: determining a target scenario template from multiple preset scenario templates based on the task scenario identifier, and searching a task processing model from a model library based on the target scenario template, wherein the model library stores multiple processing models and the request information includes the task scenario identifier of the target task; or searching a task processing model from the model library based on the task model identifier, wherein the request information includes the task model identifier of the target task. Specifically, a task scenario identifier refers to a unique or specific label used to distinguish different task application scenarios. In this embodiment of the present disclosure, the task scenario identifier is part of the request information. The cloud training platform may select a target scenario template that matches the request information from a series of preset scenario templates based on the task scenario identifier to generate task information. For example, if the task scenario identifier is "text translation," indicating that the terminal device wishes to translate the uploaded text, a text translation scenario template may be selected from multiple preset scenario templates based on the task scenario identifier. Preset scenario templates are standard configuration scenario templates predefined for different task application scenarios. Each template contains model information matching the task application scenario and information such as the task processing flow. The cloud training platform stores a series of preset scenario templates to quickly respond to task generation requests for different scenarios. Different preset scenario templates correspond to different task types, model information, and processing flows, ensuring that the pre-training platform can automatically obtain the model and process configuration that best suits the current request based on the task scenario identifier. For example, one of the preset scenario templates may be specifically designed for legal document processing, which contains the model information and processing flow of a pre-trained legal model. A target scenario template is a scenario template that matches the task scenario identifier. When parsing a task generation request, the cloud training platform locates the corresponding target scenario template based on the task scenario identifier and selects the corresponding task processing model and other related configuration information from the model library based on the model information included in the target scenario template. For example, if the task scenario identifier is "financial report analysis," the target scenario template contains the model information and related configuration parameters for the financial report analysis model. The model library is a centralized repository for deep learning models that have been trained and optimized to solve different processing tasks. On the cloud training platform, the model library stores a large number of processing models, including but not limited to text classification models, text translation models, and text analysis models.Furthermore, processing models in the model library can be divided into different versions based on their applicable scenarios. For example, the model library may contain multiple versions of text analysis models, such as text analysis models for financial data and text analysis models for social data. A task model identifier is a unique or specific label used to distinguish models applicable to different tasks. For example, the task model identifier could be "financial data." Based on this task model identifier, the model library can search for processing models suitable for processing financial data. For example, suppose a user selects the "text summarization extraction" function through the front-end interface of an e-commerce application. The application then sends a task generation request to the cloud training platform. The request includes the task scenario identifier "text summarization extraction." After receiving this task generation request, the cloud training platform identifies the task scenario identifier as "text summarization extraction." The cloud training platform then finds a target scenario template matching "text summarization extraction" from multiple preset scenario templates. This target scenario template is preconfigured with model information and processing flow suitable for the text summarization task. Based on the target scenario template, the cloud training platform retrieves a pre-trained text summarization model from the model library and loads the relevant parameters and configuration files. Furthermore, the cloud training platform can generate task information based on the information in the target scenario template. This task information includes, but is not limited to, details such as the model address, input data processing method, and output result specifications, so that the terminal device can correctly invoke the text summarization model and perform the text summarization task. Using the solution of the embodiments of the present disclosure, based on the task scenario identifier, a target scenario template is determined from multiple preset scenario templates. Based on the target scenario template, a task processing model is searched from a model library, which stores multiple processing models. Alternatively, based on the task model identifier, a task processing model is searched from the model library. By integrating cloud computing technology with predefined task scenario templates, task model identifiers, and model library resources, a flexible, efficient, and standardized task processing mechanism is implemented. In another optional embodiment of the present disclosure, in addition to selecting a pre-trained task processing model from a model library, a task processing model adapted to the needs of the end user can be trained specifically based on sample data provided by the end user. That is, the request information includes sample data of the target task; and obtaining the task processing model based on the request information can include the following steps: training an initial processing model corresponding to the target task based on the sample data to obtain a trained task processing model.It should be noted that the implementation of "training the initial processing model corresponding to the target task based on sample data to obtain a trained task processing model" is similar to the aforementioned task processing model training method and will not be further described in this embodiment. Applying the solution of this embodiment, the initial processing model corresponding to the target task is trained based on sample data to obtain a trained task processing model. This ensures that the task processing model better meets user needs and maintains the accuracy of the task processing model. Step 906: Generate task information based on the task processing model. The task information is used by the terminal device to execute the target task. Specifically, the task information is generated by the cloud training platform after parsing the received task generation request. The task information includes the model configuration and processing flow required to execute the target task. Based on the task information, the terminal device or other server-side components can correctly use the task processing model to process the target task. It should be noted that when generating task information based on the task processing model, the task processing model can be directly packaged to obtain the task information. Model information of the task processing model can also be obtained, and task information can be constructed based on the model information. This model information includes model parameter configuration, input data processing methods, expected output specifications, and any intermediate steps and other auxiliary information involved. For example, in a text summarization task, the task information may include the address of the selected text summarization model, the storage location of the input text, the target path for the output summary, and other parameters required for running the text summarization model. This information enables the terminal device to correctly load the text summarization model on a local or remote server and execute the text summarization task. Using the solution of the embodiments of the present disclosure, a task generation request is received from a terminal device, wherein the task generation request includes request information; a task processing model is obtained based on the request information; and task information is generated based on the task processing model. The task information is used by the terminal device to execute the target task. By using a cloud training platform that integrates cloud computing technology, task information is generated for the terminal device to execute the target task. This ensures the quality and efficiency of target task processing while reducing system deployment and operation and maintenance costs, providing users with convenient and efficient task information services.10 , which is a schematic structural diagram of a cloud training platform provided by an embodiment of the present disclosure. The cloud training platform includes a request interface 1002 and a response unit 1004. The request interface 1002 is configured to receive a task generation request sent by a terminal device, wherein the task generation request includes request information. The response unit 1004 is configured to obtain a task processing model based on the request information, wherein the task processing model is trained based on multiple target sample data and target sample results of the multiple target sample data, wherein the target sample data is obtained by screening based on duplicate content detection results of the multiple sample data, and the target sample results have non-duplicate content. Task information is generated based on the task processing model, wherein the task information is used for the terminal device to execute the target task. In an optional embodiment of the present disclosure, the cloud training platform further includes a model library, wherein the model library stores multiple processing models; a response unit is specifically configured to determine a target scenario template from multiple preset scenario templates based on a task scenario identifier, and search for a task processing model from the model library based on the target scenario template, wherein the model library stores multiple processing models, and the request information includes a task scenario identifier for the target task; or, search for a task processing model from the model library based on the task model identifier, wherein the request information includes the task model identifier for the target task. It should be noted that the implementation of "the response unit, based on the task scenario identifier, determines a target scenario template from multiple preset scenario templates, and based on the target scenario template, searches a model library for a task processing model, wherein the model library stores multiple processing models and the request information includes the task scenario identifier of the target task; or, based on the task model identifier, searches a model library for a task processing model, wherein the request information includes the task model identifier of the target task" can be referred to the implementation of "based on the task scenario identifier, determines a target scenario template from multiple preset scenario templates, and based on the target scenario template, searches a model library for a task processing model, wherein the model library stores multiple processing models and the request information includes the task scenario identifier of the target task; or, based on the task model identifier, searches a model library for a task processing model, wherein the request information includes the task model identifier of the target task" in the embodiment provided in FIG. This embodiment of the present disclosure will not be further described. By applying the solution of the embodiment of the present disclosure, the cloud training platform generates task information for the terminal device to perform the target task. This can reduce system deployment and operation and maintenance costs while ensuring the quality and efficiency of target task processing, thereby providing users with convenient and efficient task information services.Corresponding to the above-mentioned task processing method embodiments, the present disclosure also provides an embodiment of a task processing device. Figure 11 is a schematic diagram of the structure of a task processing device provided by one embodiment of the present disclosure. As shown in Figure 11, the device includes: a first acquisition module 1102, configured to acquire task data for a target task; a first input module 1104, configured to input the task data into a task processing model to obtain a task processing result for the target task. The task processing model is trained based on multiple target sample data and target sample results for the multiple target sample data. The target sample data is obtained by screening the multiple sample data based on duplicate content detection results, and the target sample results contain no duplicate content. Optionally, the device also includes: a first sending module, configured to send the task processing result to a front-end user; receive result feedback information sent by the front-end user, wherein the result feedback information is information providing feedback on the task processing result based on the task information of the target task; construct model optimization data based on the result feedback information; and use the model optimization data to adjust parameters of the task processing model. Optionally, the first sending module is further configured to generate optimization prompt information based on the result feedback information, wherein the optimization prompt information is used to guide the front-end user to send model optimization data for optimizing the task processing model; send the optimization prompt information to the front-end user, and receive the model optimization data sent by the front-end user based on the optimization prompt information. Optionally, the apparatus further includes: a labeling module configured to label key information in the task processing result to obtain an updated task processing result; and send the updated task processing result to the front-end user. Optionally, the apparatus further includes: a second sending module configured to send the task processing result to the front-end user; receive edit information sent by the front-end user, wherein the edit information is used to edit the task processing result; and edit the task processing result based on the edit information to obtain the edited task processing result. Optionally, the apparatus further includes: a second training module configured to obtain a plurality of target sample data; input the plurality of target sample data into an initial processing model to obtain target sample prediction results for the plurality of target sample data; and train the initial processing model based on the target sample prediction results and the target sample results for the plurality of target sample data to obtain a trained task processing model. Optionally, the apparatus further includes: a search module configured to perform a parameter search for the task processing model within a preset parameter range, and adjust parameters of the task processing model based on the parameter search results to obtain an adjusted task processing model.Optionally, the apparatus further includes: a first evaluation module configured to obtain a plurality of duplicate evaluation sample data; input the plurality of duplicate evaluation sample data into the task processing model to obtain a first evaluation result; and generate a first evaluation metric for the task processing model based on the first evaluation result; if the first evaluation metric does not meet the model metric condition, return to the step of obtaining a plurality of target sample data until the first evaluation metric meets the model metric condition, thereby obtaining an optimized task processing model. Optionally, a second training module is further configured to obtain a plurality of sample data; perform duplicate content detection on each sample result of the plurality of sample data to obtain duplicate content detection results; and, based on the duplicate content detection results, filter out, from the plurality of sample data, a plurality of target sample data having non-duplicate sample result content. Optionally, the second training module is further configured to perform duplicate content detection on the sample results of the multiple sample data based on a word frequency screening strategy to obtain duplicate content detection results, wherein the word frequency screening strategy is used to perform duplicate content detection on the sample results based on the frequency of occurrence of words in the sample results; and / or input the detection prompt information and the sample results of the multiple sample data into a duplicate content detection model to obtain duplicate content detection results. Optionally, the apparatus further includes: a third training module configured to screen out multiple duplicate sample data with duplicate sample results from the multiple sample data based on the duplicate content detection results; train an initial processing model based on the multiple duplicate sample data to obtain a trained duplicate processing model. Optionally, the apparatus further includes: a partitioning module configured to partition the multiple duplicate sample data to obtain multiple duplicate evaluation sample data and multiple duplicate training sample data; and the third training module is further configured to train the initial processing model based on the multiple duplicate training sample data to obtain a trained duplicate processing model. Optionally, the device also includes: a second evaluation module, configured to input multiple repeated evaluation sample data into the repeated processing model to obtain a second evaluation result, and generate a second evaluation indicator of the repeated processing model based on the second evaluation result; compare the first evaluation indicator and the second evaluation indicator of the task processing model to obtain an indicator comparison result, wherein the indicator comparison result is used to describe the processing capability of the task processing model.Applying the solution of the embodiments of this disclosure, since the target sample data for training the task processing model is obtained by screening the duplicate content detection results of multiple sample data, the target sample results are non-repetitive. Therefore, without compromising the task processing model's own processing capabilities and without relying on external knowledge, the problem of duplicate hallucination in the task processing model is effectively alleviated, the task processing model's precision is improved, and the accuracy and completeness of the task processing results are further enhanced. The above is a schematic diagram of a task processing device according to this embodiment. It should be noted that the technical solution of this task processing device and the technical solution of the aforementioned task processing method are based on the same concept. For details not described in detail in the technical solution of the task processing device, please refer to the description of the technical solution of the aforementioned task processing method. Corresponding to the above-mentioned text processing method embodiment, the present disclosure also provides an embodiment of a text processing device. Figure 12 is a schematic diagram of the structure of a text processing device provided by one embodiment of this disclosure. As shown in Figure 12, the apparatus includes: a second acquisition module 1202 configured to acquire text to be processed for a target text task; a second input module 1204 configured to input the text to be processed into a task processing model to obtain a text processing result for the target text task, wherein the task processing model is trained based on multiple target sample data and target sample results for the multiple target sample data, wherein the target sample data is obtained by screening the duplicate content detection results of the multiple sample data, and the target sample results contain no duplicate content. Optionally, the apparatus also includes: a third sending module configured to send the text processing result to a front-end user; receive result feedback information sent by the front-end user, wherein the result feedback information is information providing feedback on the text processing result based on the task information of the target text task; and send the result feedback information to a cloud training platform, wherein the cloud training platform uses the result feedback information to adjust parameters of the task processing model. Applying the solution of the disclosed embodiments, since the target sample data for training the task processing model is obtained based on the duplicate content detection results of multiple sample data, the target sample results do not contain duplicate content. Therefore, without compromising the task processing model's inherent processing capabilities and without relying on external knowledge, the problem of duplicate illusions in the task processing model is effectively mitigated, the task processing model's accuracy is improved, and the accuracy and completeness of the text processing results are further enhanced. The above is a schematic solution of a text processing device according to this embodiment. It should be noted that the technical solution of this text processing device and the technical solution of the aforementioned text processing method are based on the same concept. For details not described in detail in the technical solution of the text processing device, please refer to the description of the technical solution of the aforementioned text processing method.Corresponding to the above-mentioned automatic question-answering method embodiment, the present disclosure also provides an embodiment of an automatic question-answering device. Figure 13 is a schematic structural diagram of an automatic question-answering device provided by one embodiment of the present disclosure. As shown in Figure 13, the device includes: a third acquisition module 1302, configured to acquire pending questions for a target question-answering task; a third input module 1304, configured to input the pending questions into a task processing model to obtain a response result for the target question-answering task. The task processing model is trained based on multiple target sample data and target sample results for the multiple target sample data. The target sample data is obtained by screening the duplicate content detection results of the multiple sample data, and the target sample results contain no duplicate content. Optionally, the device also includes: a fourth sending module, configured to send a question-answering prompt message to a front-end user, wherein the question-answering prompt message is used to guide the front-end user to submit a pending question for the target question-answering task; and to receive the pending question for the target question-answering task sent by the front-end user based on the question-answering prompt message. Applying the solution of the embodiments of the present disclosure, since the target sample data for training the task processing model is obtained by screening the duplicate content detection results of multiple sample data, the target sample results are non-duplicate. Therefore, without compromising the task processing model's own processing capabilities and without relying on external knowledge, the problem of repetition hallucination in the task processing model is effectively alleviated, the task processing model's precision is improved, and the accuracy and completeness of the response results are further improved. The above is a schematic scheme of an automatic question-answering device according to this embodiment. It should be noted that the technical solution of this automatic question-answering device and the technical solution of the automatic question-answering method described above are based on the same concept. For details not described in detail in the technical solution of the automatic question-answering device, please refer to the description of the technical solution of the automatic question-answering method described above. Corresponding to the above-mentioned task processing model training method embodiment, the present disclosure also provides an embodiment of a task processing model training device. Figure 14 is a schematic structural diagram of a task processing model training device provided by one embodiment of the present disclosure. As shown in Figure 14, the device includes: a fourth acquisition module 1402, configured to acquire multiple target sample data, wherein the target sample data is obtained by screening based on the repeated content detection results of the multiple sample data; a fourth input module 1404, configured to input the multiple target sample data into the initial processing model to obtain target sample prediction results of the multiple target sample data; a first training module 1406, configured to train the initial processing model according to the target sample prediction results and the target sample results of the multiple target sample data to obtain a trained task processing model, wherein the target sample results have no repeated content.Applying the solution of the embodiments of the present disclosure, since the target sample data for training the task processing model is obtained by screening the duplicate content detection results of multiple sample data, the target sample results are non-duplicate. Therefore, without compromising the task processing model's own processing capabilities and without relying on external knowledge, the problem of duplicate illusions in the task processing model is effectively mitigated, thereby improving the accuracy of the task processing model. The above is a schematic diagram of a task processing model training device according to this embodiment. It should be noted that the technical solution of this task processing model training device and the technical solution of the task processing model training method described above are based on the same concept. For details not described in detail in the technical solution of the task processing model training device, please refer to the description of the technical solution of the task processing model training method described above. Corresponding to the above method embodiments, the present disclosure also provides an embodiment of an information processing device based on a task processing model. Figure 15 is a schematic diagram of the structure of an information processing device based on a task processing model according to one embodiment of the present disclosure. As shown in FIG15 , the apparatus is applied to a cloud training platform and includes: a first receiving module 1502 configured to receive a task generation request sent by a terminal device, wherein the task generation request includes request information; a fifth obtaining module 1504 configured to obtain a task processing model based on the request information, wherein the task processing model is trained based on multiple target sample data and target sample results of the multiple target sample data, wherein the target sample data is obtained by screening duplicate content detection results of the multiple sample data, and the target sample results have non-duplicate content; and a first generating module 1506 configured to generate task information based on the task processing model, wherein the task information is used for the terminal device to perform the target task. Optionally, the fifth obtaining module 1504 is further configured to determine a target scenario template from multiple preset scenario templates based on a task scenario identifier, and search a model library for a task processing model based on the target scenario template, wherein the model library stores multiple processing models, and the request information includes the task scenario identifier of the target task; or, based on the task model identifier, search the model library for a task processing model, wherein the request information includes the task model identifier of the target task. Optionally, the request information includes sample data of the target task; the fifth acquisition module 1504 is further configured to train the initial processing model corresponding to the target task based on the sample data to obtain a trained task processing model.Using the solution of an embodiment of the present disclosure, a task generation request is received from a terminal device, wherein the task generation request includes request information; a task processing model is obtained based on the request information; and task information is generated based on the task processing model, wherein the task information is used for the terminal device to perform the target task. By using a cloud training platform that integrates cloud computing technology, task information for the terminal device to perform the target task is generated. This ensures the quality and efficiency of target task processing while reducing system deployment and operation and maintenance costs, providing users with convenient and efficient task information services. The above is a schematic diagram of an information processing device based on a task processing model in this embodiment. It should be noted that the technical solution of the information processing device based on the task processing model and the technical solution of the information processing method based on the task processing model are based on the same concept. Details not described in detail in the technical solution of the information processing device based on the task processing model can be found in the description of the technical solution of the information processing method based on the task processing model. Figure 16 is a block diagram of the structure of a computing device provided by an embodiment of the present disclosure. Components of computing device 1600 include, but are not limited to, memory 1610 and processor 1620. oThe processor 1620 is connected to the memory 1610 via a bus 1630. A database 1650 is used to store data. The computing device 1600 also includes an access device 1640 that enables the computing device 1600 to communicate via one or more networks 1660. Examples of these networks include a public switched telephone network (PSTN), a local area network (LAN), a wide area network (WAN), a personal area network (PAN), or a combination of communication networks such as the Internet. The access device 1640 may include one or more of any type of wired or wireless network interface (e.g., a Network Interface Card (NIC)), such as an IEEE 802.11 Wireless Local Area Network (WLAN) wireless interface, a Worldwide Interoperability for Microwave Access (Wi-MAX) interface, an Ethernet interface, a Universal Serial Bus (USB) interface, a cellular network interface, a Bluetooth interface, a Near Field Communication (NFC) interface, and the like. In one embodiment of the present disclosure, the aforementioned components of the computing device 1600 and other components not shown in FIG. 16 may also be connected to each other, for example, via a bus. It should be understood that the computing device structure block diagram shown in FIG. 16 is for illustrative purposes only and does not limit the scope of the present disclosure. Those skilled in the art may add or replace other components as needed. Computing device 1600 can be any type of stationary or mobile computing device, including a mobile computer or mobile computing device (e.g., a tablet computer, a personal digital assistant, a laptop computer, a notebook computer, a netbook, etc.), a mobile phone (e.g., a smartphone), a wearable computing device (e.g., a smart watch, smart glasses, etc.), or other types of mobile devices, or a stationary computing device such as a desktop computer or a personal computer (PC, Persona 1 Computer).Computing device 1600 may also be a mobile or stationary server. Processor 1620 is configured to execute a computer program / instructions that, when executed by the processor, implement the steps of the aforementioned task processing method, text processing method, automatic question-answering method, task processing model training method, or information processing method based on a task processing model. The above is a schematic diagram of a computing device according to this embodiment. It should be noted that the technical solution of this computing device shares the same concept as the technical solutions of the aforementioned task processing method, text processing method, automatic question-answering method, task processing model training method, and information processing method based on a task processing model. For details not described in detail in the technical solution of the computing device, please refer to the description of the technical solution of the aforementioned task processing method, text processing method, automatic question-answering method, task processing model training method, or information processing method based on a task processing model. An embodiment of the present disclosure also provides a computer-readable storage medium storing a computer program / instructions that, when executed by the processor, implement the steps of the aforementioned task processing method, text processing method, automatic question-answering method, task processing model training method, or information processing method based on a task processing model. The above is a schematic diagram of a computer-readable storage medium according to this embodiment. It should be noted that the technical solution of this storage medium shares the same concept as the technical solutions of the aforementioned task processing method, text processing method, automatic question-answering method, task processing model training method, and information processing method based on a task processing model. For details not described in detail in the technical solution of the storage medium, reference can be made to the description of the technical solutions of the aforementioned task processing method, text processing method, automatic question-answering method, task processing model training method, or information processing method based on a task processing model. An embodiment of the present disclosure also provides a computer program product comprising a computer program / instructions that, when executed by a processor, implement the steps of the aforementioned task processing method, text processing method, automatic question-answering method, task processing model training method, or information processing method based on a task processing model. The above is a schematic diagram of a computer program product according to this embodiment.It should be noted that the technical solutions of this computer program product share the same concept as the technical solutions of the aforementioned task processing method, text processing method, automatic question-answering method, task processing model training method, and task processing model-based information processing method. For details not described in detail in the technical solutions of the computer program product, please refer to the description of the technical solutions of the aforementioned task processing method, text processing method, automatic question-answering method, task processing model training method, or task processing model-based information processing method. The above description describes specific embodiments of the present disclosure. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in an order different from that described in the embodiments and still achieve the desired results. Furthermore, the processes depicted in the accompanying drawings do not necessarily require the specific or sequential order shown to achieve the desired results. In certain embodiments, multi-tasking and parallel processing are also possible or may be advantageous. The computer instructions include computer program code, which may be in source code form, object code form, executable file, or some intermediate form. The computer-readable medium may include any entity or device capable of carrying the computer program code, a recording medium, a USB flash drive (Universal Serial Bus Flash Drive), a removable hard drive, a magnetic disk, an optical disk, a computer memory, a read-only memory (ROM), a random access memory (RAM), an electric carrier signal, a telecommunications signal, and a software distribution medium. It should be noted that the content included in the computer-readable medium may be appropriately increased or decreased based on the requirements of patent practice. For example, in some regions, according to patent practice, computer-readable media do not include electric carrier signals and telecommunications signals. It should be noted that, for ease of description, the aforementioned method embodiments are described as a series of actions. However, those skilled in the art should understand that the embodiments of the present disclosure are not limited by the order of the actions described, as certain steps may be performed in other orders or simultaneously according to the embodiments of the present disclosure. Secondly, those skilled in the art should also be aware that the embodiments described in this specification are preferred embodiments, and the actions and modules involved are not necessarily required for the embodiments of the present disclosure. In the above embodiments, the description of each embodiment has its own focus. For portions not described in detail in one embodiment, reference can be made to the relevant descriptions of other embodiments.The preferred embodiments disclosed above are merely intended to illustrate the present disclosure. The alternative embodiments do not exhaustively describe all details, nor do they limit the present invention to the specific embodiments described. Obviously, many modifications and variations are possible based on the content of the embodiments disclosed. These embodiments are selected and described in detail to better explain the principles and practical applications of the embodiments, thereby enabling those skilled in the art to better understand and utilize the present disclosure. The present disclosure is limited only by the claims and their full scope and equivalents.

Claims

Claims 1. A task processing method, comprising: Get the task data of the target task; The task data is input into a task processing model to obtain a task processing result of the target task, wherein the task processing model is trained based on a plurality of target sample data and target sample results of the plurality of target sample data, the target sample data is obtained by screening based on duplicate content detection results of the plurality of sample data, and the target sample results have no duplicate content.

2. The method according to claim 1, further comprising: Sending the task processing result to the front-end user; Receive result feedback information sent by the front-end user, wherein the result feedback information is information that provides feedback on the task processing result based on task information of the target task; construct model optimization data based on the result feedback information; and use the model optimization data to adjust parameters of the task processing model.

3. The method according to claim 2, wherein constructing model optimization data based on the result feedback information comprises: Generate optimization prompt information based on the result feedback information, wherein the optimization prompt information is used to guide the front-end user to send model optimization data for optimizing the task processing model; send the optimization prompt information to the front-end user, and receive the model optimization data sent by the front-end user based on the optimization prompt information.

4. The method according to claim 1, further comprising: Marking key information in the task processing result to obtain an updated task processing result; The updated task processing result is sent to the front-end user.

5. The method according to claim 1, further comprising: Sending the task processing result to the front-end user; receiving editing information sent by the front-end user, wherein the editing information is used to edit the task processing result; The task processing result is edited according to the editing information to obtain an edited task processing result.

6. The method according to any one of claims 1 to 5, further comprising: Acquire multiple target sample data; Inputting the plurality of target sample data into an initial processing model to obtain target sample prediction results of the plurality of target sample data; The initial processing model is trained according to the target sample prediction result and the target sample results of the multiple target sample data to obtain a trained task processing model.

7. The method according to claim 6, further comprising: training the initial processing model based on the target sample prediction result and the target sample results of the plurality of target sample data to obtain a trained task processing model; Within a preset parameter range, a parameter search is performed on the task processing model, and the parameters of the task processing model are adjusted according to the parameter search result to obtain an adjusted task processing model.

8. The method according to claim 6, further comprising: training the initial processing model based on the target sample prediction result and the target sample results of the plurality of target sample data to obtain a trained task processing model; Obtain multiple repeated evaluation sample data; 26 The multiple repeated evaluation sample data are input into the task processing model to obtain a first evaluation result, and a first evaluation indicator of the task processing model is generated based on the first evaluation result; if the first evaluation indicator does not meet the model indicator condition, the step of obtaining multiple target sample data is returned to and executed until the first evaluation indicator meets the model indicator condition, thereby obtaining an optimized task processing model.

9. The method according to claim 6, wherein obtaining a plurality of target sample data comprises: Get multiple sample data; Performing duplicate content detection on the sample results of the plurality of sample data respectively to obtain duplicate content detection results; According to the duplicate content detection result, a plurality of target sample data having non-duplicate sample result content are screened out from the plurality of sample data.

10. The method according to claim 9, wherein the performing duplicate content detection on each of the sample results of the plurality of sample data to obtain duplicate content detection results comprises: performing duplicate content detection on the sample results of the plurality of sample data according to a word frequency screening strategy to obtain duplicate content detection results, wherein the word frequency screening strategy is used to perform duplicate content detection on the sample results according to the frequency of occurrence of words in the sample results; and / or inputting detection prompt information and the sample results of the plurality of sample data into a duplicate content detection model to obtain the duplicate content detection results.

11. The method according to claim 9, further comprising: performing duplicate content detection on sample results of the plurality of sample data respectively, and obtaining duplicate content detection results; According to the duplicate content detection result, a plurality of duplicate sample data with duplicate sample result content are screened out from the plurality of sample data; and an initial processing model is trained based on the plurality of duplicate sample data to obtain a trained duplicate processing model.

12. The method according to claim 11, after filtering out a plurality of duplicate sample data having duplicate sample result content from the plurality of sample data according to the duplicate content detection result, further comprising: Dividing the plurality of repeated sample data to obtain a plurality of repeated evaluation sample data and a plurality of repeated training sample data; The training of the initial processing model according to the plurality of repeated sample data to obtain a trained repeated processing model includes: training the initial processing model according to the plurality of repeated training sample data to obtain a trained repeated processing model.

13. The method according to claim 12, further comprising: training the initial processing model according to the plurality of repeated training sample data to obtain a trained repeated processing model; Inputting the plurality of repeated evaluation sample data into the repeated processing model to obtain a second evaluation result, and generating a second evaluation indicator for the repeated processing model based on the second evaluation result; and comparing the first evaluation indicator and the second evaluation indicator of the task processing model to obtain an indicator comparison result, wherein the indicator comparison result is used to describe the processing capability of the task processing model.

14. A text processing method, comprising: Get the text to be processed for the target text task; The text to be processed is input into a task processing model to obtain a text processing result of the target text task, wherein the task processing model is trained based on a plurality of target sample data and target sample results of the plurality of target sample data, the target sample data is obtained by screening based on duplicate content detection results of the plurality of sample data, and the target sample results have no duplicate content.

15. The method according to claim 14, further comprising: after inputting the to-be-processed text into a task processing model to obtain a text processing result of the target text task; Sending the text processing result to the front-end user; Receive the result feedback information sent by the front-end user, wherein the result feedback information is based on the target The task information of the text marking task is used to provide feedback on the text processing result; and the result feedback information is sent to a cloud training platform, wherein the cloud training platform is used to adjust parameters of the task processing model using the result feedback information.

16. An automatic question-answering method, comprising: Obtain pending questions for a target question-answering task; input the pending questions into a task processing model to obtain a response result for the target question-answering task, wherein the task processing model is trained based on multiple target sample data and target sample results of the multiple target sample data, the target sample data is obtained by screening based on duplicate content detection results of multiple sample data, and the target sample result content is non-duplicate.

17. The method according to claim 16, before obtaining the pending questions for the target question-answering task, further comprising: Sending question and answer prompt information to a front-end user, wherein the question and answer prompt information is used to guide the front-end user to send pending questions for processing a target question and answer task; receiving the pending questions of the target question and answer task sent by the front-end user based on the question and answer prompt information.

18. A task processing model training method, comprising: Acquire a plurality of target sample data, wherein the target sample data is obtained by screening based on duplicate content detection results of the plurality of sample data; input the plurality of target sample data into an initial processing model to obtain target sample prediction results of the plurality of target sample data; train the initial processing model based on the target sample prediction results and the target sample results of the plurality of target sample data to obtain a trained task processing model, wherein the target sample results have no duplicate content.

19. An information processing method based on a task processing model, applied to a cloud training platform, comprising: Receive a task generation request sent by a terminal device, wherein the task generation request includes request information; obtain a task processing model based on the request information, wherein the task processing model is trained based on multiple target sample data and target sample results of the multiple target sample data, the target sample data is obtained by screening based on duplicate content detection results of multiple sample data, and the target sample result content is non-duplicate; generate task information based on the task processing model, wherein the task information is used for the terminal device to execute the target task.

20. The method according to claim 19, wherein acquiring a task processing model based on the request information comprises: Based on the task scenario identifier, a target scenario template is determined from multiple preset scenario templates, and based on the target scenario template, a task processing model is searched from a model library, wherein the model library stores multiple processing models, and the request information includes the task scenario identifier of the target task; or, based on the task model identifier, a task processing model is searched from the model library, wherein the request information includes the task model identifier of the target task.

21. The method according to claim 19, wherein the request information includes sample data of the target task; and the acquiring the task processing model based on the request information comprises: Based on the sample data, an initial processing model corresponding to the target task is trained to obtain a trained task processing model.

22. A cloud training platform, comprising a request interface and a response unit; the request interface is used to receive a task generation request sent by a terminal device, wherein: The task generation request includes request information; the response unit is used to obtain a task processing model based on the request information, wherein the task processing model is trained based on multiple target sample data and target sample results of the multiple target sample data, the target sample data is obtained by screening the duplicate content detection results of the multiple sample data, and the target sample data is obtained by filtering the duplicate content detection results of the multiple sample data. The result content is not repeated; based on the task processing model, generating task information, wherein the task information is used for the terminal device to perform the target task.

23. The cloud training platform according to claim 22, further comprising a model library, wherein: The model library stores multiple processing models; the response unit is specifically used to determine a target scenario template from multiple preset scenario templates based on a task scenario identifier, and search for a task processing model from the model library based on the target scenario template, wherein the model library stores multiple processing models, and the request information includes the task scenario identifier of the target task; or, based on the task model identifier, search for a task processing model from the model library, wherein the request information includes the task model identifier of the target task.

24. A computing device comprising: memory and processor; The memory is used to store computer programs / instructions, and the processor is used to execute the computer programs / instructions. When the computer program / instructions are executed by the processor, the steps of the method described in any one of claims 1 to 13, any one of claims 14 to 15, any one of claims 16 to 17, claim 18, or any one of claims 19 to 21 are implemented.

25. A computer-readable storage medium storing a computer program / instruction, which, when executed by a processor, implements the steps of the method described in any one of claims 1 to 13 or any one of claims 14 to 15 or any one of claims 16 to 17 or claim 18 or any one of claims 19 to 21.

26. A computer program product comprising a computer program / instructions which, when executed by a processor, implement the steps of the method of any one of claims 1 to 13 or any one of claims 14 to 15 or any one of claims 16 to 17 or claim 18 or any one of claims 19 to 21. 29

Images